Skip to content

Core Concept: Context

What is Context?

Context is the orchestrator that brings together all aiNXT components - Factories, Loaders, Parsers, Datasets, and Models - into a unified, easy-to-use interface. It's your main entry point for creating objects from configuration.

Real-world analogy: Think of Context as the conductor of an orchestra: - The Factories are different instrument sections (strings, brass, woodwinds) - The Parsers are the sheet music translators - The conductor (Context) coordinates everything, ensuring all sections work together harmoniously

Source: ainxt/scripts/context.py

Why Do We Need Context?

Without Context (Complex)

# You'd need to manage everything separately
from ainxt.serving import DATASETS, MODELS, PARSERS, parse_config

# Parse configuration
parsed_dataset_config = parse_config(dataset_config, PARSERS)
parsed_model_config = parse_config(model_config, PARSERS)

# Build objects
dataset = DATASETS.build(**parsed_dataset_config)
model = MODELS.build(**parsed_model_config)

# Pass parsers around everywhere...

With Context (Simple)

# Context manages everything for you
from myapp.context import CONTEXT

# Just load what you need
dataset = CONTEXT.load_dataset(dataset_config)
model = CONTEXT.load_model(model_config)

# Context automatically handles parsing, factory selection, etc.

Creating a Context

Basic Context Setup

from ainxt.scripts.context import Context
from ainxt.serving.serialization import ainxtJSONEncoder, ainxtJSONDecoder
from myapp.serving import DATASETS, MODELS, PARSERS

# Specify your data type
DataType = str  # Or Image, np.ndarray, etc.

# Create context
CONTEXT = Context[DataType](
    encoder=ainxtJSONEncoder(),
    decoder=ainxtJSONDecoder(),
    dataset_builder=DATASETS,
    model_builder=MODELS,
    parsers=PARSERS
)

Real-World Example: DigitalNXT.Vision

# File: DigitalNXT.Vision/context.py
from ainxt.scripts.context import Context
from vision.data import Image
from vision.serving import (
    VisionJSONEncoder,
    VisionJSONDecoder,
    DATASETS,
    MODELS,
    PARSERS,
    METRICS,
    VISUALIZATIONS
)

# Vision works with Image data type
CONTEXT = Context[Image](
    encoder=VisionJSONEncoder(),
    decoder=VisionJSONDecoder(),
    dataset_builder=DATASETS,
    model_builder=MODELS,
    metric_builder=METRICS,
    visualization_builder=VISUALIZATIONS,
    parsers=PARSERS
)

# Export for use in scripts
__all__ = ("CONTEXT",)

Now in any script: 

from context import CONTEXT

# Simple and clean!
dataset = CONTEXT.load_dataset(config["dataset"])
model = CONTEXT.load_model(config["model"])

Context Components

1. Encoders and Decoders

Handle serialization/deserialization of your data type:

# For text data
from ainxt.serving import ainxtJSONEncoder, ainxtJSONDecoder

context = Context[str](
    encoder=ainxtJSONEncoder(),  # Str → JSON
    decoder=ainxtJSONDecoder()   # JSON → Str
)

# For custom data types (like images)
context = Context[Image](
    encoder=VisionJSONEncoder(),  # Image → JSON
    decoder=VisionJSONDecoder()   # JSON → Image
)

2. Builders (Factories)

Connect your factories to the Context:

from myapp.serving import DATASETS, MODELS, METRICS

context = Context[DataType](
    ...,
    dataset_builder=DATASETS,      # For loading datasets
    model_builder=MODELS,          # For loading models
    metric_builder=METRICS,        # For loading metrics
    visualization_builder=VISUALS  # For loading visualizations
)

3. Parsers

Register parsers for configuration transformation:

from myapp.serving import PARSERS

context = Context[DataType](
    ...,
    parsers=PARSERS  # Transform config → objects
)

Using Context

Loading Datasets

# Configuration
dataset_config = {
    "task": "classification",
    "name": "imagenet",
    "path": "/data/imagenet",
    "augmentation": {
        "name": "flip",
        "probability": 0.5
    }
}

# Load dataset - Context handles parsing and creation
dataset = CONTEXT.load_dataset(dataset_config)

# Dataset is ready to use
for instance in dataset:
    print(instance)

What Context does internally: 1. Parse configuration with registered parsers 2. Transform augmentation dict into augmenter object 3. Use dataset_builder (Factory) to create dataset 4. Apply any decorators 5. Return the final dataset

Loading Models

# Configuration
model_config = {
    "task": "classification",
    "name": "resnet50",
    "num_classes": 1000,
    "pretrained": True
}

# Load model
model = CONTEXT.load_model(model_config)

# Use model
predictions = model(instance)

Loading Metrics

# Configuration
metrics_config = [
    {"name": "accuracy"},
    {"name": "precision", "average": "macro"},
    {"name": "recall", "average": "macro"}
]

# Load metrics
metrics = CONTEXT.load_metrics(metrics_config, task="classification")

# Use metrics
for metric in metrics:
    score = metric(predictions, ground_truth)
    print(f"{metric.name}: {score}")

Loading Visualizations

# Configuration
viz_config = [
    {"name": "confusion_matrix"},
    {"name": "roc_curve"}
]

# Load visualizations
visualizations = CONTEXT.load_visualizations(viz_config, task="classification")

# Create visualizations
for viz in visualizations:
    figure = viz(predictions, ground_truth)
    figure.savefig(f"{viz.name}.png")

Complete Example: Training Script

# train.py
import yaml
from context import CONTEXT

def main():
    # Load configuration
    with open("config/experiment.yaml") as f:
        config = yaml.safe_load(f)

    # Load all components through Context
    dataset = CONTEXT.load_dataset(config["dataset"])
    model = CONTEXT.load_model(config["model"])
    metrics = CONTEXT.load_metrics(config["metrics"], task=config["task"])

    # Parse training configuration
    from ainxt.serving import parse_config
    training_kwargs = parse_config(config["training"], CONTEXT.parsers)

    # Train model (optimizer, loss, etc. already parsed!)
    model.fit(dataset, **training_kwargs)

    # Evaluate
    predictions = model(test_dataset)
    for metric in metrics:
        score = metric(predictions, ground_truth)
        print(f"{metric.name}: {score:.3f}")

if __name__ == "__main__":
    main()

Configuration file: 

# config/experiment.yaml
task: classification

dataset:
  task: classification
  name: imagenet
  path: /data/imagenet
  augmentation:  # Parsed by Context
    name: standard
    flip: true
    rotate: 15

model:
  task: classification
  name: resnet50
  num_classes: 1000

training:
  epochs: 100
  batch_size: 32
  optimizer:  # Parsed by Context
    name: adam
    learning_rate: 0.001
  loss_function:  # Parsed by Context
    name: cross_entropy

metrics:
  - name: accuracy
  - name: f1_score
    average: macro

Context Benefits

1. Centralized Configuration

All factories and parsers in one place:

# Single source of truth
CONTEXT = Context[DataType](
    encoder=...,
    decoder=...,
    dataset_builder=DATASETS,
    model_builder=MODELS,
    parsers=PARSERS
)

# Use everywhere
from context import CONTEXT

2. Automatic Parsing

Context automatically applies parsers:

# Configuration with nested objects
config = {
    "name": "my_model",
    "optimizer": {  # Will be parsed automatically
        "name": "adam",
        "learning_rate": 0.001
    }
}

# Context handles parsing
model = CONTEXT.load_model(config)
# model.optimizer is already an Adam object!

3. Type Safety

Generic type parameter ensures consistency:

# Context knows it works with Images
CONTEXT = Context[Image](...)

# Type-safe loading
dataset: Dataset[Instance[Image]] = CONTEXT.load_dataset(config)
model: Model[Instance[Image]] = CONTEXT.load_model(config)

# IDE autocomplete and type checking work!

4. Simplified Scripts

Scripts become cleaner and more readable:

# Old way - verbose
from ainxt.serving import DATASETS, MODELS, PARSERS, parse_config
parsed_config = parse_config(config, PARSERS)
dataset = DATASETS.build(**parsed_config)

# With Context - clean
from context import CONTEXT
dataset = CONTEXT.load_dataset(config)

Multi-Package Context

When extending aiNXT (like DigitalNXT.Vision does), combine contexts:

# vision/serving/singletons.py
from ainxt.serving import (
    DATASETS as CORE_DATASETS,
    MODELS as CORE_MODELS,
    PARSERS as CORE_PARSERS
)

# Extend with vision-specific components
DATASETS = CORE_DATASETS + create_dataset_factory("vision", Task)
MODELS = CORE_MODELS + create_model_factory("vision", Task)
PARSERS = {**CORE_PARSERS, **create_parsers("vision")}

# vision/context.py
CONTEXT = Context[Image](
    encoder=VisionJSONEncoder(),
    decoder=VisionJSONDecoder(),
    dataset_builder=DATASETS,  # Has both core + vision datasets
    model_builder=MODELS,      # Has both core + vision models
    parsers=PARSERS            # Has both core + vision parsers
)

Now you have access to everything: - Core aiNXT datasets and models - Vision-specific datasets and models - All parsers from both packages

Default Values

Context provides sensible defaults if you don't specify builders:

# Minimal context
context = Context[str](
    encoder=ainxtJSONEncoder(),
    decoder=ainxtJSONDecoder()
)

# Automatically uses:
# - dataset_builder = ainxt.serving.DATASETS
# - model_builder = ainxt.serving.MODELS
# - metric_builder = ainxt.serving.METRICS
# - visualization_builder = ainxt.serving.VISUALIZATIONS
# - parsers = {} (empty)

This is handled in context.py:111-121:

def __post_init__(self):
    if self.dataset_builder is None:
        self.dataset_builder = DATASETS
    if self.model_builder is None:
        self.model_builder = MODELS
    # ... etc

Best Practices

1. One Context Per Package

# GOOD - single context for your package
# myapp/context.py
CONTEXT = Context[DataType](...)

# myapp/__init__.py
from myapp.context import CONTEXT
__all__ = ("CONTEXT",)

# Usage
from myapp import CONTEXT

2. Type Your Context

# GOOD - explicit data type
from myapp.data import CustomData

CONTEXT = Context[CustomData](...)

# BAD - no type information
CONTEXT = Context(...)  # What data type?

3. Export Context Properly

# GOOD - clear export
# context.py
CONTEXT = Context[Image](...)
__all__ = ("CONTEXT",)

# scripts can import directly
from context import CONTEXT

# BAD - hidden in serving
# serving/singletons.py
CONTEXT = Context[Image](...)  # Hard to find!

4. Use Context Methods

# GOOD - use Context methods
dataset = CONTEXT.load_dataset(config)

# AVOID - bypass Context
from myapp.serving import DATASETS, PARSERS, parse_config
parsed = parse_config(config, PARSERS)
dataset = DATASETS.build(**parsed)  # Verbose and error-prone

Troubleshooting

Issue 1: "Context has no attribute 'load_X'"

Problem: Typo or missing builder

Solution: Check Context initialization 

# Ensure all builders are set
CONTEXT = Context[DataType](
    ...,
    dataset_builder=DATASETS,  # Check this exists
    model_builder=MODELS,      # Check this exists
)

Issue 2: Parsers Not Working

Problem: Parsers not registered with Context

Solution: Pass parsers to Context 

# Make sure parsers are included
CONTEXT = Context[DataType](
    ...,
    parsers=PARSERS  # Don't forget this!
)

Issue 3: Type Mismatches

Problem: Context type doesn't match data

Solution: Ensure consistent types 

# BAD - mismatch
CONTEXT = Context[str](...)  # Says str
dataset: Dataset[Image] = CONTEXT.load_dataset(...)  # But loads Images

# GOOD - consistent
CONTEXT = Context[Image](...)
dataset: Dataset[Instance[Image]] = CONTEXT.load_dataset(...)

Summary

  • Context orchestrates Factories, Loaders, and Parsers
  • Simplifies object creation from configuration
  • Centralizes all aiNXT components in one place
  • Type-safe with generic data type parameter
  • Automatic parsing of nested configuration
  • Clean scripts with simple, readable code

Context is the glue that makes aiNXT's configuration-driven architecture practical and easy to use.

See Also