Skip to content

The Factory System Architecture in aiNXT

Introduction: Why Do We Need This?

Imagine you're building a restaurant ordering system. You have: - A menu (configuration file) that lists dishes by name - Recipes (Python classes/functions) that know how to make each dish - A kitchen (Factory) that coordinates everything - Waiters (Loaders) who know where to find recipes - Translators (Parsers) who convert special requests into instructions the kitchen understands

The Factory system in aiNXT works exactly like this! It takes simple text configurations and turns them into complex Python objects like datasets, models, and metrics.

The Big Picture

Configuration File          Factory System              Python Objects
┌─────────────────┐        ┌─────────────┐            ┌──────────────┐
│ dataset:        │   →    │   Context   │       →    │  Dataset     │
│   name: csv     │        │      ↓      │            │  Instance    │
│   path: data.csv│        │   Factory   │            │              │
└─────────────────┘        │      ↓      │            └──────────────┘
                           │   Loader    │
                           │      ↓      │
                           │   Parser    │
                           └─────────────┘

Core Components Overview

1. Context - The Orchestrator

The Context (ainxt/scripts/context.py) is your main entry point. It holds all the factories you need and provides simple methods to create objects.

Think of it as your head waiter who knows: - Which kitchen (Factory) handles which type of order (models, datasets, etc.) - How to communicate special requests (Parsers) - Where everything is stored

2. Factory - The Kitchen

The Factory (ainxt/factory/factory.py) is where the magic happens. It: - Keeps a registry of all available "recipes" (constructors) - Can combine multiple kitchens (factories) together - Applies modifications (decorators) to your orders

3. Loader - The Recipe Finder

The Loader (ainxt/factory/loader.py) automatically discovers code in your project: - Scans Python modules for classes and functions - Filters them based on patterns and types - Registers them with the Factory

4. Parser - The Special Request Translator

Parsers transform configuration values into Python objects: - Convert strings like "adam" into actual optimizer objects - Handle nested configurations - Apply transformations before objects are created

How They Work Together

Let's follow a request through the system:

Step 1: You Write a Configuration

# config/models/classifier.yaml
task: classification
name: resnet
layers: 50
pretrained: true
optimizer:
  name: adam
  learning_rate: 0.001

Step 2: Context Reads the Configuration

from ainxt.scripts.context import Context
context = Context[Image](...)  # Your context with all factories
model = context.load_model(config)

Step 3: Parser Transforms Special Fields

The optimizer field gets intercepted by a Parser that knows how to create optimizer objects: 

# The parser sees optimizer: {name: adam, learning_rate: 0.001}
# And transforms it into: optimizer: AdamOptimizer(learning_rate=0.001)

Step 4: Factory Finds the Right Constructor

The Factory looks up (task="classification", name="resnet") and finds the ResNet constructor.

Step 5: Object Gets Created

The constructor is called with the parsed arguments, and you get your model!

Real-World Example: Vision Package

Let's look at how DigitalNXT.Vision sets this up:

1. Define Parsers (vision/parsers/)

# vision/parsers/augmenter.py
from ainxt import Factory

AUGMENTERS = Factory()
AUGMENTERS.register("classification", "flip", HorizontalFlip)
AUGMENTERS.register("classification", "rotate", RandomRotation)

2. Create Singletons (vision/serving/singletons.py)

# Combine core aiNXT factories with vision-specific ones
DATASETS = AINXT_DATASETS + create_dataset_factory("vision", Task)
MODELS = AINXT_MODELS + create_model_factory("vision", Task)
PARSERS = {**AINXT_PARSERS, **create_parsers("vision")}

3. Setup Context (context.py)

CONTEXT = Context[Image](
    encoder=VisionJSONEncoder(),
    decoder=VisionJSONDecoder(),
    dataset_builder=DATASETS,
    model_builder=MODELS,
    parsers=PARSERS
)

4. Use It!

# In your script
dataset = CONTEXT.load_dataset({
    "task": "classification",
    "name": "imagenet",
    "augment": {
        "flip": {"probability": 0.5},
        "rotate": {"degrees": 15}
    }
})

The Power of This System

  1. Separation of Concerns: Configuration is separate from implementation
  2. Extensibility: Easy to add new components without changing core code
  3. Reusability: Same configurations work across different projects
  4. Type Safety: The system validates that objects match expected types
  5. Discoverability: Loaders automatically find new components

Key Concepts for Python Beginners

The ** Operator

When you see **kwargs or **config, this "unpacks" a dictionary: 

config = {"name": "adam", "learning_rate": 0.001}
# These two lines do the same thing:
optimizer = create_optimizer(**config)
optimizer = create_optimizer(name="adam", learning_rate=0.001)

Task and Name Tuples

Factories use (task, name) pairs as keys: - task: The ML task type (classification, segmentation, etc.) or None for general - name: The specific implementation (resnet, vgg, adam, etc.)

Wildcards with None

Using None acts as a wildcard: - (None, "adam") matches any task with name "adam" - ("classification", None) matches any classifier

Next Steps

Visual Overview

The included diagrams show: - src/serving/singletons.py: How singletons (global factories) are created and registered - src/serving/serialization.py: How objects are serialized/deserialized - src/task.py: How tasks connect different components - context.py: How Context ties everything together

These visual guides help you understand the data flow and relationships between components.