task_protocols

orchard.core.task_protocols

Task-Agnostic Strategy Protocols.

Defines the structural contracts that task-specific components must satisfy. Each protocol represents one dimension of task-specific behavior:

• Criterion: Loss function construction
• Validation metrics: Per-epoch metric computation
• Eval pipeline: Full evaluation orchestration (inference, visualization, reporting) as a single cohesive unit

Concrete implementations live under orchard.tasks.<task_type>/ and are registered in the :mod:orchard.core.task_registry.

TaskCriterionFactory

Bases: Protocol

Protocol for task-specific loss function construction.

get_criterion(training, class_weights=None)

Build a loss function from training config.

Parameters:

Name	Type	Description	Default
training	TrainingConfig	Training sub-config with criterion parameters.	required
class_weights	Tensor | None	Optional per-class weights for imbalanced datasets.	None

Returns:

Type	Description
Module	Configured loss module.

Source code in orchard/core/task_protocols.py

def get_criterion(
    self,
    training: TrainingConfig,
    class_weights: torch.Tensor | None = None,
) -> nn.Module:
    """
    Build a loss function from training config.

    Args:
        training: Training sub-config with criterion parameters.
        class_weights: Optional per-class weights for imbalanced datasets.

    Returns:
        Configured loss module.
    """
    ...  # pragma: no cover

TaskValidationMetrics

Bases: Protocol

Protocol for per-epoch validation metric computation.

compute_validation_metrics(model, val_loader, criterion, device)

Compute task-specific validation metrics.

Must return an immutable mapping with at least a "loss" key.

Parameters:

Name	Type	Description	Default
model	Module	Neural network model to evaluate.	required
val_loader	DataLoader[Any]	Validation data provider.	required
criterion	Module	Loss function.	required
device	device	Hardware target (CUDA/MPS/CPU).	required

Returns:

Type	Description
Mapping[str, float]	Immutable mapping of metric name to value.

Source code in orchard/core/task_protocols.py

def compute_validation_metrics(
    self,
    model: nn.Module,
    val_loader: DataLoader[Any],
    criterion: nn.Module,
    device: torch.device,
) -> Mapping[str, float]:
    """
    Compute task-specific validation metrics.

    Must return an immutable mapping with at least a ``"loss"`` key.

    Args:
        model: Neural network model to evaluate.
        val_loader: Validation data provider.
        criterion: Loss function.
        device: Hardware target (CUDA/MPS/CPU).

    Returns:
        Immutable mapping of metric name to value.
    """
    ...  # pragma: no cover

TaskEvalPipeline

Bases: Protocol

Protocol for end-to-end evaluation (inference + visualization + reporting).

run_evaluation(model, test_loader, train_losses, val_metrics_history, class_names, paths, training, dataset, augmentation, evaluation, arch_name, aug_info='N/A', tracker=None)

Execute the complete evaluation pipeline.

Coordinates inference, visualization, and structured reporting as a single task-specific unit.

Parameters:

Name	Type	Description	Default
model	Module	Trained model (already on target device).	required
test_loader	DataLoader[Any]	DataLoader for test set.	required
train_losses	list[float]	Training loss history per epoch.	required
val_metrics_history	list[Mapping[str, float]]	Validation metrics history per epoch.	required
class_names	list[str]	List of class label strings.	required
paths	RunPaths	RunPaths for artifact output.	required
training	TrainingConfig	Training sub-config.	required
dataset	DatasetConfig	Dataset sub-config.	required
augmentation	AugmentationConfig	Augmentation sub-config.	required
evaluation	EvaluationConfig	Evaluation sub-config.	required
arch_name	str	Architecture identifier.	required
aug_info	str	Augmentation description string.	'N/A'
tracker	TrackerProtocol | None	Optional experiment tracker for final metrics.	None

Returns:

Type	Description
tuple[float, float, float]	3-tuple of (macro_f1, test_acc, test_auc).

Source code in orchard/core/task_protocols.py

def run_evaluation(
    self,
    model: nn.Module,
    test_loader: DataLoader[Any],
    train_losses: list[float],
    val_metrics_history: list[Mapping[str, float]],
    class_names: list[str],
    paths: RunPaths,
    training: TrainingConfig,
    dataset: DatasetConfig,
    augmentation: AugmentationConfig,
    evaluation: EvaluationConfig,
    arch_name: str,
    aug_info: str = "N/A",
    tracker: TrackerProtocol | None = None,
) -> tuple[float, float, float]:
    """
    Execute the complete evaluation pipeline.

    Coordinates inference, visualization, and structured reporting
    as a single task-specific unit.

    Args:
        model: Trained model (already on target device).
        test_loader: DataLoader for test set.
        train_losses: Training loss history per epoch.
        val_metrics_history: Validation metrics history per epoch.
        class_names: List of class label strings.
        paths: RunPaths for artifact output.
        training: Training sub-config.
        dataset: Dataset sub-config.
        augmentation: Augmentation sub-config.
        evaluation: Evaluation sub-config.
        arch_name: Architecture identifier.
        aug_info: Augmentation description string.
        tracker: Optional experiment tracker for final metrics.

    Returns:
        3-tuple of (macro_f1, test_acc, test_auc).
    """
    ...  # pragma: no cover