evaluation_config

orchard.core.config.evaluation_config

Evaluation Reporting & Visualization Schema.

Defines post-training diagnostic phase requirements: visual artifacts (confusion matrices, prediction grids) and quantitative data persistence (Excel/JSON/CSV).

Key Features

• Aesthetic standardization: DPI, colormaps, plot styles for reproducible comparative analysis across experiments
• Diagnostic layouts: Configurable prediction grid geometry for inspecting model errors and blind spots
• Tabular export: Validated serialization formats compatible with downstream analysis tools
• Resource efficiency: Configurable output artifacts for memory optimization

Centralizes reporting parameters to ensure standardized, publication-quality diagnostic output for every experiment.

EvaluationConfig

Bases: BaseModel

Visual reporting and performance metric persistence configuration.

Controls inference settings, visualization aesthetics, and export formats for confusion matrices and prediction grids.

Attributes:

Name	Type	Description
n_samples	PositiveInt	Number of samples to display in prediction grid.
fig_dpi	PositiveInt	DPI resolution for saved figure files.
cmap_confusion	str	Matplotlib colormap for confusion matrix.
plot_style	str	Matplotlib style preset for all visualizations.
grid_cols	PositiveInt	Number of columns in prediction grid layout.
fig_size_predictions	tuple[PositiveInt, PositiveInt]	Figure dimensions (width, height) in inches.
report_format	str	Export format for metrics report (xlsx, csv, json).
save_confusion_matrix	bool	Whether to generate confusion matrix plot.
save_predictions_grid	bool	Whether to generate prediction samples grid.

validate_format(v) classmethod

Validate and normalize report format.

Parameters:

Name	Type	Description	Default
v	str	Requested report format string.	required

Returns:

Type	Description
str	Normalized format string (xlsx, csv, or json).

Raises:

Type	Description
OrchardConfigError	If format is not one of xlsx, csv, json.

Source code in orchard/core/config/evaluation_config.py

@field_validator("report_format")
@classmethod
def validate_format(cls, v: str) -> str:
    """
    Validate and normalize report format.

    Args:
        v: Requested report format string.

    Returns:
        Normalized format string (xlsx, csv, or json).

    Raises:
        OrchardConfigError: If format is not one of xlsx, csv, json.
    """
    supported = {"xlsx", "csv", "json"}
    normalized = v.lower()
    if normalized not in supported:
        raise OrchardConfigError(
            f"Unsupported report_format '{v}'. Choose from: {sorted(supported)}"
        )
    return normalized

validate_cmap(v) classmethod

Validate colormap name against matplotlib registry.

Parameters:

Name	Type	Description	Default
v	str	Requested colormap name.	required

Returns:

Type	Description
str	Validated colormap name.

Raises:

Type	Description
OrchardConfigError	If colormap is not registered in matplotlib.

Source code in orchard/core/config/evaluation_config.py

@field_validator("cmap_confusion")
@classmethod
def validate_cmap(cls, v: str) -> str:
    """
    Validate colormap name against matplotlib registry.

    Args:
        v: Requested colormap name.

    Returns:
        Validated colormap name.

    Raises:
        OrchardConfigError: If colormap is not registered in matplotlib.
    """
    from matplotlib import colormaps

    if v not in colormaps:
        raise OrchardConfigError(
            f"Unknown matplotlib colormap '{v}'. "
            f"See matplotlib.org/stable/gallery/color/colormap_reference.html"
        )
    return v

validate_style(v) classmethod

Validate plot style against matplotlib available styles.

Parameters:

Name	Type	Description	Default
v	str	Requested style name.	required

Returns:

Type	Description
str	Validated style name.

Raises:

Type	Description
OrchardConfigError	If style is not available in matplotlib.

Source code in orchard/core/config/evaluation_config.py

@field_validator("plot_style")
@classmethod
def validate_style(cls, v: str) -> str:
    """
    Validate plot style against matplotlib available styles.

    Args:
        v: Requested style name.

    Returns:
        Validated style name.

    Raises:
        OrchardConfigError: If style is not available in matplotlib.
    """
    from matplotlib.pyplot import style

    if v not in style.available:
        raise OrchardConfigError(
            f"Unknown matplotlib style '{v}'. Available: {sorted(style.available)}"
        )
    return v