colour-science/learning-munsell

Learning Munsell - Machine Learning for Munsell Color Conversions

A project implementing machine learning-based methods for bidirectional conversion between CIE xyY colourspace values and Munsell specifications.

Two Conversion Directions:

• from_xyY: CIE xyY to Munsell specification
• to_xyY: Munsell specification to CIE xyY

Project Overview

Objective

Provide 100-1000x speedup for batch Munsell conversions compared to colour-science routines while maintaining high accuracy.

Results

from_xyY (CIE xyY to Munsell) — 33 models evaluated on all 2,734 REAL Munsell colors:

Model	Delta-E	Speed (ms)
Colour Library (Baseline)	0.00	116.3
Multi-MLP + Multi-Error Predictor	0.51	0.061
Multi-ResNet + Multi-Error Predictor (Large Dataset)	0.52	0.096
Transformer + Error Predictor (Large Dataset)	0.52	0.163
Multi-MLP (Classification Code) + Code-Aware Multi-Error Predictor (Large Dataset)	0.53	0.118
Multi-Head + Multi-Error Predictor (Large Dataset)	0.53	0.046
Multi-MLP (Classification Code) + Code-Aware Multi-Error Predictor	0.53	0.052
Multi-MLP (Classification Code) + Multi-Error Predictor	0.53	0.050
MLP + Error Predictor	0.53	0.036
Multi-Head + Multi-Error Predictor	0.54	0.057
Multi-ResNet (Large Dataset)	0.56	0.047
Multi-Head (Large Dataset)	0.57	0.012
FT-Transformer	0.70	0.067
Unified MLP	0.71	0.074
Transformer (Large Dataset)	0.73	0.120
Mixture of Experts	0.74	0.021
Multi-MLP	0.91	0.027
Multi-MLP (Classification Code)	0.92	0.026
Multi-MLP (Classification Code) (Large Dataset)	0.89	0.066
MLP + Self-Attention	0.88	0.185
Deep + Wide	1.18	0.078
MLP (Base Only)	1.30	0.009
Multi-MLP (Hue Angle sin/cos)	1.78	0.021

• Best Accuracy: Multi-MLP + Multi-Error Predictor — Delta-E 0.51, 1,905x faster
• Fastest: MLP Base Only (0.009 ms/sample) — 13,365x faster than Colour library
• Best Balance: MLP + Error Predictor — 3,196x faster with Delta-E 0.53

to_xyY (Munsell to CIE xyY) — 6 models evaluated on all 2,734 REAL Munsell colors:

Model	Delta-E	Speed (ms)
Colour Library (Baseline)	0.00	1.40
Multi-MLP + Multi-Error Predictor	0.48	0.066
Multi-Head + Multi-Error Predictor	0.51	0.054
Simple MLP	0.52	0.002
Multi-MLP	0.57	0.028
Multi-Head	0.60	0.013
Multi-MLP + Error Predictor	0.61	0.060

• Best Accuracy: Multi-MLP + Multi-Error Predictor — Delta-E 0.48, 21x faster
• Fastest: Simple MLP (0.002 ms/sample) — 669x faster than Colour library

Approach

• 33 models tested for from_xyY (MLP, Multi-Head, Multi-MLP, Multi-ResNet, Transformers, FT-Transformer, Mixture of Experts)
• 6 models tested for to_xyY (Simple MLP, Multi-Head, Multi-MLP with error predictors)
• Two-stage models (base + error predictor) proved most effective
• Best model: Multi-MLP + Multi-Error Predictor with Delta-E 0.51
• Training data: ~1.4M samples from dense xyY grid with boundary refinement and forward Munsell sampling
• Deployment: ONNX format with ONNX Runtime

For detailed architecture comparisons, model benchmarks, training pipeline details, and experimental results, see docs/learning_munsell.md.

Installation

Dependencies (Runtime):

• numpy >= 2.0
• onnxruntime >= 1.16

Dependencies (Training):

• torch >= 2.0
• scikit-learn >= 1.3
• matplotlib >= 3.9
• mlflow >= 2.10
• optuna >= 3.0
• colour-science >= 0.4.7
• click >= 8.0
• onnx >= 1.15
• onnxscript >= 0.5.6
• tqdm >= 4.66
• jax >= 0.4.20
• jaxlib >= 0.4.20
• flax >= 0.10.7
• optax >= 0.2.6
• scipy >= 1.12
• tensorboard >= 2.20

From the project root:

cd learning-munsell

# Install all dependencies (creates virtual environment automatically)
uv sync

Usage

Generate Training Data

uv run python learning_munsell/data_generation/generate_training_data.py

Note: This step is computationally expensive (uses iterative algorithm for ground truth).

Train Models

xyY to Munsell (from_xyY)

Best performing model (Multi-MLP + Multi-Error Predictor):

# Train base Multi-MLP
uv run python learning_munsell/training/from_xyY/train_multi_mlp.py

# Train multi-error predictor
uv run python learning_munsell/training/from_xyY/train_multi_mlp_multi_error_predictor.py

Alternative architectures:

uv run python learning_munsell/training/from_xyY/train_multi_resnet_large.py
uv run python learning_munsell/training/from_xyY/train_multi_head_large.py
uv run python learning_munsell/training/from_xyY/train_ft_transformer.py
uv run python learning_munsell/training/from_xyY/train_unified_mlp.py
uv run python learning_munsell/training/from_xyY/train_deep_wide.py
uv run python learning_munsell/training/from_xyY/train_mlp_attention.py

Munsell to xyY (to_xyY)

Best performing model (Multi-MLP + Multi-Error Predictor):

# Train base Multi-MLP
uv run python learning_munsell/training/to_xyY/train_multi_mlp.py

# Train multi-error predictor
uv run python learning_munsell/training/to_xyY/train_multi_mlp_multi_error_predictor.py

Other architectures:

uv run python learning_munsell/training/to_xyY/train_multi_head.py
uv run python learning_munsell/training/to_xyY/train_multi_mlp_error_predictor.py
uv run python learning_munsell/training/to_xyY/train_multi_head_multi_error_predictor.py

Train the differentiable approximator for use in Delta-E loss:

uv run python learning_munsell/training/to_xyY/train_munsell_to_xyY_approximator.py

Hyperparameter Search

uv run python learning_munsell/training/from_xyY/hyperparameter_search_multi_head.py
uv run python learning_munsell/training/from_xyY/hyperparameter_search_multi_head_error_predictor.py

Compare All Models

uv run python learning_munsell/comparison/from_xyY/compare_all_models.py
uv run python learning_munsell/comparison/to_xyY/compare_all_models.py

Generates comprehensive HTML reports at reports/from_xyY/model_comparison.html and reports/to_xyY/model_comparison.html.

Monitor Training

MLflow:

uv run mlflow ui --backend-store-uri "sqlite:///mlruns.db" --port=5000

Open http://localhost:5000 in your browser.

Directory Structure

learning-munsell/
+-- data/                          # Training data
|   +-- training_data.npz          # Generated training samples
|   +-- training_data_large.npz    # Large dataset (~1.4M samples)
|   +-- training_data_params.json  # Generation parameters
|   +-- training_data_large_params.json
+-- models/                        # Trained models (ONNX + PyTorch)
|   +-- from_xyY/                  # xyY to Munsell models (38 ONNX models)
|   |   +-- multi_mlp_multi_error_predictor.onnx  # BEST
|   |   +-- ... (additional model variants)
|   +-- to_xyY/                    # Munsell to xyY models (6 ONNX models)
|       +-- multi_mlp_multi_error_predictor.onnx  # BEST
|       +-- ... (additional model variants)
+-- learning_munsell/              # Source code
|   +-- analysis/                  # Analysis scripts
|   +-- comparison/                # Model comparison scripts
|   +-- data_generation/           # Data generation scripts
|   +-- interpolation/             # Classical interpolation methods
|   +-- losses/                    # Loss functions (JAX Delta-E)
|   +-- models/                    # Model architecture definitions
|   +-- training/                  # Model training scripts
|   +-- utilities/                 # Shared utilities
+-- docs/                          # Documentation
+-- reports/                       # HTML comparison reports
+-- logs/                          # Script output logs
+-- mlruns.db                      # MLflow experiment tracking database

About

Learning Munsell by Colour Developers Research project for the Colour library https://github.com/colour-science/colour