Spaces:
Running
Running
| # Validation checklist | |
| Release checks prevent software, deployment, and provenance mistakes; they do not pretend the analytical profile has measured-hardware fidelity. | |
| ## Automated software checks | |
| - deterministic constant/Poisson/bursty workload generation | |
| - exact trace replay and trace validation | |
| - prefill latency monotonicity | |
| - quantization footprint ordering | |
| - latency sensitivity-scale behavior | |
| - colocated end-to-end completion | |
| - static vs continuous behavioral difference | |
| - component TTFT/E2E SLO accounting | |
| - robust repetition-aware capacity search | |
| - bottleneck-diagnosis provenance | |
| - prefix hits do not alter the underlying generated request trace | |
| - prefix reuse reduces modeled prefill work | |
| - P/D pipeline completion and transfer telemetry | |
| - configurable P/D worker counts | |
| - four-scenario topology/cache comparison | |
| - bounded design-space sweep and two Pareto objectives | |
| - paired common-seed research study | |
| - bootstrap paired-effect intervals | |
| - analytical-profile perturbation study | |
| - external-measurement import, normalization, and validation report generation | |
| - train-only calibration with held-out evaluation when enough cases exist | |
| - repeated-seed robust policy ranking and bootstrap intervals | |
| - Pareto-stability accounting across matched seeds | |
| - bounded full-trace oracle candidate search and policy-regret accounting | |
| - Markdown research-report generation with interpretation guardrails | |
| - deterministic stateful agent-session trace generation | |
| - zero cross-turn hits under immediate eviction | |
| - KV reuse and reduced recomputation under retention + affinity | |
| - common-trace four-policy agent comparison | |
| - TTL latency/memory Pareto frontier | |
| - ASCII-only public UI labels | |
| - chart export controls present | |
| - explicit planner worst-repetition and target columns | |
| - no product-style footer | |
| - Hugging Face `short_description` <= 60 characters | |
| - `sdk: static` metadata | |
| - canonical Python source equals browser mirror | |
| - every browser Python module is included by the worker | |
| - provenance remains `analytical-reference` | |
| - JavaScript syntax parse | |
| - Python compilation | |
| ## Empirical validation protocol | |
| A future hardware run should be stored as external validation cases, not copied into analytical profiles without provenance. At minimum each case should record: | |
| - model / revision | |
| - serving topology and scheduler | |
| - accelerator / count | |
| - precision | |
| - workload arrival process or trace ID | |
| - prompt/output distribution or exact trace | |
| - SLO definition | |
| - p95 TTFT / p95 E2E / goodput | |
| - runtime/software versions | |
| For already-normalized cases run: | |
| ```bash | |
| python scripts/validate_measurements.py measured_cases.json --output validation_report.json | |
| ``` | |
| For serving-tool artifacts run: | |
| ```bash | |
| python scripts/import_measurements.py result.json --source auto --output measurement_cases.json | |
| python scripts/calibrate_profiles.py measurement_cases.json --holdout 0.33 --output calibration_result.json | |
| ``` | |
| The calibration script fits only on the training cases and reports baseline/calibrated residuals on held-out cases when at least three cases are available. A smaller case set is explicitly marked as resubstitution rather than held-out validation. | |
| ## Not claimed | |
| - empirical L4/A10G/A100 latency accuracy | |
| - exact vLLM/SGLang scheduler equivalence | |
| - CUDA-kernel modeling | |
| - real network-protocol fidelity | |
| - radix-tree prefix-cache eviction/scheduling | |
| - speculative decoding | |
| - Attention-FFN disaggregation | |
| - production-fidelity agentic serving or dynamic batching inside Stateful Sessions | |
| - real tool execution or production agent-runtime integration | |
| - claims that the bounded offline oracle is globally optimal | |
| - claims that simulator bootstrap intervals quantify real-hardware error | |
| These are explicit scope boundaries, not hidden assumptions. | |