Kiwen-27B

A fine-tune of Qwen/Qwen3.8-27B trained on long chain-of-thought traces from Kimi K3.

This is not a broadly stronger model. The fine-tune primarily teaches Qwen3.8-27B to terminate its reasoning and produce an answer before exhausting its generation budget.

Model Summary

Qwen3.8-27B is a strong reasoning model that can sometimes spend its entire token budget deliberating without producing a final answer.

On GSM8K, this behavioral change produces a +10.31 percentage-point improvement under flexible extraction.

Key results

Benchmark	Qwen3.8-27B	Kiwen-27B	Delta	t
GSM8K exact_match (flexible)	0.7407	0.8438	+10.31	+6.58
GSM8K exact_match (strict)	0.6672	0.7278	+6.07	+3.40
IFEval prompt-level strict	0.8226	0.8429	+2.03	+0.90
IFEval prompt-level loose	0.8447	0.8669	+2.22	+1.04
IFEval instruction-level strict	0.8393	0.8645	+2.52	—
IFEval instruction-level loose	0.8537	0.8801	+2.64	—
VMLU val (744 Vietnamese MMLU questions)	83.47	86.02	+2.55	+1.37

Evaluated using lm-evaluation-harness 0.4.12 with both models served on identical hardware:

• Hardware: 1× H200
• Inference: SGLang
• Model dtype: BF16
• KV cache: FP8
• Generation limit: max_gen_toks=4096
• Prompts: identical
• Decoding settings: identical

The largest improvement is on GSM8K. Other benchmarks show smaller but consistently positive changes, with no measured regression.

The Mechanism: Learning When to Stop

GSM8K is evaluated using two extraction methods:

• Strict match: requires the canonical #### <number> answer format.
• Flexible extraction: extracts the final number from the generated response.

The improvement under flexible extraction is substantially larger:

+10.31 points flexible vs. +6.07 points strict

This asymmetry suggests that much of the improvement comes from successfully producing an answer before the generation budget is exhausted, rather than from a fundamental improvement in mathematical reasoning.

Out-of-Domain Regression

To test whether the fine-tuning damaged unrelated capabilities, both models were evaluated on 100 held-out Vietnamese enterprise documents covering:

• Accounting
• CRM
• Internal RAG workloads

The metric was token-level negative log-likelihood:

Model	NLL / token
Qwen3.8-27B	3.3390
Kiwen-27B	3.3543

Paired difference:

• Δ: +0.0067
• Standard error: 0.0041
• t: 1.65

Usage

from transformers import AutoModelForImageTextToText, AutoProcessor

model = AutoModelForImageTextToText.from_pretrained(
    "beyoru/Kiwen-27B",
    dtype="bfloat16",
    device_map="auto",
)

proc = AutoProcessor.from_pretrained(
    "beyoru/Kiwen-27B"
)

MTP Tensors

MTP tensors are absent from the released model.

transformers declares:

_keys_to_ignore_on_load_unexpected = [r"^mtp.*"]

Therefore, the multi-token-prediction block is dropped by HF-based merging, including this release.

For speculative decoding, use an external draft model such as z-lab/Qwen3.8-27B-DFlash2 rather than NEXTN.

A draft model trained against the original weights may have a lower acceptance rate against the merged Kiwen weights. This does not affect correctness because speculative decoding uses rejection verification.

Benchmark Caveat

VMLU validation is publicly distributed with answer keys and may have some representation in pretraining data.

Its absolute score should therefore be interpreted cautiously.

However, the base-versus-fine-tuned comparison remains informative, since both models are exposed to the same potential contamination.

Citation

@misc{kiwen27bk3,
  title  = {Kiwen-27B},
  author = {beyoru},
  year   = {2026},
  url    = {https://huggingface.co/beyoru/Kiwen-27B-K3}
}

License & Attribution

Built on:

• Qwen3.8-27B — Apache-2.0

Kiwen-27B is released under the Apache-2.0 license.