Qwen3-4B-Base-ThinkCode-A-NH025 — PEFT Adapter

This repository contains a PEFT LoRA adapter only. It does not contain the Qwen3 base-model weights and cannot be loaded as a standalone causal language model.

The required base is Qwen/Qwen3-4B-Base at the fixed revision 906bfd4b4dc7f14ee4320094d8b41684abff8539.

Adapter construction

A-NH025 is the Phase A no-head arm. Starting from the completed source LoRA, every selected transformer-body LoRA B tensor is multiplied by 0.25 in FP32, while the lm_head LoRA B tensor is multiplied by 0, making its effective head/shared-embedding delta exactly zero. LoRA A tensors are unchanged. With lora_alpha=128 and r=64, PEFT applies the intended body delta without a language-model-head delta across the 253 declared modules.

The effective-zero lm_head adapter is omitted from the release state and target list; this is exactly equivalent to its validated zero delta and avoids packaging any base-layer tensor. MODULE_SCALE_MANIFEST.json retains the explicit zero-head contract and records every logical module, source tensor key, physical base weight, and scale. This release is from the completed Phase A delta-scaling line; it is not the later failed NEXTGEN route and does not include subsequent protocol-repair experiments.

Loading with PEFT

Use recent transformers and peft versions. Load the fixed base first, then attach this adapter:

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base_id = "Qwen/Qwen3-4B-Base"
base_revision = "906bfd4b4dc7f14ee4320094d8b41684abff8539"
adapter_id = "modrill/Qwen3-4B-Base-ThinkCode-A-NH025"

tokenizer = AutoTokenizer.from_pretrained(base_id, revision=base_revision)
base = AutoModelForCausalLM.from_pretrained(
    base_id,
    revision=base_revision,
    torch_dtype="auto",
    device_map="auto",
)
model = PeftModel.from_pretrained(base, adapter_id)

messages = [{"role": "user", "content": "Write a Python function that checks whether a number is prime."}]
prompt = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
    enable_thinking=False,
)
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
eos_ids = [
    tokenizer.eos_token_id,
    tokenizer.convert_tokens_to_ids("<|im_end|>"),
]
outputs = model.generate(**inputs, max_new_tokens=2048, eos_token_id=eos_ids)
print(tokenizer.decode(outputs[0][inputs.input_ids.shape[-1]:], skip_special_tokens=True))

The base tokenizer's chat template supports enable_thinking. Disable it for direct code generation matching the concise screening style, or enable it when explicit reasoning is desired. Pass both <|endoftext|> and <|im_end|> as EOS IDs. Keep the combined prompt and generated sequence within 32K tokens, the fixed base model configuration limit, unless a separate long-context extension is validated.

Development evaluation

Across three preregistered seeds on the corrected EvalScope Full1055 development suite, the resolved code_only aggregate was 794/3165 = 25.09%. Relative to the fixed BASE, the estimated change was approximately +0.98 percentage points, with an approximate 95% confidence interval of [+0.095, +1.833] percentage points. The Holm-adjusted p-value was 0.489.

These results are development-only, not a held-out formal claim. In the original bidirectional scoring for seed=3407, some outcomes flipped between PASS and TLE because of the execution environment. Those cases were resolved by fixed single-CPU serial rejudgment, which does not eliminate all scorer, timing, or environment uncertainty.

Limitations

• This adapter requires the exact base model and should not be loaded alone.
• The evidence is development-only and includes scorer-environment uncertainty.
• Generated code can be incorrect, insecure, or non-compiling; sandbox and test it independently.
• No production safety, security, or suitability certification is implied.

License

The fixed base card and included license identify Apache-2.0. This adapter preserves that license text and metadata. Users should independently verify the upstream Qwen3 license, notices, training-data terms, and applicability to their use case.