The Master Key Hypothesis: Unlocking Cross-Model Capability Transfer via Linear Subspace Alignment
Abstract
Post-trained model capabilities can be transferred across different model scales through linear alignment of latent subspace directions without requiring retraining.
We investigate whether post-trained capabilities can be transferred across models without retraining, with a focus on transfer across different model scales. We propose the Master Key Hypothesis, which states that model capabilities correspond to directions in a low-dimensional latent subspace that induce specific behaviors and are transferable across models through linear alignment. Based on this hypothesis, we introduce UNLOCK, a training-free and label-free framework that extracts a capability direction by contrasting activations between capability-present and capability-absent Source variants, aligns it with a Target model through a low-rank linear transformation, and applies it at inference time to elicit the behavior. Experiments on reasoning behaviors, including Chain-of-Thought (CoT) and mathematical reasoning, demonstrate substantial improvements across model scales without training. For example, transferring CoT reasoning from Qwen1.5-14B to Qwen1.5-7B yields an accuracy gain of 12.1% on MATH, and transferring a mathematical reasoning direction from Qwen3-4B-Base to Qwen3-14B-Base improves AGIEval Math accuracy from 61.1% to 71.3%, surpassing the 67.8% achieved by the 14B post-trained model. Our analysis shows that the success of transfer depends on the capabilities learned during pre-training, and that our intervention amplifies latent capabilities by sharpening the output distribution toward successful reasoning trajectories.
Community
Sharing our new work on The Master Key Hypothesis: Unlocking Cross-Model Capability Transfer via Linear Subspace Alignment
This is an automated message from the Librarian Bot. I found the following papers similar to this paper.
The following papers were recommended by the Semantic Scholar API
- Nudging Hidden States: Training-Free Model Steering for Chain-of-Thought Reasoning in Large Audio-Language Models (2026)
- Thinking in Different Spaces: Domain-Specific Latent Geometry Survives Cross-Architecture Translation (2026)
- CapTrack: Multifaceted Evaluation of Forgetting in LLM Post-Training (2026)
- Where to Steer: Input-Dependent Layer Selection for Steering Improves LLM Alignment (2026)
- SteerRM: Debiasing Reward Models via Sparse Autoencoders (2026)
- Closing the Confidence-Faithfulness Gap in Large Language Models (2026)
- SAE as a Crystal Ball: Interpretable Features Predict Cross-domain Transferability of LLMs without Training (2026)
Please give a thumbs up to this comment if you found it helpful!
If you want recommendations for any Paper on Hugging Face checkout this Space
You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: @librarian-bot recommend
Models citing this paper 0
No model linking this paper
Datasets citing this paper 0
No dataset linking this paper