--- license: mit tags: - text-generation - causal-lm - grpo - reasoning - reinforcement-learning - mini-llm - text-generation-inference - lm-evaluation-harness datasets: - openai/gsm8k - openai/openai_humaneval - cais/mmlu - allenai/ai2_arc - bespokelabs/Bespoke-Stratos-17k language: - en pipeline_tag: text-generation metrics: - accuracy - exact_match model-index: - name: Atomight-V2.1-0.5B-Inference results: - task: type: text-generation name: ARC-Challenge dataset: name: AI2 ARC-Challenge type: allenai/ai2_arc config: ARC-Challenge split: test metrics: - name: Accuracy type: acc value: 0.30802 - name: Accuracy (Norm) type: acc_norm value: 0.33788 - task: type: text-generation name: GSM8K dataset: name: GSM8K type: openai/gsm8k config: main split: test metrics: - name: Exact Match (Strict) type: exact_match value: 0.19788 - name: Exact Match (Flexible) type: exact_match value: 0.32449 - task: type: text-generation name: ARC-Easy dataset: name: AI2 ARC-Easy type: allenai/ai2_arc config: ARC-Easy split: test metrics: - name: Accuracy type: acc value: 0.65446 - name: Accuracy (Norm) type: acc_norm value: 0.59343 - task: type: text-generation name: HellaSwag dataset: name: HellaSwag type: Rowan/hellaswag split: validation metrics: - name: Accuracy type: acc value: 0.40709 - name: Accuracy (Norm) type: acc_norm value: 0.5235 --- # Atomight-V2.1-0.5B-Inference

Atomight Logo

**Atomight-V2.1-0.5B-Inference** is an ultra-compact, reasoning-oriented causal language model developed under the **[Atomight Ecosystem](https://huggingface.co/NovatasticRoScript/collections)**. Built on a Qwen-derived 494M parameter foundation, the model has been refined using **GRPO (Group Relative Policy Optimization)** reinforcement tuning. Despite its tiny physical footprint, Atomight-V2.1-0.5B targets highly efficient edge-device reasoning, structured text outputs, lightweight coding assistance, and rapid deployment workflows under severe compute constraints. ### 🚀 Key Highlights - **Parameter Footprint:** ~494M parameters (Loads into ~1GB VRAM at FP16). - **Training Paradigm:** GRPO reinforcement learning focusing on high-signal reasoning vectors instead of brute-force dataset scale. - **Edge-Optimized:** Designed specifically for low-overhead mobile, local, and browser-based inference loops (Google Colab / Kaggle native workflow). Weighted/Imatrix Quants, and Static Quants by [mradermacher](https://huggingface.co/mradermacher) are available at: https://huggingface.co/mradermacher/Atomight-V2.1-0.5B-Inference-i1-GGUF https://huggingface.co/mradermacher/Atomight-V2.1-0.5B-Inference-GGUF **NOTICE: Think tags (chain-of-thought reasoning) may not be working now due to some errors. But rest assured, the initial model is actively working & SAFE. We'll try our best to fix this as soon as possible.** --- ## 📊 Evaluation & Benchmark Results Official evaluations were conducted using the **EleutherAI LM Evaluation Harness** at FP16 precision. ### Core Evaluation Metrics | Benchmark Task | Metric Typology | Atomight-V2.1-0.5B Score | Focus Domain | | :--- | :--- | :--- | :--- | | **ARC-Easy** | Accuracy (Normalized) | **59.34%** | Scientific Fact Retrieval | | **HellaSwag** | Accuracy (Normalized) | **52.35%** | Commonsense Reasoning & Next-Sentence Prediction | | **ARC-Challenge** | Accuracy (Normalized) | **33.79%** | Hard Analytical Exclusion Logic | | **GSM8K (Flexible Extract)** | Exact Match (Regex Clean) | **32.45%** | Mathematical Thought & Resolution | | **GSM8K (Strict)** | Exact Match (Rigid Parse) | **19.79%** | Formatted Mathematical Output |

Atomight V2.1 Benchmark

### 🔍 Comparative Engineering Insights * **Punching Above Weight Classes:** Atomight-V2.1-0.5B outpaces Meta's larger **Llama-3.2-1B-Instruct** on localized logic-retrieval metrics, clearing **59.3%** on ARC-Easy and **33.8%** on ARC-Challenge compared to Llama's *56.7%* and *31.8%* respectively. * **The Reasoning Gap:** On mathematical reasoning (GSM8K), when evaluated with **Flexible Extraction parsing (32.45%)**, Atomight demonstrates higher raw mathematical accuracy than both Qwen2.5-0.5B-Instruct (*26.8%*) and Llama-3.2-1B-Instruct (*24.4%*). * **The Formatting Note:** The delta between Atomight's Strict Math score (19.8%) and Flexible Math score (32.5%) stems from the internal reasoning tokens generated during the inference step. While the mathematical conclusion is correct nearly 1/3 of the time, the model frequently bypasses rigid formatting constraints in favor of dense thinking traces. --- ## 💻 Quickstart: Inference Execution Atomight utilizes system and sequence prompts to partition thinking spaces. For optimal reasoning convergence, use explicit `` and `` encapsulation layers. ```python import torch from transformers import AutoModelForCausalLM, AutoTokenizer model_id = "NovatasticRoScript/Atomight-V2.1-0.5B-Inference" tokenizer = AutoTokenizer.from_pretrained(model_id) model = AutoModelForCausalLM.from_pretrained( model_id, torch_dtype=torch.float16, device_map="auto" ) # Structuring system guidelines for GRPO activation messages = [ { "role": "system", "content": "You are a reasoning model. Think inside and answer inside ." }, { "role": "user", "content": "A farmer has 12 apples. He gives 4 to his neighbor and loses 2 on the way home. How many apples does he have left?" } ] inputs = tokenizer.apply_chat_template( messages, tokenize=True, add_generation_prompt=True, return_tensors="pt" ).to("cuda") with torch.no_grad(): outputs = model.generate( inputs, max_new_tokens=250, temperature=0.01, pad_token_id=tokenizer.eos_token_id ) print(tokenizer.decode(outputs[0], skip_special_tokens=True)) --- ## 📄 Citations @misc{shao2024deepseekmathpushinglimitsmathematical, title={DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models}, author={Zhihong Shao and Peiyi Wang and Qihao Zhu and Junxiao Song and Daya Guo and et al.}, year={2024}, eprint={2402.03300}, archivePrefix={arXiv}, primaryClass={cs.CL}, url={https://arxiv.org/abs/2402.03300} } @misc{qwen_qwen25_2025, title={Qwen2.5 Technical Report}, author={Qwen and Yang, An and Yang, Baosong and Zhang, Beichen and Hui, Binyuan and et al.}, year={2024}, eprint={2412.15115}, archivePrefix={arXiv}, primaryClass={cs.CL}, url={https://arxiv.org/abs/2412.15115} } @misc{eval-harness, author={Gao, Leo and Tow, Jonathan and Abbasi, Baber and Biderman, Stella and Black, Sid and et al.}, title={A framework for few-shot language model evaluation}, month={12}, year={2023}, publisher={Zenodo}, version={v0.4.0}, doi={10.5281/zenodo.10257521}, url={https://github.com/EleutherAI/lm-evaluation-harness} } @article{allenai:arc, author = {Peter Clark and Isaac Cowhey and Oren Etzioni and Tushar Khot and Ashish Sabharwal and Carissa Schoenick and Oyvind Tafjord}, title = {Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge}, journal = {arXiv:1803.05457v1}, year = {2018}, } @article{cobbe2021gsm8k, title={Training Verifiers to Solve Math Word Problems}, author={Cobbe, Karl and Kosaraju, Vineet and Bavarian, Mohammad and Chen, Mark and Jun, Heewoo and Kaiser, Lukasz and Plappert, Matthias and Tworek, Jerry and Hilton, Jacob and Nakano, Reiichiro and Hesse, Christopher and Schulman, John}, journal={arXiv preprint arXiv:2110.14168}, year={2021} }