README.md

---
language:
- en
license: mit
tags:
- finance
- continual-learning
- qwen2
- causal-lm
- ewc
base_model: Qwen/Qwen2.5-0.5B
datasets:
- gbharti/finance-alpaca
- sujet-ai/Sujet-Finance-Instruct-177k
- nvidia/OpenMathInstruct-2
- HuggingFaceFW/fineweb-edu
- yahma/alpaca-cleaned
library_name: transformers
pipeline_tag: text-generation
---

# Meridian.AI — Continual-Learning Finance LLM

Meridian.AI is a finance-specialized language model that continuously fine-tunes a
**Qwen2.5-0.5B** backbone every hour on 25+ finance and math datasets, using **Elastic
Weight Consolidation (EWC)** to prevent catastrophic forgetting across training sessions.
The entire pipeline runs unattended on free GitHub Actions infrastructure — no GPUs.

- **Base model:** [`Qwen/Qwen2.5-0.5B`](https://huggingface.co/Qwen/Qwen2.5-0.5B) (~494M params, Qwen2 architecture)
- **Continual learning:** Elastic Weight Consolidation (diagonal Fisher)
- **Training cadence:** hourly GitHub Actions CI on CPU runners
- **Source code & full docs:** [github.com/MeridianAlgo/FinAI](https://github.com/MeridianAlgo/FinAI)

## Usage

The deployed checkpoint is a **standard Qwen2 model** — `trust_remote_code=True` is **not**
required.

```python
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

repo_id = "meridianal/FinAI"

tokenizer = AutoTokenizer.from_pretrained(repo_id, subfolder="checkpoint")
model = AutoModelForCausalLM.from_pretrained(
    repo_id,
    subfolder="checkpoint",
    torch_dtype=torch.float32,
    low_cpu_mem_usage=True,
)
model.eval()

prompt = """### Instruction:
Explain the difference between a bond's yield to maturity and its coupon rate.

### Response:
"""

inputs = tokenizer(prompt, return_tensors="pt")
with torch.no_grad():
    output = model.generate(
        **inputs,
        max_new_tokens=200,
        do_sample=True,
        temperature=0.8,
        top_p=0.92,
        repetition_penalty=1.3,
        no_repeat_ngram_size=3,
        pad_token_id=tokenizer.pad_token_id,
        eos_token_id=tokenizer.eos_token_id,
    )

print(tokenizer.decode(output[0], skip_special_tokens=True))
```

Inputs are formatted with the `### Instruction: / ### Response:` template used during training.

## Model details

| Specification | Value |
|:---|:---|
| Base model | Qwen2.5-0.5B |
| Architecture | Qwen2ForCausalLM |
| Parameters | ~494M |
| Context window | 32,768 tokens (Qwen2.5 default) |
| Training dtype | bfloat16 |
| Continual learning | Elastic Weight Consolidation (EWC) |

## Training data

A weighted streaming mix of 25+ finance and instruction datasets, including
`gbharti/finance-alpaca`, `sujet-ai/Sujet-Finance-Instruct-177k`,
`nvidia/OpenMathInstruct-2`, `HuggingFaceFW/fineweb-edu`, `yahma/alpaca-cleaned`, and the
FinanceMTEB suite. See the
[repository README](https://github.com/MeridianAlgo/FinAI#dataset-curriculum) for the full
curriculum and weights.

## Limitations & disclaimer

This is an experimental research project on continual learning for financial NLP. Outputs
may contain factual errors and are intended for academic and research purposes only.
**Nothing generated by this model constitutes financial advice. Do not use outputs to make
real financial decisions or execute trades.**