---
license: apache-2.0
library_name: transformers
tags:
- reward-model
- rlhf
- alignment
---
# BestRewardModel
## Model Description
This is a reward model trained for RLHF alignment, selected from multiple experimental runs based on validation accuracy and reward alignment quality.
## Selection Criteria
The best checkpoint was chosen according to:
- **Highest `val_accuracy`** among all final checkpoints
- **Minimum `reward_alignment_score` threshold of 0.80**
Only checkpoints satisfying **both** conditions were eligible.
## Training Runs Comparison
| Run | Base Model | Learning Rate | Final Step | Val Accuracy | Reward Alignment | Train Loss |
|-----|-----------|--------------|------------|-------------|-----------------|------------|
| run_gpt2_base_lr1e4 | GPT-2 Base | 1e-4 | 1000 | 0.907 | 0.876 | 0.115 |
| run_gpt2_base_lr5e5 | GPT-2 Base | 5e-5 | 1000 | 0.870 | 0.839 | 0.207 |
| run_gpt2_large_lr1e4 | GPT-2 Large | 1e-4 | 1000 | 0.958 | 0.928 | 0.061 |
| run_gpt2_large_lr5e5 | GPT-2 Large | 5e-5 | 1000 | 0.901 | 0.854 | 0.159 |
| run_deberta_lr1e4 | DeBERTa-v2 | 1e-4 | 1000 | 0.837 | 0.827 | 0.301 |
## Best Run Metrics
| Metric | Value |
|--------|-------|
| Run Name | run_gpt2_large_lr1e4 |
| Val Accuracy | 0.958 |
| Reward Alignment Score | 0.928 |
| Final Train Loss | 0.061 |
## Intended Uses
This model is intended for use as a reward model in RLHF pipelines to score and rank model outputs based on human preference alignment.
## How to Use
```python
from transformers import AutoModelForSequenceClassification, AutoTokenizer
model = AutoModelForSequenceClassification.from_pretrained("BestRewardModel-TestRepo")
tokenizer = AutoTokenizer.from_pretrained("BestRewardModel-TestRepo")
inputs = tokenizer("prompt", "response", return_tensors="pt")
score = model(**inputs).logits[0].item()
```
## License
Apache-2.0
