finmigodeveloper/distilbert-transaction-classifier-lora

Transaction Category Classifier - LoRA Version

This is a LoRA adapter for DistilBERT that classifies bank transactions into 10 categories with 98.53% accuracy.

Model Details

• Base Model: distilbert-base-uncased
• Fine-tuned Model: finmigodeveloper/distilbert-transaction-classifier
• Adapter Size: ~2.5 MB (98.7% smaller than full model)
• Categories: 10 transaction types

Performance

Metric	Value
Accuracy	98.53%
Loss	0.0221
Training Samples	80,000
Validation Samples	20,000

Categories

• Charity & Donations
• Entertainment & Recreation
• Financial Services
• Food & Dining
• Government & Legal
• Healthcare & Medical
• Income
• Shopping & Retail
• Transportation
• Utilities & Services

How to Use

from transformers import pipeline

# Load directly
classifier = pipeline("text-classification", 
                     model="finmigodeveloper/distilbert-transaction-classifier-lora")

# Test it
transactions = [
    "Starbucks coffee",
    "Monthly salary deposit", 
    "Uber ride to airport"
]

for text in transactions:
    result = classifier(text)[0]
    print(f"{text}: {result['label']} ({result['score']:.2%})")

Training Details

• LoRA Rank (r): 8
• LoRA Alpha: 16
• Target Modules: q_lin, k_lin, v_lin, out_lin
• Dropout: 0.1
• Epochs: 3
• Batch Size: 64
• Learning Rate: 2e-5

Why LoRA?

• 98.7% smaller than the full model
• Faster loading (~0.3 seconds vs 2-3 seconds)
• Same accuracy as the full model
• Perfect for mobile apps and edge deployment

Files in this repository

• adapter_model.safetensors: The LoRA adapter weights (2.5 MB)
• adapter_config.json: LoRA configuration
• training_stats.json: Detailed training statistics
• tokenizer.json & tokenizer_config.json: Tokenizer files