lora-tinyllama

Overview

lora-tinyllama is a fine-tuned version of the tinyllama-1.1b model, created using LoRA (Low-Rank Adaptation). This model specializes in adapting the tinyllama-1.1b base for specific tasks with minimal computational overhead.

Key Features

• Model Size: ~90MB (LoRA adapter weights only).
• Efficiency: Keeps the base model frozen and adds small trainable layers.
• Flexibility: Requires the original tinyllama-1.1b base model for usage.
• Purpose: Designed for specialized NLP tasks, leveraging the compact and powerful nature of the base model.

---

Usage Instructions

Prerequisites

Before using lora-tinyllama, ensure you have:

1. The base model: tinyllama-1.1b.
2. The fine-tuned LoRA weights: lora-tinyllama.

---

Loading the Model

Here’s how to load and use lora-tinyllama with the base model:

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

# Step 1: Load the base model
base_model_path = "path/to/tinyllama-1.1b"
base_model = AutoModelForCausalLM.from_pretrained(base_model_path)

# Step 2: Load the LoRA weights
lora_model_path = "path/to/lora-tinyllama"
lora_model = PeftModel.from_pretrained(base_model, lora_model_path)

# Step 3: Load the tokenizer
tokenizer = AutoTokenizer.from_pretrained(base_model_path)

# Step 4: Use the model for inference
inputs = tokenizer("Hello, world!", return_tensors="pt")
outputs = lora_model.generate(inputs["input_ids"])
print(tokenizer.decode(outputs[0]))