svenschwieger/llama-70b-contract-finetuned

Llama 3.1 70B - Contract Risk Analysis (LoRA)

A fine-tuned LoRA adapter for Meta Llama 3.1 70B specialized in construction contract risk analysis.

Model Details

• Base Model: meta-llama/Llama-3.1-70B
• Fine-tuning Method: LoRA (Low-Rank Adaptation)
• LoRA Rank: 32
• Target Modules: all-linear
• Task: Contract clause risk analysis

Training Details

Parameter	Value
Batch Size	2
Gradient Accumulation	4
Effective Batch Size	8
Learning Rate	2e-5
LR Schedule	Cosine
Epochs	1
Max Sequence Length	2048
Warmup Steps	100

Usage

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

# Load base model
base_model = AutoModelForCausalLM.from_pretrained(
    "meta-llama/Llama-3.1-70B",
    torch_dtype="auto",
    device_map="auto"
)

# Load LoRA adapter
model = PeftModel.from_pretrained(base_model, "svenschwieger/llama-70b-contract-finetuned")

# Load tokenizer
tokenizer = AutoTokenizer.from_pretrained("meta-llama/Llama-3.1-70B")

Training Data

The model was fine-tuned on synthetic construction contract clauses covering:

• Liquidated damages
• Termination for convenience
• Insurance requirements
• Audit rights
• Governing law provisions

License

This model inherits the Llama 3.1 license.