Instructions to use twinkle-ai/twinkle-sqlcoder with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use twinkle-ai/twinkle-sqlcoder with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="twinkle-ai/twinkle-sqlcoder")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("twinkle-ai/twinkle-sqlcoder")
model = AutoModelForCausalLM.from_pretrained("twinkle-ai/twinkle-sqlcoder")

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use twinkle-ai/twinkle-sqlcoder with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "twinkle-ai/twinkle-sqlcoder"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "twinkle-ai/twinkle-sqlcoder",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/twinkle-ai/twinkle-sqlcoder

SGLang

How to use twinkle-ai/twinkle-sqlcoder with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "twinkle-ai/twinkle-sqlcoder" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "twinkle-ai/twinkle-sqlcoder",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "twinkle-ai/twinkle-sqlcoder" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "twinkle-ai/twinkle-sqlcoder",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use twinkle-ai/twinkle-sqlcoder with Docker Model Runner:
```
docker model run hf.co/twinkle-ai/twinkle-sqlcoder
```

twinkle-sqlcoder / README.md

whats2000

Upload Devstral SQLCoder SFT model

4546fa9 verified 2 months ago

preview code

raw

history blame contribute delete

1.82 kB

	---
	language:
	- en
	license: other
	base_model:
	- mistralai/Devstral-Small-2505
	tags:
	- text-to-sql
	- sql
	- mistral
	- transformers
	- safetensors
	pipeline_tag: text-generation
	library_name: transformers
	---

	# Devstral SQLCoder SFT

	This model is a full-parameter SFT checkpoint for SQL generation, trained from `mistralai/Devstral-Small-2505` and exported to Hugging Face safetensors format.

	## Model Details

	- Base model: `mistralai/Devstral-Small-2505`
	- Architecture: `MistralForCausalLM`
	- Precision used in training: bf16
	- Max sequence length (training config): 4096
	- Export format: sharded `safetensors` with `model.safetensors.index.json`

	## Training Data (Merged)

	The SFT run merged the following datasets:

	- spider
	- bird
	- bird23-train-filtered
	- synsql-2.5m
	- wikisql
	- gretelai-synthetic
	- sql-create-context

	## Intended Use

	- Text-to-SQL research and experimentation
	- SQL generation benchmarks and evaluation pipelines

	## Limitations

	- This model may generate incorrect SQL and should be validated before production use.
	- Performance depends on prompt format, schema context quality, and decoding settings.
	- Evaluate safety and compliance requirements before deployment.

	## Usage

	```python
	from transformers import AutoModelForCausalLM, AutoTokenizer

	repo_or_path = "<hf-username-or-org>/<model-repo>"

	tokenizer = AutoTokenizer.from_pretrained(repo_or_path, trust_remote_code=True)
	model = AutoModelForCausalLM.from_pretrained(
	repo_or_path,
	torch_dtype="bfloat16",
	)
	```

	## Local Files Included

	- `config.json`
	- `generation_config.json`
	- `tekken.json`
	- `model-00001-of-00021.safetensors` ... `model-00021-of-00021.safetensors`
	- `model.safetensors.index.json`

	## Citation

	If you use this model, please cite this repository:

	- https://github.com/ai-twinkle/twinkle-sqlcoder