Instructions to use bunnycore/SmartToxic-7B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use bunnycore/SmartToxic-7B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="bunnycore/SmartToxic-7B")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("bunnycore/SmartToxic-7B")
model = AutoModelForCausalLM.from_pretrained("bunnycore/SmartToxic-7B")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use bunnycore/SmartToxic-7B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "bunnycore/SmartToxic-7B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "bunnycore/SmartToxic-7B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/bunnycore/SmartToxic-7B

SGLang

How to use bunnycore/SmartToxic-7B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "bunnycore/SmartToxic-7B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "bunnycore/SmartToxic-7B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "bunnycore/SmartToxic-7B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "bunnycore/SmartToxic-7B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use bunnycore/SmartToxic-7B with Docker Model Runner:
```
docker model run hf.co/bunnycore/SmartToxic-7B
```
Browse Quantizations to use this model in llama.cpp, Ollama, LM Studio, or any compatible app.

SmartToxic-7B

SmartToxic-7B is a creative and smart language model designed to provide users with engaging and satisfying responses. This model is a merger of several high-performing models, resulting in a unique blend of capabilities. While the model is not uncensored, it aims to maintain a balance between creativity and appropriateness.

Performance Benchmarks:

SmartToxic-7B has demonstrated strong performance on various benchmark tests, showcasing its ability to generate creative and engaging content. However, users are encouraged to test the model themselves to determine if it meets their specific needs and requirements.

Limitations:

While SmartToxic-7B is a powerful language model, it may still struggle with certain types of queries or generate responses that are not entirely accurate or appropriate. Users should be aware of these potential limitations and use the model's outputs with discretion.

SmartToxic-7B is a merge of the following models using mergekit:

🧩 Configuration

models:
  - model: ResplendentAI/Datura_7B
  - model: BarryFutureman/WestLakeX-7B-EvoMerge-Variant2
  - model: MaziyarPanahi/Calme-7B-Instruct-v0.9
merge_method: model_stock
base_model: FuseAI/FuseChat-7B-VaRM
dtype: bfloat16

Downloads last month: 7

Safetensors

Model size

7B params

Tensor type

BF16

Model tree for bunnycore/SmartToxic-7B

Quantizations

2 models