feat: update model references to openbmb/MiniCPM-V-4.6 in documentation
Browse files- README.md +8 -6
- progress.md +0 -2
- pyproject.toml +1 -1
- rune_goblin_plan.md +9 -9
README.md
CHANGED
|
@@ -14,7 +14,7 @@ pinned: false
|
|
| 14 |
[](https://github.com/ASH1998/Rune-Goblin/actions/workflows/deploy-hf-space.yml)
|
| 15 |
|
| 16 |
A tiny dungeon crawler where players draw spells in an invented symbolic
|
| 17 |
-
language (**RuneLang**) and a fine-tuned [`openbmb/
|
| 18 |
acts as the **spell engine** β reading glyph combinations and emitting JSON
|
| 19 |
that drives attacks, curses and game-state changes. Runtime visuals are not
|
| 20 |
image-generated; spell metadata recolors, resizes, retargets and animates
|
|
@@ -27,7 +27,7 @@ See [`rune_goblin_plan.md`](./rune_goblin_plan.md) for the full design doc.
|
|
| 27 |
```
|
| 28 |
Rune buttons (Gradio / React)
|
| 29 |
β serialized rune sequence + game state
|
| 30 |
-
β fine-tuned
|
| 31 |
β spell outcome JSON (validated by rune_goblin.schema)
|
| 32 |
β game state engine (rune_goblin.game, clamps HP)
|
| 33 |
β updated UI
|
|
@@ -271,7 +271,7 @@ context, then returns attack/VFX metadata for the renderer.
|
|
| 271 |
|
| 272 |
## Notes on the model
|
| 273 |
|
| 274 |
-
- **Fine-tuning base**: `openbmb/
|
| 275 |
LoRA-trainable. This is what `rune-goblin-download` and `finetune.py` use.
|
| 276 |
- **Vision fine-tune**: `ASHu2/goblinV1` β merged MiniCPM-V model for canvas
|
| 277 |
drawings. The local GGUF files live in `models/goblinV1-gguf/gguf/`; use
|
|
@@ -281,9 +281,11 @@ context, then returns attack/VFX metadata for the renderer.
|
|
| 281 |
- **Asset planner model**: `models/MiniCPM-V-4.6-gguf/MiniCPM-V-4_6-Q8_0.gguf`
|
| 282 |
β planned higher-quality model for attack type, palette, size, area, path,
|
| 283 |
impact reaction, particle tags and animation timing from validated spell JSON.
|
| 284 |
-
-
|
| 285 |
-
|
| 286 |
-
|
|
|
|
|
|
|
| 287 |
|
| 288 |
Renderer rule: do not generate new images during combat or exploration. Use
|
| 289 |
model metadata to drive existing sprites, particles, overlays, CSS/canvas
|
|
|
|
| 14 |
[](https://github.com/ASH1998/Rune-Goblin/actions/workflows/deploy-hf-space.yml)
|
| 15 |
|
| 16 |
A tiny dungeon crawler where players draw spells in an invented symbolic
|
| 17 |
+
language (**RuneLang**) and a fine-tuned [`openbmb/MiniCPM-V-4.6`](https://huggingface.co/openbmb/MiniCPM-V-4.6)
|
| 18 |
acts as the **spell engine** β reading glyph combinations and emitting JSON
|
| 19 |
that drives attacks, curses and game-state changes. Runtime visuals are not
|
| 20 |
image-generated; spell metadata recolors, resizes, retargets and animates
|
|
|
|
| 27 |
```
|
| 28 |
Rune buttons (Gradio / React)
|
| 29 |
β serialized rune sequence + game state
|
| 30 |
+
β fine-tuned openbmb/MiniCPM-V-4.6 + LoRA (rune_goblin.inference)
|
| 31 |
β spell outcome JSON (validated by rune_goblin.schema)
|
| 32 |
β game state engine (rune_goblin.game, clamps HP)
|
| 33 |
β updated UI
|
|
|
|
| 271 |
|
| 272 |
## Notes on the model
|
| 273 |
|
| 274 |
+
- **Fine-tuning base**: `openbmb/MiniCPM-V-4.6` β safetensors, vision-capable,
|
| 275 |
LoRA-trainable. This is what `rune-goblin-download` and `finetune.py` use.
|
| 276 |
- **Vision fine-tune**: `ASHu2/goblinV1` β merged MiniCPM-V model for canvas
|
| 277 |
drawings. The local GGUF files live in `models/goblinV1-gguf/gguf/`; use
|
|
|
|
| 281 |
- **Asset planner model**: `models/MiniCPM-V-4.6-gguf/MiniCPM-V-4_6-Q8_0.gguf`
|
| 282 |
β planned higher-quality model for attack type, palette, size, area, path,
|
| 283 |
impact reaction, particle tags and animation timing from validated spell JSON.
|
| 284 |
+
- **Dialogue / story model**: the base (non-fine-tuned) `openbmb/MiniCPM-V-4.6`
|
| 285 |
+
drives NPC dialogue and story progression β loaded as the GGUF
|
| 286 |
+
`models/MiniCPM-V-4.6-gguf/MiniCPM-V-4_6-Q4_K_M.gguf` via
|
| 287 |
+
`RG_DIALOGUE_MODEL`. It's a **quantized GGUF multimodal** model and **cannot**
|
| 288 |
+
be LoRA-fine-tuned. Download it with `--gguf` only if you need it.
|
| 289 |
|
| 290 |
Renderer rule: do not generate new images during combat or exploration. Use
|
| 291 |
model metadata to drive existing sprites, particles, overlays, CSS/canvas
|
progress.md
CHANGED
|
@@ -147,8 +147,6 @@ uv run --extra gguf python app/vision_app.py # β http://localhost:7861
|
|
| 147 |
Play with `RG_USE_MODEL=0` to skip the model entirely (drawings then fall back
|
| 148 |
to the rule engine; rune-button casts are unaffected). Rune-button casts are
|
| 149 |
always instant; the first drawing cast loads the vision model (~30s on CPU).
|
| 150 |
-
`.claude/launch.json` has a `rune-goblin` preview config (model off) for quick
|
| 151 |
-
UI checks.
|
| 152 |
|
| 153 |
---
|
| 154 |
|
|
|
|
| 147 |
Play with `RG_USE_MODEL=0` to skip the model entirely (drawings then fall back
|
| 148 |
to the rule engine; rune-button casts are unaffected). Rune-button casts are
|
| 149 |
always instant; the first drawing cast loads the vision model (~30s on CPU).
|
|
|
|
|
|
|
| 150 |
|
| 151 |
---
|
| 152 |
|
pyproject.toml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
| 1 |
[project]
|
| 2 |
name = "rune-goblin"
|
| 3 |
version = "1.0.0"
|
| 4 |
-
description = "A fine-tuned spell-language dungeon game powered by
|
| 5 |
readme = "README.md"
|
| 6 |
requires-python = ">=3.10,<3.13"
|
| 7 |
license = { text = "Apache-2.0" }
|
|
|
|
| 1 |
[project]
|
| 2 |
name = "rune-goblin"
|
| 3 |
version = "1.0.0"
|
| 4 |
+
description = "A fine-tuned spell-language dungeon game powered by openbmb/MiniCPM-V-4.6."
|
| 5 |
readme = "README.md"
|
| 6 |
requires-python = ">=3.10,<3.13"
|
| 7 |
license = { text = "Apache-2.0" }
|
rune_goblin_plan.md
CHANGED
|
@@ -61,10 +61,10 @@ Rune Goblin fits because the AI is the core game mechanic. The player invents sp
|
|
| 61 |
|
| 62 |
## 4. What We Fine-Tune
|
| 63 |
|
| 64 |
-
We fine-tune
|
| 65 |
|
| 66 |
```text
|
| 67 |
-
|
| 68 |
```
|
| 69 |
|
| 70 |
The fine-tuned text model becomes:
|
|
@@ -86,7 +86,7 @@ The full drawing pipeline is:
|
|
| 86 |
Player drawing
|
| 87 |
β goblinV1-gguf Q4_K_M sketch reader
|
| 88 |
β spell sketch metadata / detected rune sequence
|
| 89 |
-
β fine-tuned
|
| 90 |
β validated spell outcome JSON
|
| 91 |
β MiniCPM-V-4.6 Q8 asset planner
|
| 92 |
β animation / attack / asset metadata JSON
|
|
@@ -347,7 +347,7 @@ Split:
|
|
| 347 |
6. Use a larger model or template system to create funny spell names and flavor text.
|
| 348 |
7. Validate JSON.
|
| 349 |
8. Save as JSONL.
|
| 350 |
-
9. Fine-tune
|
| 351 |
```
|
| 352 |
|
| 353 |
---
|
|
@@ -394,7 +394,7 @@ Gradio UI
|
|
| 394 |
|
|
| 395 |
Rune Serializer
|
| 396 |
|
|
| 397 |
-
Fine-tuned
|
| 398 |
|
|
| 399 |
JSON Spell Result
|
| 400 |
|
|
|
@@ -583,7 +583,7 @@ Modal credits can be used for:
|
|
| 583 |
Local / Modal:
|
| 584 |
1. Generate rune_spell_dataset.jsonl
|
| 585 |
2. Upload dataset to Hugging Face
|
| 586 |
-
3. Fine-tune
|
| 587 |
4. Save LoRA adapter
|
| 588 |
5. Push adapter to Hugging Face
|
| 589 |
6. Load adapter in Gradio or Modal endpoint
|
|
@@ -594,13 +594,13 @@ Local / Modal:
|
|
| 594 |
### 13.3 Suggested Training Setup
|
| 595 |
|
| 596 |
```text
|
| 597 |
-
Base model:
|
| 598 |
Method: LoRA / QLoRA
|
| 599 |
Dataset size: 5k examples
|
| 600 |
Epochs: 2β4
|
| 601 |
Max sequence length: 1024 or 2048
|
| 602 |
Batch size: based on GPU
|
| 603 |
-
Output: rune-goblin-
|
| 604 |
```
|
| 605 |
|
| 606 |
---
|
|
@@ -678,7 +678,7 @@ Good output:
|
|
| 678 |
|
| 679 |
| Badge | How We Earn It |
|
| 680 |
|---|---|
|
| 681 |
-
| Well-Tuned | Fine-tuned
|
| 682 |
| Off-Brand | Custom Gradio UI that looks like a tiny cursed dungeon |
|
| 683 |
| Field Notes | Blog post explaining RuneLang and training process |
|
| 684 |
| Open Trace | Publish sample game traces / spell logs |
|
|
|
|
| 61 |
|
| 62 |
## 4. What We Fine-Tune
|
| 63 |
|
| 64 |
+
We fine-tune the OpenBMB vision model:
|
| 65 |
|
| 66 |
```text
|
| 67 |
+
openbmb/MiniCPM-V-4.6
|
| 68 |
```
|
| 69 |
|
| 70 |
The fine-tuned text model becomes:
|
|
|
|
| 86 |
Player drawing
|
| 87 |
β goblinV1-gguf Q4_K_M sketch reader
|
| 88 |
β spell sketch metadata / detected rune sequence
|
| 89 |
+
β fine-tuned openbmb/MiniCPM-V-4.6 RuneLang spell engine
|
| 90 |
β validated spell outcome JSON
|
| 91 |
β MiniCPM-V-4.6 Q8 asset planner
|
| 92 |
β animation / attack / asset metadata JSON
|
|
|
|
| 347 |
6. Use a larger model or template system to create funny spell names and flavor text.
|
| 348 |
7. Validate JSON.
|
| 349 |
8. Save as JSONL.
|
| 350 |
+
9. Fine-tune openbmb/MiniCPM-V-4.6 with LoRA.
|
| 351 |
```
|
| 352 |
|
| 353 |
---
|
|
|
|
| 394 |
|
|
| 395 |
Rune Serializer
|
| 396 |
|
|
| 397 |
+
Fine-tuned openbmb/MiniCPM-V-4.6 LoRA
|
| 398 |
|
|
| 399 |
JSON Spell Result
|
| 400 |
|
|
|
|
|
| 583 |
Local / Modal:
|
| 584 |
1. Generate rune_spell_dataset.jsonl
|
| 585 |
2. Upload dataset to Hugging Face
|
| 586 |
+
3. Fine-tune openbmb/MiniCPM-V-4.6 using LoRA/QLoRA
|
| 587 |
4. Save LoRA adapter
|
| 588 |
5. Push adapter to Hugging Face
|
| 589 |
6. Load adapter in Gradio or Modal endpoint
|
|
|
|
| 594 |
### 13.3 Suggested Training Setup
|
| 595 |
|
| 596 |
```text
|
| 597 |
+
Base model: openbmb/MiniCPM-V-4.6
|
| 598 |
Method: LoRA / QLoRA
|
| 599 |
Dataset size: 5k examples
|
| 600 |
Epochs: 2β4
|
| 601 |
Max sequence length: 1024 or 2048
|
| 602 |
Batch size: based on GPU
|
| 603 |
+
Output: rune-goblin-minicpm-v-4.6-lora
|
| 604 |
```
|
| 605 |
|
| 606 |
---
|
|
|
|
| 678 |
|
| 679 |
| Badge | How We Earn It |
|
| 680 |
|---|---|
|
| 681 |
+
| Well-Tuned | Fine-tuned openbmb/MiniCPM-V-4.6 on RuneLang |
|
| 682 |
| Off-Brand | Custom Gradio UI that looks like a tiny cursed dungeon |
|
| 683 |
| Field Notes | Blog post explaining RuneLang and training process |
|
| 684 |
| Open Trace | Publish sample game traces / spell logs |
|