PixelModel 🖼️

A neural network where the weights are the image.

🧪 Dataset vs Outputs

Ground truth dataset images compared with generated outputs.

Red	Green	Blue
dataset Model output	dataset Model output	dataset Model output
White	Yellow	Dark
dataset Model output	dataset Model output	dataset Model output

What is this?

model.png is not a picture of anything. It is the model. Every pixel’s RGB values encode neural network weights:

• R channel Model output weight magnitude
• B channel Model output weight sign (≥128 = positive)
• G channel Model output bias values

At inference time, pixels are parsed into 3 weight matrices (a tiny MLP). The prompt is embedded into a vector, then a forward pass generates a 32×32 image.

Training directly optimizes pixel values via gradient descent until the PNG itself becomes the model.

📁 Files

model.png       ← THE MODEL (64×3200 px, ~284 KB)
main.py         ← inference
train.py        ← training
model.py        ← architecture (pixels Model output weights Model output forward pass)
dataset/        ← training data
  cat.png
  cat.txt       ← prompt: "a cat"
...
  

⚙️ Usage

Train

python train.py
python train.py --epochs 500 --lr 0.05
  

Generate

python main.py "red"
python main.py "a cat" --out cat_out.png --scale 8
  

--scale 8 upscales 32×32 Model output 256×256 using nearest-neighbour interpolation to preserve pixel structure.

🧠 Architecture

prompt string
  Model output char-level embedding Model output 32-dim vector
  Model output W1 (64×32) Model output tanh
  Model output W2 (64×64) Model output tanh  
  Model output W3 (3072×64) Model output sigmoid
  Model output reshape Model output 32×32×3 image
  

All weights live inside model.png. Opening the PNG is literally opening the neural network.

📊 Dataset Tips

• 6–20 image-prompt pairs is enough
• Simple targets converge fastest (solid colors, gradients, shapes)
• 200–500 epochs typically sufficient
• Loss below 0.001 is good for simple datasets
• Model capacity is fixed (~600K implicit parameters)

Why?

It’s a toy. It’s not useful. But it’s cool that it works.

Seton Labs

Coordinate, evaluate, upgrade