Post
440
Neural Gas is a classical unsupervised learning algorithm for vector quantization and topology learning, introduced in the early 1990s. It maintains a set of prototype vectors that move through the data space and gradually approximate the underlying distribution by ranking samples and adapting all units accordingly.
While the original formulation is algorithmically elegant, most existing implementations remain procedural and non-differentiable, which limits their integration with modern deep learning systems.
This project introduces a **differentiable** implementation of Neural Gas in PyTorch:
https://github.com/francesco-p/ngas-pytorch
The key idea is to reinterpret the update rules in a way that is compatible with autograd, allowing the algorithm to be embedded inside end-to-end trainable pipelines.
This shift enables several directions that are difficult or impossible with standard implementations:
- joint optimization of Neural Gas with neural networks
- inclusion of topology-learning modules inside differentiable models
- gradient-based tuning of algorithm parameters
- hybrid architectures combining representation learning and vector quantization
The repository provides a clean PyTorch implementation and focuses on making the core mechanism usable as a first-class differentiable component, rather than a standalone preprocessing step.
In parallel, an interactive playground was built to visualize the behavior of Neural Gas during training and better understand how prototypes adapt to the data distribution:
https://francesco-p.github.io/res/neural-gas/playground.html
The goal is to revisit a well-known algorithm and make it compatible with current machine learning workflows, where differentiability is a central constraint rather than an afterthought.
While the original formulation is algorithmically elegant, most existing implementations remain procedural and non-differentiable, which limits their integration with modern deep learning systems.
This project introduces a **differentiable** implementation of Neural Gas in PyTorch:
https://github.com/francesco-p/ngas-pytorch
The key idea is to reinterpret the update rules in a way that is compatible with autograd, allowing the algorithm to be embedded inside end-to-end trainable pipelines.
This shift enables several directions that are difficult or impossible with standard implementations:
- joint optimization of Neural Gas with neural networks
- inclusion of topology-learning modules inside differentiable models
- gradient-based tuning of algorithm parameters
- hybrid architectures combining representation learning and vector quantization
The repository provides a clean PyTorch implementation and focuses on making the core mechanism usable as a first-class differentiable component, rather than a standalone preprocessing step.
In parallel, an interactive playground was built to visualize the behavior of Neural Gas during training and better understand how prototypes adapt to the data distribution:
https://francesco-p.github.io/res/neural-gas/playground.html
The goal is to revisit a well-known algorithm and make it compatible with current machine learning workflows, where differentiability is a central constraint rather than an afterthought.
