Hugging Face's logo

Join the conversation

Join the community of Machine Learners and AI enthusiasts.

Sign Up
Back to feed
AbstractPhil 
posted an update 3 days ago
Post
63
https://huggingface.co/blog/AbstractPhil/aleph-autoregressive-differentiation-ft1

After some analysis and a bit of research the upgraded aleph autoregression is capable as a prototype selection tool. I approached the direct aleph attention routing mechanism and formed a progression from it, which already provided the necessary footholds to continue into an upgraded core mechanism. The followup mechanisms show autoregression is very possible and will be simpler than expected.

The results are promising and the autoregression stable enough to scale up. Thanks to Claude Fable who is able to keep my entire research context window in scope, the progression was rapid and the results quick. The tests yielded improved accuracy over standard MLP in many cases. I believe the improvement is not topical and will scale with a bit of effort.

Fingers crossed my friends, the addressing is part of the distillation paradigm and it now learns directly without needing an expert controller. I'll be progressing the mechanism over the coming days. With enough effort and time I hope the standard mechanism becomes a universal improvement on autoregression.
AbstractPhil
•
edited 1 day ago

I left Fable unattended for 3 days, only checking back once every 6 hours or so to answer questions or select one of a few options from the next list of experiments or answering the alarm that said it was kicked over to Opus - I predominantly went with the Fable selection. I attempted to have Fable handle geometric distillation anchor implantation. Instead of sticking to the paradigm, the model defaulted to some sort of genetic and biological wordplay - I have no idea what it was based on specifically. I'm guessing I ran aground into something that wasn't helpful, but gave the impression of helpful.

This unknown divergence grew over time and I simply let it go to see what would happen. The results did not yield as expected, the model bypassed the constellation entirely and rewrote the alephs system 5 times before the results for experiment 15 and 16 were completed.

Those two are essentially experiments to see how Claude Fable would behave if left unattended. Sure the model DID in fact finish some experiments, and the results were... entirely different than the expected structural models would require. In fact, the results were almost entirely deviant while disregarding the experimental line leading to the system.

Fable may be good at running autonomously, but not good at skilled research differentiation yet. The biases from programming still creep in. I'm also surprised I didn't hit more safeguards, as they did hit a few times but I would just snap the model back over to fable and the system would continue on like it never happened.

The results are basically just, if ran would these systems outperform MLP. I gave little structure and little expert input, however I did give Fable my ENTIRE research line and everything related to the necessary systems in the use-case.

The results literally rivaled MLP, but if you inspect the code you'll find the system is essentially a decision-tree that hybridizes aleph addressing internally with a structured bypass system akin to MLP. It's essentially a controlled MLP, which is kind of okay, and it's quite different in it's own rite. However, it was not using the necessary research on many fronts, and it completely bypassed the expected tooling to train the next case, additionally the system completely disregarded the implementations built around the codebooks - instead defaulting to testing the codebooks over and over in hundreds of ways.

The codebooks are already explained, it's a sphere, the math is deterministic, and the outcomes are based on a sector of space forming infinite and finite aleph structures fused with differentiated decoupled shapes. A big knot of 5 point connections if you go looking hard enough, explicitly or implicitly. This isn't news, and somehow the model behaved as though this structure is in fact some sort of news. The codebooks are built on functional math specifically because that's how we debugged them. Fable spent 3 days figuring out what we already knew.

·
AbstractPhil
•
edited about 7 hours ago

Hard parts over now. We're making some genuine progress. I'm currently testing the modularity of the adapters and the results are PROMISING. Multiple adapters are PROMISING and are not required to be present during training, so you don't need to create a full collective TOGETHER, they can be independently trained and the decision selector gate for multi-model is currently in the planning stages.

The internal arguments for the tiny MOE are based on specific selection rules and lookup potentials that enable a sort of task-driven lookup hierarchy internally, along with a generalizable increase in accuracy for the task depending on the differentiated utilization required.

In conjunction, the anchored constellation system has also been heavily prototyped. The constellation anchoring provides the necessary generalization and contextualization capacity when attached to alephs, along with aleph addressing being utilizable for "overfitted" selector trees tuned specifically to memorize better. In conjunction the secondary tree for the constellation is meant to underfit and provide connectivity directly to the core model adapted from as well.

Each adapter at their next stage will most likely have a similar or more complex internal debate system to allow the model to select for itself which is most fit for the results autonomously. So you won't need to activate experts, but you can manually activate or deactivate them as required. The micro-MOE structure is yielding some substantially accurate gated deferral to the original model in conjunction to referral to the internal overfitted portion with the task, as well as the more generalizable portion for the adapter's capacity improvement.

It's working. The ByteLM and AlephLM are yielding fruit, and the fruit is showing the capacity for modularity. It needs many tests but the results are showing some serious promise, and the results are verifiable and testable per as per the paradigm established.
Alongside, they are LINEAR. Meaning lightning quick.
In this post