Post
493
just published a short article about something that bit me hard while porting PI05ās subtask prediction to PyTorch: left vs right alignment in transformer padding.
turns out JAX (what Physical Intelligence used) and Hugging Face use opposite padding conventions ā and if you donāt catch it, your model silently produces nonsense instead of crashing. no NaN, no error, just garbled subtasks š¤”
i walk through the full tensor pipeline ā images ā embeddings ā pad masks ā attention masks ā position IDs ā and show exactly where the mismatch corrupts everything. also included the implementation file with the fix.
if youāve ever ported a model between frameworks or messed with custom attention patterns, i think you will enjoy it
turns out JAX (what Physical Intelligence used) and Hugging Face use opposite padding conventions ā and if you donāt catch it, your model silently produces nonsense instead of crashing. no NaN, no error, just garbled subtasks š¤”
i walk through the full tensor pipeline ā images ā embeddings ā pad masks ā attention masks ā position IDs ā and show exactly where the mismatch corrupts everything. also included the implementation file with the fix.
if youāve ever ported a model between frameworks or messed with custom attention patterns, i think you will enjoy it