AI Challanges
Collection
4 items • Updated • 14
ginipick
ginipick
AI & ML interests
None yet
Recent Activity
reacted to SeaWolf-AI's post with 🔥 3 days ago
🔍 The attention mask stopped being an audit.
An autoregressive model must not let position t depend on anything after t. Everyone checks this by inspecting the causal mask — but hybrid stacks now mix attention with state-space scans, and a scan has no mask. Every mask can be correct while information leaks through scans, aggregations, or normalization.
⚙️ So we test the property directly. Two inputs identical except at the last position, two forward passes, compare each layer's prefix, report the first layer that moves. No training, no gradients, no accelerator — seconds on CPU.
📊 Across 192 injected faults on eight checkpoints, mask inspection detected 0. The per-layer audit localized 192/192 to the exact layer.
🎯 Then we read the source before running anything. In transformers 5.7.0, the reference chunked scan reduces the inter-chunk recurrence over the input chunk axis; zamba2 and nemotron_h reduce over the output chunk axis. One axis. The dynamic audit confirmed the prediction exactly: Zamba2-1.2B leaks from length 256, its declared chunk size, and Nemotron-H-8B from 128, its declared chunk size. Bamba, Falcon-H1, Granite-4.0-H, Mamba2 and RecurrentGemma came back clean.
⚠️ Scope: the defect is on the PyTorch chunked-scan path, which runs whenever the fused kernels are absent — CPU, CI, stock installs. We could not build those kernels, so the fast path is untested and open. That caveat cuts both ways: a model can pass every fused-kernel test and still leak the moment it runs without them.
🧪 AX-RAY now carries this as its own axis. 39 models scored across causal, white-box and behavioral axes: 21 A, 3 B, 1 C, 14 F — with exactly 2 Causal-LEAK verdicts, the two the paper predicted. Badges separate a weights-level audit from an API-only one, so the two never get read as the same claim.
📄 https://arxiv.org/abs/2608.22876
🔬 https://huggingface.co/spaces/FINAL-Bench/AX-RAY
🤗 https://huggingface.co/papers/2608.22876 reacted to SeaWolf-AI's post with 👍 3 days ago
🔍 The attention mask stopped being an audit.
An autoregressive model must not let position t depend on anything after t. Everyone checks this by inspecting the causal mask — but hybrid stacks now mix attention with state-space scans, and a scan has no mask. Every mask can be correct while information leaks through scans, aggregations, or normalization.
⚙️ So we test the property directly. Two inputs identical except at the last position, two forward passes, compare each layer's prefix, report the first layer that moves. No training, no gradients, no accelerator — seconds on CPU.
📊 Across 192 injected faults on eight checkpoints, mask inspection detected 0. The per-layer audit localized 192/192 to the exact layer.
🎯 Then we read the source before running anything. In transformers 5.7.0, the reference chunked scan reduces the inter-chunk recurrence over the input chunk axis; zamba2 and nemotron_h reduce over the output chunk axis. One axis. The dynamic audit confirmed the prediction exactly: Zamba2-1.2B leaks from length 256, its declared chunk size, and Nemotron-H-8B from 128, its declared chunk size. Bamba, Falcon-H1, Granite-4.0-H, Mamba2 and RecurrentGemma came back clean.
⚠️ Scope: the defect is on the PyTorch chunked-scan path, which runs whenever the fused kernels are absent — CPU, CI, stock installs. We could not build those kernels, so the fast path is untested and open. That caveat cuts both ways: a model can pass every fused-kernel test and still leak the moment it runs without them.
🧪 AX-RAY now carries this as its own axis. 39 models scored across causal, white-box and behavioral axes: 21 A, 3 B, 1 C, 14 F — with exactly 2 Causal-LEAK verdicts, the two the paper predicted. Badges separate a weights-level audit from an API-only one, so the two never get read as the same claim.
📄 https://arxiv.org/abs/2608.22876
🔬 https://huggingface.co/spaces/FINAL-Bench/AX-RAY
🤗 https://huggingface.co/papers/2608.22876 upvoted a collection 3 days ago
AI ChallangesOrganizations
upvoted a collection 3 days ago
upvoted a paper 3 days ago
upvoted a paper 4 days ago
upvoted 3
collections 13 days ago
upvoted a paper about 1 month ago
upvoted a collection about 1 month ago
upvoted an article 3 months ago
Article
FINAL-Bench Quantum: An Open, Neutral Benchmark for Quantum-Computing Methods
FINAL-Bench
• • 17 upvoted a paper 3 months ago
upvoted a collection about 1 year ago
upvoted 5
collections over 1 year ago