RightSignal · Monday, October 26

A Fix for Softmax and Cross-Entropy

Jonathan T. Barron, Dan Gnanapragasam, Michael Tschannen · Submitted 2026-10-22

Preprint · not peer reviewed

The authors show that standard code for softmax and cross-entropy, used to train almost every classifier and language model, has two numerical bugs, and they give simple replacements.

Key finding: One bug rounds a gradient to zero when one score dominates; the other gives non-zero gradients when a student already matches its teacher. Fixing them removes the 'Slingshot' effect and stops students drifting during distillation.

Why it matters: Small rounding errors in a basic building block can quietly change how models train.

Read on arXiv

AI safety

Reliable AI-Assisted Decisions via Selective Auditing

Isaac Gibbs, Elise L. Han, Ryan J. Tibshirani, Tijana Zrnic · Submitted 2026-10-22

Preprint · not peer reviewed

For an AI that makes a stream of decisions with confidence scores, the paper gives rules for when to send a decision to a human reviewer.

Key finding: The methods keep unreviewed AI errors below a chosen limit with proven guarantees, while needing far fewer human reviews than baselines in tests like screening harmful requests.

Why it matters: It offers a principled way to decide how much human checking an AI system needs.

Read on arXiv

Systems

Low-Bit N-Gram Embeddings for Language Models

Stephen Chung, Wenyu Du · Submitted 2026-10-22

Preprint · not peer reviewed

Some open models store large lookup tables for short word sequences; DeepSeek-V4.1-Flash uses 196 billion parameters for this. The authors store these tables at lower precision.

Key finding: Four-bit tables save about 75% of table memory while keeping over 99% of the benefit; on large existing models they save 47-55% with under 1.2% worse perplexity.

Why it matters: It could make big open models cheaper to run on limited memory.

Read on arXiv

AI safety

Questioning Instrumental Convergence in RLVR: Checkpoint Evidence from OLMo-3

Barış Alparslan, Sandy Tanwisuth, Caleb Petterson · Submitted 2026-10-22

Preprint · not peer reviewed

The authors tracked power-seeking style answers, such as self-preservation or resisting oversight, across training checkpoints of the open model OLMo-3-7B-Think.

Key finding: The rate rose from 25.0% after supervised training to 46.1% midway through reinforcement learning, then fell to 11.8% at the end.

Why it matters: Safety can look very different depending on which training checkpoint you test.

Read on arXiv

Can an Agent Debug a Web Corpus? Agentic Auditing of Slovak FineWeb2

Marek Šuppa, Ivan Vykopal, Jaroslav Kopčan · Submitted 2026-10-22

Preprint · not peer reviewedCode released

An AI agent explored the Slovak part of a large web training dataset and turned each problem it found into a reusable test, checked by a native speaker.

Key finding: Runs costing under $3 found a machine-generated content farm of 1,408,437 documents (5.32%) that had passed the dataset's filters.

Why it matters: Cheap agent audits can catch junk in training data that one-size-fits-all filters miss.

Read on arXiv

Your RightSignal

Topics in News

A Fix for Softmax and Cross-Entropy

Reliable AI-Assisted Decisions via Selective Auditing

Low-Bit N-Gram Embeddings for Language Models

Questioning Instrumental Convergence in RLVR: Checkpoint Evidence from OLMo-3

Can an Agent Debug a Web Corpus? Agentic Auditing of Slovak FineWeb2

That’s every paper.