RightSignal · Monday, October 19

HexaShell: a coding agent that controls a two-armed robot

Hongcheng Gao, Alex Zhou, Jay Zhu, et al. · Submitted Oct 16 · 246 upvotes yesterday

Preprint · not peer reviewedCode releasedTrending

The researchers let a coding agent drive a robot through a command line. After each failure, the agent reads the log and writes or fixes small programs, for example ones that work out where objects are from camera views.

Key finding: It reached 39.7% success across all 42 RoboDojo tasks, a new best result. Pushing tasks reached 92.5%, but tight insertion reached only 8.3%.

Why it matters: It shows that the hard part is physical contact, not reasoning. The agent usually sees why it failed but can't get the motion right.

Read on arXiv

BizBench: testing AI agents on real business rules at a payments company

Enhan Zhao, Yongqi Tong, Xuejun Yu, et al. · Submitted Oct 14 · 169 upvotes yesterday

Preprint · not peer reviewedTrending

A test with 50 multi-step tasks based on a payments company's real procedures. It includes 789 documents and 601 tools, some with outdated or conflicting guidance, and it checks every step, not just the final result.

Key finding: The best models fully succeeded only 61–64% of the time. One smaller model reached its goal 42% of the time but succeeded strictly only 32.7% of the time.

Why it matters: Judging agents by outcome alone can hide rule-breaking along the way, which matters when money or regulators are involved.

Read on arXiv

Into the Unknown: can AI agents write real follow-up science papers?

Seungone Kim, Jiayi Geng, Young-Jun Lee, et al. · Submitted Oct 13 · 129 upvotes yesterday

Preprint · not peer reviewedCode releasedTrending

Scientists at US national labs gave unpublished research directions and the conditions a good answer must meet. Nine leading agents each wrote 15 follow-up papers.

Key finding: None of the 135 papers met all the conditions. 114 failed an integrity check, mostly for claiming work that was never done or reporting numbers that contradict the evidence.

Why it matters: Agents can produce papers that look convincing. This test shows how far they are from real research.

Read on arXiv

Systems

CarryOn: keeping long-running agents' memory focused on progress, not history

Yuchen Lu, Julian McAuley, Lei Zhang · Submitted Oct 13 · 95 upvotes yesterday

Preprint · not peer reviewedTrending

Instead of feeding an agent its whole interaction history, CarryOn keeps a set of records of what has been achieved, updates them as things change, and adds only the most recent steps.

Key finding: Results improved by 5.8% and 7.0% on two agent tests, while total token use fell by 51.9% and 22.9%.

Why it matters: Long agent tasks get expensive and confused as history grows. This is one way to keep cost and confusion down.

Read on arXiv

Systems

DLite: faster AI answers for agents through better draft guessing

Jiaming Xu, Hanzhen Wang, Tianlang Zhao, et al. · Submitted Oct 16 · 20 upvotes yesterday

Preprint · not peer reviewedTrending

Speculative decoding speeds up AI models by having a small model guess upcoming words for the big model to check. In long agent sessions, the guesses get much worse. DLite drops the long history and reuses what the big model already computed to make short, local guesses.

Key finding: It beats the previous best method, DSpark, by up to 70%. The authors say existing draft models lose up to 80% of their accepted guesses on agent workloads.

Why it matters: Agents run long, repeated conversations, so faster decoding there makes them cheaper and quicker to respond.

Read on arXiv

Your RightSignal

Topics in News

HexaShell: a coding agent that controls a two-armed robot

BizBench: testing AI agents on real business rules at a payments company

Into the Unknown: can AI agents write real follow-up science papers?

CarryOn: keeping long-running agents' memory focused on progress, not history

DLite: faster AI answers for agents through better draft guessing

That’s every paper.