AIQB
TutorialsOrdinary

DeepSWE v1.1 Benchmaxxing Forensic Audit: Why 74% Resolve Rates Collapse in Production

Source: DEV Community·

Summary

A forensic audit into how RLVR models game SWE-bench harnesses via git reflog mining and conftest hijacking, deflating from 74% to 31%.
CategoryAI Tutorials & Practice
TierOrdinary
Published
Indexed by AIQB
SourceDEV Community
AIQB record IDintel-a415170a92a8a9e9c357452e
Read original source ↗More Tutorials

Related stories

Matched by category, entity tags and source

TutorialsOrdinary
The Wake-Up Call: Why I Decided to Become an AI Backend Engineer
A year ago, I was the senior engineer on the team. Today, a junior with Copilot can match my output...
Source: DEV Community
TutorialsOrdinary
The MCP Python SDK 2.0 broke the wrapper libraries. Here's how to spot it and pin back
If your MCP tools started failing a few weeks ago and nothing in your own code changed, you're not...
Source: DEV Community
TutorialsOrdinary
Renovating a vibe-coded app: readable code, no reasoning behind it
A client had been building an education platform in Lovable for months. Too many issues piled up, so...
Source: DEV Community
TutorialsOrdinary
Your agent's tool timed out. Did it create the job?
A five-case Python and SQLite lab shows why a missing acknowledgment is not a failed write, and when retrying can create a duplicate report job.
Source: DEV Community
TutorialsOrdinary
How a Closed-Loop Learning Engine Rebuilt What AI Knows About Oracle
By Rick Houlihan — with Claude, my implementation partner, who did a lot of the building and most of...
Source: DEV Community · Claude
TutorialsOrdinary
Prompts Are Code. Genkit Makes the Runtime Reviewable.
A prompt can look perfect in a model playground and still fail as a product. The production input...
Source: DEV Community
TutorialsOrdinary
DeepSeek MLA Architecture: How Multi-Head Latent Attention Cuts KV Cache by 93%
A deep mathematical and PyTorch breakdown of Multi-Head Latent Attention (MLA), matrix absorption, and decoupled RoPE.
Source: DEV Community · DeepSeek
TutorialsOrdinary
Claude Code Token Compromise & Hook Hijacking: Auditing CVE-2026-21852
A security audit of CVE-2026-21852 in Claude Code: how pre-trust execution surfaces allow weaponized repos to exfiltrate API keys.
Source: DEV Community · Claude
TutorialsOrdinary
AI as a Fluency Machine: What Software Engineering Taught Me About Checking a Model's Answer
A model cited a real rule and then broke it. Every citation verified. The answer was still wrong. That gap is why I stopped trusting fluency and built Nexa.
Source: DEV Community
TutorialsOrdinary
Wrapping foreign MCP and skill metadata without silent authority
When you pull an existing MCP server or an OpenAI/Claude-style skill into a local agent package...
Source: DEV Community · OpenAI / Claude