Author here, coding agents on C/C++ repos still have <10% acceptance on PRs. This problem is amplified on private repos which are not in models' training data. Anthropic released a clangd plugin for this. These experiments included vanilla claude code, vs with the plugin + pre-built index + instructions to use it on 255 issues from 40 C/C++ repos, including the bugs fixed after the model’s training cut off date. Couple of interesting findings:
The model prefers to use grep disproportionately. When LSP is optional, agent called it in 5% of runs
When required, 89% calls were plain name lookups and still 70% of the runs used grep
Fixes are close to flat (270 vs 275 out of 359 repair attempts) and cost went up 11%.
From the issue text alone, with no repo, the model already knows every fix site for 126 out of 255 issues.
A caveat here is the setup tests this workflow and instructions. Has anyone seen LSP clearly help an agent on large codebases work on end to end tasks?
Author here, coding agents on C/C++ repos still have <10% acceptance on PRs. This problem is amplified on private repos which are not in models' training data. Anthropic released a clangd plugin for this. These experiments included vanilla claude code, vs with the plugin + pre-built index + instructions to use it on 255 issues from 40 C/C++ repos, including the bugs fixed after the model’s training cut off date. Couple of interesting findings:
The model prefers to use grep disproportionately. When LSP is optional, agent called it in 5% of runs
When required, 89% calls were plain name lookups and still 70% of the runs used grep
Fixes are close to flat (270 vs 275 out of 359 repair attempts) and cost went up 11%.
From the issue text alone, with no repo, the model already knows every fix site for 126 out of 255 issues.
Code and sample tasks: https://github.com/ByteAsk/lsp-claude-code-cpp
A caveat here is the setup tests this workflow and instructions. Has anyone seen LSP clearly help an agent on large codebases work on end to end tasks?
[flagged]