Alibaba’s Open Code Review finished reviews faster and used roughly one-ninth the tokens of general-purpose agents in a 200-pull-request test, but detected a smaller share of known issues.
The tool ranked higher in precision and F1 than its general-purpose peers, but it had lower recall. Precision reflects how often flagged items are valid; recall measures how much of the known issue set gets caught. It favored avoiding false-positive alerts, even though it caught a smaller share of known issues. For software teams weighing automated reviews, the comparison puts speed and token use alongside the completeness of issue detection.
The test spanned 50 open-source projects and 10 languages, using 200 pull requests. More than 80 senior engineers cross-validated the dataset’s 1,505 annotated issues. The evaluation assessed whether the tool could identify coding problems across varied codebases. Compared with the tool, general-purpose agents, including Claude Code, used about nine times as many tokens.
Written mainly in Go, Open Code Review combines deterministic pipelines and a large language model; it attaches comments to individual code lines. Models from OpenAI and Anthropic are supported; ocr review handles pull-request changes, while ocr scan checks complete files. Rules built into the tool flag null pointer exceptions, thread-safety issues, cross-site scripting and SQL injection. The project is licensed under Apache License 2.0 and distributed through npm.
On Saturday, Nov. 14, GitHub listed 51,009 stars and 3,804 forks for the project, following a one-day increase of 897 stars.
