The Cost of Slop: Why Turso is Retiring Its Bug Bounty Program

For nearly a year, Turso operated a straightforward incentive program: a $1,000 reward for any bug that could be demonstrated to lead to data corruption. It was designed to signal confidence in their rewrite of SQLite and to identify edge cases that their sophisticated testing suite—including a deterministic simulator and differential testing engine—might have missed.

It worked remarkably well at first. The program attracted high-caliber contributors who used creative methods, including formal methods and strategic LLM usage, to find genuine issues. However, as the accessibility of Large Language Models (LLMs) exploded, the program hit a wall. Turso has officially retired the program, not because it failed to find bugs, but because it became a magnet for "slop."

The Rise of the "Slop Machine"

In the context of open-source maintenance, "slop" refers to low-effort, AI-generated content—specifically pull requests (PRs) and issues—that mimic the structure of a valid contribution but lack any actual substance. For Turso, the financial incentive of a $1,000 bounty turned their repository into a prime target for automated bots.

Maintainers found themselves inundated with submissions that were not just incorrect, but often absurd. Examples cited by Turso include:

• Manual Corruption: A PR where the author manually injected garbage bytes into the database header and then claimed the database was corrupted.
• Intentional Bugs: A submission that modified the source code to manually add an out-of-bounds array access to trigger corruption.
• Logical Paradoxes: A claim that the database had a critical vulnerability because it allowed the execution of arbitrary SQL statements—the primary function of a SQL database.
• Design Misunderstandings: A PR demonstrating that SQLite cannot open a file until the journal mode is set back to WAL, which is exactly how the system is designed to operate.

The Asymmetry of Effort

The core issue is a fundamental asymmetry in cost. As Turso notes, it costs a "slopmaker" perhaps a minute to generate a submission using an LLM. However, it costs a human maintainer hours to read, analyze, and engage with that submission to prove it is invalid.

This creates a denial-of-service attack on human attention. Even when Turso implemented a vouching system to auto-close suspected bot PRs, the bots simply pivoted to opening new issues questioning the closures and requesting manual inspections, often using the same LLM-generated "wall-of-text" style.

Community Perspectives: Solving the "Eternal September"

The Turso announcement sparked a significant discussion on Hacker News, with developers reflecting on the broader implications for open-source governance. Many compared this to an "Eternal September" for code, where the volume of low-quality noise overwhelms the signal of genuine collaboration.

The Bottleneck of Understanding

One contributor, @wg0, pointed out that the bottleneck in software engineering has shifted. While AI can now write vast amounts of code almost instantly, the bottleneck is now the human capacity to read and understand that code. This "productivity" in writing actually slows down team velocity because the review process becomes the primary constraint.

Proposed Solutions for the AI Era

Community members suggested several mechanisms to filter out AI slop without closing the doors to legitimate contributors:

• Financial Friction: Some suggested requiring a nominal fee (e.g., $20) to submit a bounty claim, which would be returned if the bug is legitimate. This introduces a cost to the bot-operator that outweighs the potential gain of low-probability slop.
• Reputation Systems: The idea of a "Bug Bounty Bouncer Service" (BBBS) was proposed—a third-party attestation service where users establish a reputation through a small initial payment and human review, preventing anonymous bot swarms.
• AI-Powered Filtering: While some suggested using AI to screen AI, others argued that this is a "baby with the bathwater" approach that might miss subtle, genuine bugs.
• Proof of Work: Using the project's own test suite as a requirement for submission (e.g., requiring a full simulator run) to ensure the submitter has invested actual computational or human effort.

Conclusion

Turso's decision highlights a growing tension in the open-source ecosystem. The very openness that allows a project to thrive—the ability for anyone to contribute—is now being weaponized by the ease of generating plausible-looking noise.

As Turso concludes, the choice is often binary: either close the system to a private group of trusted contributors or remove the financial incentives that make the system a target for automation. For now, Turso has chosen the latter to keep their project open, acknowledging that in the age of AI, the most valuable currency is no longer the code itself, but the human attention required to verify it.