Beyond the Sigmoid: Predicting AI Capability Growth

The debate over the future of artificial intelligence often boils down to a mathematical tug-of-war: is the current trajectory of AI capability an exponential curve that will lead us to a singularity, or is it a sigmoid—a curve that starts fast but inevitably flattens into a plateau?

This question is not merely academic; it determines whether we should prepare for a world of super-intelligence or expect a steady, incremental improvement in software tools. While the intuitive response to any rapid growth is that it must eventually level off, the timing and nature of that plateau remain the most contentious points of discussion among researchers and observers.

The Sigmoid Trap and the Illusion of Plateaus

At its core, the "sigmoid" argument suggests that every technology hits a physical or logical ceiling. Whether it is the speed of aircraft or the capacity of a transistor, there is a point where diminishing returns make further progress prohibitively expensive or physically impossible.

However, a critical counter-argument is that what looks like a single sigmoid is often a series of stacked sigmoids. As one technology reaches its limit, a paradigm shift—a new discovery or a different architecture—kicks in, creating a new curve with a higher ceiling.

"You don't have one sigmoid, you have multiple each stacked on top of each other... It only looks exponential(ish) because of unpredictable discoveries that let you switch to another sigmoid that has a higher maximum potential."

In the context of AI, this has already happened. The transition from early neural networks to Transformers allowed for massive scaling. More recently, the industry has shifted from "vertical scaling" (simply adding more parameters and data) to "inference-time scaling," where models allocate more compute during the generation process to "think" and verify logic dynamically.

The Hardware and Energy Wall

While algorithmic breakthroughs can shift the ceiling, the physical infrastructure supporting AI cannot be ignored. Several technical constraints threaten to turn the current exponential growth into a hard plateau:

• Compute and Silicon: Moore's Law is widely considered to be hitting diminishing returns. While some suggest that analogue logamp matmuls or memristors could break the memory-compute barrier, the reality of building fabrication plants (fabs) is a slow, linear process that cannot scale exponentially.
• Energy Constraints: The demand for electricity to power massive data centers is hitting real-world supply chain limits. There is no exponential supply of fossil fuels or power plants, and while renewable energy is improving, it may not keep pace with the exponential demand of frontier models.
• Data Exhaustion: The "data wall"—the point where models have consumed all high-quality human-generated text—is a primary concern. While synthetic data generation is a potential workaround, its long-term viability remains unproven.

Applying Lindy's Law to AI Trends

One provocative framework for predicting the duration of a trend is Lindy's Law, which suggests that the future life expectancy of some non-perishable things (like an idea or a technology) is proportional to their current age. In other words, the longer a trend has persisted, the more likely it is to continue.

Applying this to AI suggests that if a capability trend has been growing for a decade, the default assumption should be that it will continue for roughly another decade. This is counterintuitive because we typically view time as a countdown to an end; Lindy's Law views time as a validation of robustness. However, critics argue that Lindy's Law applies to static objects or ideas, not to dynamic, resource-dependent technological trends.

The Measurement Problem: What are we actually scaling?

A recurring theme in technical critiques is the question of what "capability" actually means. Many of the graphs showing exponential growth are based on benchmarks that may not reflect true intelligence.

"We’re measuring something orthogonal, basically how well a universal function approximator can fit to a function we define, given arbitrary computing power, and calling that progress."

If AI is merely interpolating its training data rather than demonstrating true reasoning or extrapolation, the "exponential growth" we see may simply be the realization of the vast amount of human knowledge stored in digital libraries, rather than the emergence of a new form of intelligence. If the goal is "realism," the limit is reached when the AI becomes indistinguishable from reality—a ceiling that cannot be exceeded.

Conclusion: The Uncertainty of the Curve

Ultimately, the debate between exponentials and sigmoids highlights the impossibility of precise forecasting in a chaotic system. Whether the curve flattens tomorrow or continues for another twenty years, the underlying drivers—compute, data, and algorithmic efficiency—are subject to geopolitical shocks, economic collapses, and physical limits.

As one observer noted, the most rational position may not be to bet on a specific curve, but to recognize that mathematical models are tools for approximation, not prophecies. The real argument lies not in the shape of the line, but in the specific reasons why a technology may or may not emerge.