Investing in AI Models: Advantages and Costs

The Fierce Competition Among AI Models: What's at Stake?

The competition between cutting-edge AI models from American giants and open-source models from Chinese companies is more intense than ever. A report from Mozilla reveals that the performance gap between these models has shrunk to just 4.4 months. This explains why many companies are migrating to open models, which are significantly cheaper for routine tasks. However, there is still a niche where cutting-edge models justify the extra cost.

The "State of Open Source AI" report from Mozilla, published in September, highlights that most organizations should use open models as the standard for the majority of their activities. One example is the Kimi K3 model from Moonshot AI, which nearly matches the performance of the closed Fable 5 model from Anthropic, but costs only 30% of the price. According to Raffi Krikorian, CTO of Mozilla, the value of closed models lies in specific tasks, such as complex professional work and intensive data retrieval.

The Choice Between Open and Closed Models

Open-source models allow anyone to download their core components and run them on their own computers. However, vital information, such as training data and code, is often kept confidential. On the other hand, American companies like Anthropic and OpenAI offer closed models that keep everything proprietary, charging more for that access.

Despite the cost, many organizations still pay for closed models because they work immediately and come with compliance and support packages. However, as the performance gap between open and closed models narrows, the cost-effectiveness of open models is being widely recognized. Companies like DoorDash are already using Kimi for routine tasks, reserving Fable for more complex challenges.

Time as a Decisive Factor in Choice

Performance differences are measured in various ways. For example, METR defines the "time horizon" of an AI model as the duration of tasks it can perform with a 50% success rate. Currently, the best closed model can handle tasks 1.7 times longer than the best open model. But this difference is rapidly decreasing.

Closed models still have an advantage in tasks that require between eight and 12 hours, while open models still cannot handle that. However, for shorter tasks, open models are already a viable and more economical option. The question is whether to pay for the closed model only when that time advantage really matters, such as in tight deadlines.

The Impact of Open Models on the Market

The popularity of open-source models is growing, but they still lag behind closed models in terms of revenue. A study by the Linux Foundation showed that open models accounted for only 4% of total revenue, compared to 96% for closed models. However, this difference is expected to decrease as open models gain more traction.

Interestingly, most open models come from China, while closed models are dominated by American companies. This raises concerns about the concentration of power and the need for labs in the US and Europe to compete on the same open field.

Lessons from Open Source Software Development

The development of open-source software, such as Linux, can offer valuable lessons for creating an alternative coalition that supports more open AI models. This coalition would likely include mission-driven institutions rather than market-driven ones and would need public funding for open computing programs.

Krikorian suggests that this coalition goes beyond open-source models, adopting a broader open approach. After all, it is difficult to fully trust a model if we do not know how it was trained or evaluated. Transparency and collaboration may be key to a more balanced and accessible AI ecosystem.

The landscape of artificial intelligence is constantly evolving, and the choice between open and closed models depends on many factors. But one thing is certain: the debate over cost, performance, and accessibility is far from over.