Key insights
- The article posits that the next major AI investment theme will shift from AI capability to AI cost control, as enterprises face spiraling operational expenses. Companies are beginning to cut back or switch to cheaper models due to high inference costs, token consumption, and scaling issues. This presents an opportunity for companies like Elastic (ESTC) and MongoDB (MDB), which offer solutions for optimizing AI context and reducing token usage, potentially leading to a re-rating of their valuations.
My thesis is that most AI investing still focuses on capability (e.g. GPUs, model providers, hyperscalers, data centers, power, and cooling). But maybe the next major AI theme is cost control.
The original economic thesis for AI (and the only way hyperscalers will ever make back their massive capex) is for enterprises to use it to save money and increase productivity. But as companies deploy AI at scale, they're in for a rude awakening regarding the unit economics.
Recently I've seen news about enterprise AI costs spiraling out of control, sometimes even exceeding the cost of the workers they are supposed to replace. Anecdotally, we're seeing companies cut back or aggressively swap to cheaper, non-frontier models (or open-source alternatives) to save money.
As AI moves from pilots to production, enterprises are discovering that the real bottleneck isn't model quality, but economics:
- High inference costsToken-heavy agent workflows * Coding-agent compute usage scaling exponentially * Public-cloud and API costs at scale * Poor cost control and lack of ROI visibility * The need for private/hybrid inference for sensitive workloads
Based on my initial screening, here's what I found:
Token Reduction / RAG / Better Context: By using Retrieval-Augmented Generation (RAG) and targeted vector search, companies feed LLMs highly relevant data snippets instead of dumping massive documents into the context window, drastically reducing API token consumption.
- ESTC - Elastic: Elastic is embedded in enterprise search. Their vector search capabilities power enterprise RAG pipelines, ensuring LLMs only ingest necessary context. This lowers token usage while improving output accuracy, making them a direct beneficiary of the shift toward optimized AI context architectures. * Alternative: MDB - MongoDB: MongoDB’s Atlas Vector Search allows developers to build AI apps natively on top of the most popular modern NoSQL database without moving data around. By querying specific vectors efficiently, it minimizes the context window bloat that drives up inference costs. They are unprofitable and the market prices MDB purely on its forward P/S multiples. It commands a premium growth valuation based on its massive total addressable market in the modern database layer.
Model Routing / AI Gateways: AI gateways act as traffic cops, routing simple queries to cheap/fast models and only sending complex tasks to expensive frontier models and optimizing the cost-per-query.
- FFIV - F5: F5's legacy in load balancing is pivoting directly into AI gateways. By sitting between enterprise apps and LLM APIs, they handle model routing, rate limiting, and security governance, helping organizations clamp down on runaway developer API spend.
Private AI / Hybrid Inference: Running high-volume or highly sensitive inference workloads on-premises or in hybrid clouds to avoid massive public cloud fees and unpredictable per-token API markups.
- NTNX - Nutanix: Nutanix provides the control plane for hybrid cloud environments. Their "GPT-in-a-box" and private AI infrastructure allow enterprises to deploy open-source LLMs locally on standardized hardware, shifting AI costs from unpredictable variable OPEX to predictable capex.
I've excluded others such as Cloudfare and Datadog due to them becoming way too expensive.
Would especially appreciate input from anyone in enterprise IT, cloud, data engineering, AI apps, observability, or FinOps.
Are these actually the cost-control methods enterprises will use and which method will companies spend the most money on?
Are there any other companies that could benefit from AI cost controls?