Over 500 billion AI tokens pass through Portkey every single day.

Every AI product eventually runs into the same problem. The prototype works, but once it goes live the system has to manage multiple models, rising token costs, unpredictable latency, and infrastructure that was never built for AI workloads.

That is the problem Rohit Agarwal is solving with Portkey, an AI gateway that sits between applications and the models, whether that’s GPT-4, Claude, or Gemini.

With 24,000 companies routing their AI through Portkey, Rohit sits on ground-level data on how AI is actually being used in production. Which models enterprises are betting on. Where costs are quietly climbing. How usage patterns shift as companies move from pilots to real products.

When AI spend surpasses cloud spend, and Rohit believes it will, the infrastructure running underneath it becomes one of the most important bets in tech. This episode explores what it takes to run AI systems at that scale.

00:00 – Trailer
01:05 – 500 billion AI tokens every day
04:05 – First to call an "AI gateway"
07:26 – Where did the Gateway insight come from?
12:08 – How Portkey is winning this space
13:05 – Picking the right gambles over wrong ones
14:16 – What are LLM endpoints?
15:21 – AI will 100% surpass cloud spend
19:00 – Hype is coming from people still in Q&A mode
19:33 – AI employees over humans in customer support?
23:00 – For AI startups, traffic > revenue
24:43 – The bubble is in valuations, not utility
26:05 – How Rohit built his personal automations
28:38 – Costliest model is most used now
33:21 – What's going right and wrong for AI companies
37:49 – Hiring a VP of sales after $15M is possible today
39:57 – What edge does Claude have over other models?
43:35 – Founders need a "why me vs. why Anthropic" story
52:56 – What if Anthropic or AWS builds a gateway?
55:41 – Predictions for the next 12 months
59:40 – How big is the opportunity in Agents?
01:00:48 – Startups now have to prove it's not a weekend project
01:01:54 – Is Build v/s Buy no longer a Debate?
01:03:50 – What would Rohit build if starting up today?
01:05:58 – How Portkey is different from an API gateway
01:08:26 – MCP / tool calling enables agentic workflows
01:12:00 – Portkey's Community-driven early GTM
01:13:34 – Startups have only 2 reasons for Open core

-------------

India’s talent has built the world’s tech—now it’s time to lead it.
This mission goes beyond startups. It’s about shifting the center of gravity in global tech to include the brilliance rising from India.

What is Neon Fund?
We invest in seed and early-stage founders from India and the diaspora building world-class Enterprise AI companies. We bring capital, conviction, and a community that’s done it before.

Subscribe for real founder stories, investor perspectives, economist breakdowns, and a behind-the-scenes look at how we’re doing it all at Neon.

-------------

Check us out on:
Website: https://neon.fund/
Instagram: https://www.instagram.com/theneonshoww/
LinkedIn: https://www.linkedin.com/company/beneon/
Twitter: https://x.com/TheNeonShoww

Connect with Siddhartha on:
LinkedIn: https://www.linkedin.com/in/siddharthaahluwalia/
Twitter: https://x.com/siddharthaa7

-------------

This video is for informational purposes only. The views expressed are those of the individuals quoted and do not constitute professional advice.

Send us Fan Mail

How 24,000 companies keep their AI from Breaking in Production | Rohit Agarwal, Portkey

The Neon Show