About
SiteGPT is a personalized chatbot that answers your visitors’ questions using a model trained specifically on your website content.
Industry
AI-powered Customer Support
Company Size
11-50 employees
Headquarters
India
Founded
2023
Why Portkey:
Multi-provider model switching, enterprise-grade observability, and failover support
requests processed
models in prod
tokens processed
Overcoming infrastructure complexities for enterprise AI
SiteGPT started in early 2023 with a simple goal: to bring generative AI to customer support. Within months, the platform evolved into a full-fledged solution used by thousands of businesses, from startups to large enterprises.
As their customer base expanded, the engineering team faced increasing infrastructure complexity. They needed to support multiple LLMs across providers, monitor usage at a granular level, and meet strict performance and compliance standards.
“When you're handling millions of requests, even a 0.1% failure rate is unacceptable. We needed infrastructure that could scale with our enterprise customers while maintaining perfect reliability.”
— Bhanu Teja Pachipulusu, Founder, SiteGPT
Building reliable infrastructure for AI support at scale
To maintain performance at scale, SiteGPT turned to Portkey as their centralized AI infrastructure layer.
Key reasons why Portkey stood out:
Seamless switching between 13+ models in production
Built-in rate limit handling for each provider
Enterprise-grade logging and observability
Metadata tagging for easy segmentation and debugging
End-to-end performance tracking and failover support
Portkey gave SiteGPT the control, visibility, and reliability required to serve enterprise-grade SLAs, while continuing to experiment with new models.
Managing complex operations with complete visibility
Portkey handles all LLM interactions across SiteGPT’s platform. Every call is logged with detailed metadata, token usage, and response data — helping the team debug, optimize, and iterate fast.
With automated failover and real-time performance tracking, SiteGPT can route requests dynamically and maintain service even when individual providers experience latency or rate limits.
That’s where Portkey plays a central role.
Prompt management: 100+ prompts are versioned in Portkey’s library. Changing tone or format for a region is as simple as updating one prompt, no need to alter n8n logic.
Full observability: Every call is logged with latency, token usage, model metadata, and output quality. The team can track underperforming calls and compare across providers.
Provider flexibility: With Portkey’s forward-compatible architecture, switching or trying out a new model takes minutes, no flow changes needed.
The impact: more reliability, less overhead
Portkey's logging capabilities have been invaluable for debugging complex tool calls and understanding user patterns. The metadata features allow us to quickly segment and analyze user behavior in ways we couldn't before.
— Bhanu Teja Pachipulusu, Founder, SiteGPT
The results have validated SiteGPT's decision to build on Portkey:
Successfully processed 6 billion tokens with high reliability
Achieved 5% reduction in support tickets for enterprise clients
Maintained enterprise-grade uptime across all deployments
Enabled complex workflow automation capabilities
SiteGPT continues to expand its AI capabilities with Portkey, focusing on testing new models while maintaining its high standards for reliability and performance. Their roadmap includes enhanced guardrails, improved CSAT collection, and deeper support workflow integration.
Lessons for AI teams scaling their operations
For teams building AI-powered customer support, SiteGPT's experience offers valuable insights:
Start with robust infrastructure - SiteGPT's ability to scale rapidly while maintaining reliability was crucial for enterprise adoption
Prioritize observability - Comprehensive logging and metadata management proved essential for debugging and optimization
Plan for scale - The ability to support multiple models and handle billions of requests requires sophisticated infrastructure from day one
Focus on reliability - Enterprise customers require perfect uptime, making automated failover and rate limit handling essential
SiteGPT’s journey proves that with the right infrastructure, AI products can scale from idea to enterprise without rewriting the backend.
