Ravali Kandur

When AI Meets Distributed Systems: The Engineering Behind Every Prompt

Abstract:

AI applications appear deceptively simple: a prompt goes in, an answer comes out. In production, however, every request traverses a distributed system of retrieval services, vector indexes, caches, orchestration layers, inference endpoints, streaming pipelines, and observability tooling. The reliability of an AI application is rarely determined by the model alone—it is determined by how well these distributed components work together under load, failure, and change.

This talk reframes modern AI applications through the lens of distributed systems engineering. We will deconstruct the lifecycle of an AI request, identify where latency accumulates and failures propagate, and examine architectural patterns for resilient data pipelines, retrieval systems, model routing, intelligent caching, asynchronous processing, workload isolation, and end-to-end observability — a close knit ecosystem of systems that enable AI platforms to operate reliably under increasing scale and complexity.

Attendees will leave with a practical mental model for designing AI platforms, an understanding of the distributed systems challenges hidden beneath every AI interaction, and a set of architectural principles they can immediately apply to build more scalable, reliable, and observable AI applications.

Profile:

Ravali Kandur is an Engineering Leader specializing in distributed systems, data infrastructure, and AI platforms. Throughout her career, she has designed and scaled production systems at companies including Meta, Roblox, Google, and Apple, solving complex engineering challenges across distributed data processing, cloud infrastructure, machine learning platforms, and large-scale AI systems.

Her work spans the full lifecycle of data-intensive applications—from building high-throughput streaming and batch data pipelines to architecting reliable infrastructure for machine learning and generative AI workloads. She has led engineering initiatives involving distributed storage, workflow orchestration, data quality, retrieval systems, model-serving infrastructure, and platform observability, with a strong focus on building resilient systems that operate reliably under massive scale and continuously evolving workloads.

Ravali is particularly passionate about applying classical distributed systems principles to modern AI infrastructure. As organizations increasingly transition AI applications from experimentation to production, her work focuses on solving the engineering problems that emerge beyond the model itself: reliability, scalability, fault tolerance, data freshness, operational visibility, and cost efficiency. Through technical talks and community engagement, she enjoys sharing practical architectural lessons that help engineers design AI platforms capable of supporting real-world production demands.