Anastasia Kozlova

Programmatic Labeling and Expert Data Pipelines: Engineering and Managing Data for Specialized AI

Abstract:

Advances in generative AI have shown that models are only as good as the data behind them, yet most enterprises still rely on labour‑intensive labeling processes and generic evaluation methods. Building trustworthy, high‑impact AI systems requires a shift toward data‑centric program management that combines expert knowledge, scalable pipelines and rigorous evaluation. In this talk, Anastasia Kozlova draws on her experience leading hardware programs at Apple and building data pipelines at Snorkel AI to outline how programmatic labeling and expert data‑as‑a‑service are unlocking the next frontier of agentic AI. She explains how programmatic labeling functions allow organizations to capture domain expertise and generate large training sets quickly (Team Snorkel, 2022); why scalability, adaptability and governability are critical benefits (Team Snorkel, 2022); and how specialized evaluation frameworks can close the gaps between simple “LLM‑as‑a‑judge” metrics and the stringent requirements of high‑stakes domains (Ratner, 2025). Through case studies from Apple’s MacBook Air programme and Mercedes‑Benz’s “Hey Mercedes” assistant, she illustrates how cross‑functional communication, design thinking and AI‑driven workflows reduce risk, accelerate delivery and safeguard billions in revenue. Finally, the talk highlights emerging practices for building expert data pipelines and evaluation datasets at scale, drawing on Snorkel’s latest data development platform and early enterprise results (Ratner, 2025). Attendees will leave with actionable strategies for aligning communication, computing and data analytics to build reliable, specialized AI systems.

Profile:

Anastasia Kozlova is a data‑centric engineering leader with deep experience bridging product, engineering and AI. As a Technical Delivery Manager at Snorkel AI, she oversees data‑pipeline programmes that transform domain expertise into specialized training and evaluation datasets. Previously she spent three years as a Software Systems Engineering Program Manager at Apple Inc., where she led the product development lifecycle for new Mac and iPhone hardware. She coordinated eight cross‑functional teams and more than 100 stakeholders, delivering the MacBook Air, Mac mini, Mac Studio and iPhone 15 Pro on schedule. Anastasia spearheaded the resolution of critical thermal and fan issues in the Mac mini, aligning engineering, QA and manufacturing teams to ship on time and enable roughly US$700 million in revenue. She also introduced LLM‑assisted bug triage and summarization for program management, reducing manual workload by about 30 percent and accelerating decision‑making. Her work on a unified product‑quality dashboard improved release stability and helped prevent critical rollbacks. 
Before Apple, Anastasia was a software engineer at Mercedes‑Benz Research & Development North America, where she designed and shipped more than 20 voice‑AI features for the “Hey Mercedes” in‑car assistant. She optimized 
natural‑language‑processing pipelines to reduce load time by around 20 percent and latency by 15 percent, and served as Scrum Master for a cross‑functional team, increasing sprint velocity by 20 percent. She also managed cross‑country teams for Daimler AG’s CRM and digital‑commerce initiatives. Anastasia holds a Master’s in Computer Science from Westcliff University and a Bachelor’s in Communication Studies & Computer Science from Freie Universität Berlin. She is fluent in English, German and Russian and is certified as a Scrum Master. 
Her current work focuses on democratizing expert data pipelines and evaluation frameworks for specialized AI, drawing on research‑led innovations at Snorkel AI (Ratner, 2025).