In-House LLM Serving at Netflix

T1Covered by 1 source · first reported by Stripe / Shopify Eng / Netflix TechBlog / Dropbox Tech / Slack Eng / Spotify Eng at 17 Jul, 21:32 UTC

By AI Platform’s Model Runtime team and Inference team Introduction Most organizations consume LLMs through hosted APIs. Netflix went further — we run the full stack ourselves, from model deployment through inference, inside our existing production environment rather than a separate ML silo. Some…

aidev

Sources

• T1In-House LLM Serving at NetflixStripe / Shopify Eng / Netflix TechBlog / Dropbox Tech / Slack Eng / Spotify Eng17 Jul, 21:32