Racing the Playhead: Real-time Model Inference in a Video Streaming Environment // Brannon Dorsey // Coffee Sessions #98

MLOps Coffee Sessions #98 with Brannon Dorsey, Racing the Playhead: Real-time Model Inference in a Video Streaming Environment co-hosted by Vishnu Rachakonda. // Abstract Runway ML is doing an incredibly cool workaround applying machine learning to video editing. Brannon is a software engineer there and he’s here to tell us all about machine learning in video and how Runway maintains their machine learning infrastructure. // Bio Brannon Dorsey is an early employee at Runway, where he leads the Backend team. His team keeps infrastructure and high-performance models running at scale and helps to enable a quick iteration cycle between the research and product teams. Before joining Runway, Brannon worked on the Security Team at Linode. Brannon is also a practicing artist who uses software to explore ideas of digital literacy, agency, and complex systems. // MLOps Jobs board   https://mlops.pallet.xyz/jobs // Related Links Website: https://brannon.online Blog: https://runwayml.com/blog/distributing-work-adventures-queuing-and-autoscaling/ --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Vishnu on LinkedIn: https://www.linkedin.com/in/vrachakonda/ Connect with Brannon on LinkedIn: https://www.linkedin.com/in/brannon-dorsey-79b0498a/ Timestamps: [00:00] Introduction to Brannon Dorsey [00:56] Takeaways [05:42] Runway ML [07:00] Replacement for Imovie? [09:07] Machine Learning use cases of Runway ML [10:40] Journey of starting as a model zoo to video editor [14:42] Rotoscoping   [16:23] Intensity of ML models in Runway ML and engineering challenges [19:55] Deriving requirements [23:10] Runway's model perspective [25:25] Why browser hosting? [27:19] Abstracting away hardware [32:04] Kubernetes is your friend [35:29] Statelessness is your friend [38:17] Merge to master quickly [42:57] Brannon's winding history becoming an engineer [46:49] How much do you use Runway? [49:37] Last book read [50:36] Last bug smashed [52:21] MLOps marketing that made eyes rolling [54:11] Bullish on technology that might surprise people [54:39] Spot by netapp [56:42] Implementing Spot by netapp [56:55] How do you want to be remembered? [57:22] Wrap up