MLX: Opinionated ML Pipelines in MLflow // Xiangrui Meng // Coffee Sessions #112

MLOps Coffee Sessions #112 with Xiangrui Meng, Principal Software Engineer of Databricks, MLX: Opinionated ML Pipelines in MLflow co-hosted by Vishnu Rachakonda. // Abstract MLX is to enable data scientists to stay mostly within their comfort zone utilizing their expert knowledge while following the best practices in ML development and delivering production-ready ML projects, with little help from production engineers and DevOps. // Bio Xiangrui Meng is a Principal Software Engineer at Databricks and an Apache Spark PMC member. His main interests center around simplifying the end-to-end user experience of building machine learning applications, from algorithms to platforms and to operations. // MLOps Jobs board   https://mlops.pallet.xyz/jobs MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links Good Strategy Bad Strategy: The Difference and Why It Matters book by Richard Rumelt: https://www.amazon.com/Good-Strategy-Bad-Difference-Matters/dp/0307886239 --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Vishnu on LinkedIn: https://www.linkedin.com/in/vrachakonda/ Connect with Xiangrui on LinkedIn: https://www.linkedin.com/in/mengxr/ Timestamps: [00:00] Introduction to Xiangrui Meng [00:39] Takeaways [02:09] Xiangrui's background [03:38] What kept Xiangrui in Databricks [07:33] What needs to be done to get there [09:20] Machine Learning passion of Xiangrui [11:52] Changes in building that keep you fresh for the future [14:35] Evolution core challenges to real-time and use cases in real-time [17:33] DevOps + DataOps + ModelOps = MLOps [19:21] MLFlow Support [21:37] Notebooks to production debates   [25:42] Companies tackling Notebooks to production [27:40] MLOoops stories [31:03] Opinionated MLOps productionizing in a good way [40:23] Xiangrui's MLOps Vision [44:47] Lightning round [48:45] Wrap up