The Role of Infrastructure in ML // Niels Bantilan // #197

MLOps podcast #197 with Niels Bantilan, Chief Machine Learning Engineer at Union, The Role of Infrastructure in ML Leveraging Open Source brought to us by Union. // Abstract When we start out building and deploying models in a new organization, life is simple: all I need to do is grab some data, iterate on a model that fits the data well and performs reasonably well on some held-out test set. Then, if you’re fortunate enough to get to the point where you want to deploy it, it’s fairly straightforward to wrap it in an app framework and host it on a cloud server. However, once you get past this stage, you’re likely to find yourself needing: More scalable data processing framework Experiment tracking for models Heavier duty CPU/GPU hardware Versioning tools to link models, data, code, and resource requirements Monitoring tools for tracking data and model quality There’s a rich ecosystem of open-source tools that solves each of these problems and more: but how do you unify all of them together into a single view? This is where orchestration tools like Flyte can help. Flyte not only allows you to compose data and ML pipelines, but it also serves as “infrastructure as code” so that you can leverage the open-source ecosystem and unify purpose-built tools for different parts of the ML lifecycle on a single platform. ML systems are not just models: they are the models, data, and infrastructure combined. // Bio Niels is the Chief Machine Learning Engineer at Union.ai, and core maintainer of Flyte, an open-source workflow orchestration tool, author of UnionML, an MLOps framework for machine learning microservices, and creator of Pandera, a statistical typing and data testing tool for scientific data containers. His mission is to help data science and machine learning practitioners be more productive. He has a Masters in Public Health with a specialization in sociomedical science and public health informatics, and prior to that a background in developmental biology and immunology. His research interests include reinforcement learning, AutoML, creative machine learning, and fairness, accountability, and transparency in automated systems. // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links Website: https://github.com/cosmicBboy, https://union.ai/Flyte: https://flyte.org/ MLOps vs ML Orchestration // Ketan Umare // MLOps Podcast #183 - https://youtu.be/k2QRNJXyzFg ⁠ --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Niels on LinkedIn: https://www.linkedin.com/in/nbantilan/ Timestamps: [00:00] Niels' preferred coffee [00:17] Takeaways [03:45] Shout out to our Premium Brand Partner, Union! [04:30] Pandera [08:12] Creating a company [14:22] Injecting ML for Data [17:30] ML for Infrastructure Optimization [22:17] AI Implementation Challenges [24:25] Generative DevOps movement [28:27] Pushing Limits: Code Responsibility [29:46] Orchestration in OpenAI's Dev Day [34:27] MLOps Stack: Layers & Challenges [42:45] Mature Companies Embrace Kubernetes [45:29] Horizon Challenges [47:24] Flexible Integration for Resources [49:10] MLOps Reproducibility Challenges [53:14] MLOps Maturity Spectrum [57:48] First-Class Citizens in Design [1:00:16] Delegating for Efficient Collaboration [1:04:55] Wrap up