Machine in Production = Data Engineering + ML + Software Engineering // Satish Chandra Gupta // MLOps Coffee Sessions #16

//Bio Satish built compilers, profilers, IDEs, and other dev tools for over a decade. At Microsoft Research, he saw his colleagues solving hard program analysis problems using Machine Learning. That is when he got curious and started learning. His approach to ML is influenced by his software engineering background of building things for production.   He has a keen interest in doing ML in production, which is a lot more than training and tuning the models. The first step is to understand the product and business context, then building an efficient pipeline, then training models, and finally monitoring its efficacy and impact on the business.  He considers ML as another tool in the software engineering toolbox, albeit a very powerful one.  He is a co-founder of Slang Labs, a Voice Assistant as a Service platform for building in-app voice assistants.   //Talk Takeaways ML-driven product features will grow manifold. Organizations take an evolutionary approach to absorb tech innovations. ML will be no exception. How Organizations adopted cloud can offer useful lessons. ML/DS folks who invest in an understanding business context and tech environment of the org will make a bigger impact. Organizations that invest in data infrastructure will be more successful in extracting value from machine learning.   //Other links you can check Satish on An Engineer’s trek into Machine Learning:   https://scgupta.link/ml-intro-for-developers Architecture for High-Throughput Low-Latency Big Data Pipeline on Cloud: https://scgupta.link/big-data-pipeline-architecture Data pipeline article: https://scgupta.link/big-data-pipeline-architecture or https://towardsdatascience.com/scalable-efficient-big-data-analytics-machine-learning-pipeline-architecture-on-cloud-4d59efc092b5 Tips for software engineers based on my experience of getting into ML: https://scgupta.link/ml-intro-for-developers or https://towardsdatascience.com/software-engineers-trek-into-machine-learning-46b45895d9e0 Linkedin: https://www.linkedin.com/in/scgupta Twitter: https://twitter.com/scgupta Personal Website: http://scgupta.me Company Website: https://slanglabs.in Voice Assistants info: https://www.slanglabs.in/voice-assistants Timestamps: 0:00 - Intro to Satish Chandra Gupta 1:05 - Background of Satish on Machine Learning 3:29 - Satish's background on what he's doing now 5:34 - Why were you interested in the challenges of the workload? 9:53 - As you're looking at the data pipeline, do you see much overlap there? 15:38 - Relationships between engineering pipeline characteristics and how they relate to data. 20:24 - Tips for saving when you're building these pipeline. 24:44 - First point of engagement: Collection 31:26 - Possibilities of Data Architecture 38:03 - Why is it beneficial to save money? 44:22 - Learnings of Satish with his current project, Voice Assistant as a service.