The Future of ML and Data Platforms // Michael Del Balso - Erik Bernhardsson // Coffee Sessions #57

Coffee Sessions #57 with Michael Del Balso and Erik Bernhardsson, The Future of ML and Data Platforms. // Abstract Machine learning, data analytics, and software engineering are converging as data-intensive systems become more ubiquitous.  Erik Bernhardsson, ex-CTO at Better and former Spotify machine learning lead, and Mike Del Balso, CEO at Tecton and former Uber machine learning lead and co-creator of Michelangelo sit down to chat with us today.    These two jammed with us about building machine learning platform systems and teams, the modern operational data stack and how it allows more machine learning applications to thrive, and how to successfully take advantage of data in the process of building products and companies. // Bio Michael Del Balso Mike is the co-founder of Tecton, where he is focused on building next-generation data infrastructure for Operational ML. Before Tecton, Mike was the PM lead for the Uber Michelangelo ML platform. He was also a product manager at Google where he managed the core ML systems that power Google’s Search Ads business. Previous to that, he worked on Google Maps. He holds a BSc in Electrical and Computer Engineering summa cum laude from the University of Toronto. Erik Bernhardsson Erik is currently working on some crazy data stuff since early 2021 but previously spent 6 years as the CTO of Better.com, growing the tech team from 1 to 300. Before Better, Erik spent 6 years at Spotify, building the music recommendation system and managing a team focused on machine learning. // Relevant Links Building a Data Team at a Mid-stage Startup: A Short Story https://erikbern.com/2021/07/07/the-data-team-a-short-story.html --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Vishnu on LinkedIn: https://www.linkedin.com/in/vrachakonda/ Connect with Mike on LinkedIn: https://www.linkedin.com/in/michaeldelbalso/ Connect with Erik on LinkedIn: https://www.linkedin.com/in/erikbern Timestamps: [01:12] Introduction to Michael Del Balso and Erik Bernhardsson [03:23] High-level space in data [07:25] Complexity in the data world [09:13] Data lake + data bricks [15:20] Platform strategy [16:05] "Platform is when the economic value of everybody that uses this exceeds the value of the company that creates it." - Bill Gates [18:17] Centralizing platforms [21:06] Team spin up centralization or decentralization [27:18] Manifestations of being too far from a centralized and decentralized platform [29:24] Centralized vs Decentralized [33:33] Platform value and appropriate sizing [35:43] Building a Data Team at a Mid-stage Startup: A Short Story blog post by Erik Bernhardsson [38:51] Machine Learning as a sub-problem of Data [42:16] Operational ML [46:30] Spotify recommendations [47:13] Real-time data flows at Spotify [49:40] Data stack, Machine Learning stack, and Back-end stack reusability [51:40] Container management