Software at Scale 22 - Sujay Jayakar

Sujay Jayakar was a Software Engineer at Microsoft Research where he worked on kernel bypass networking. He was previously a Principal Engineer at Dropbox where he worked on the migration of user data from S3 to the internal storage system (Magic Pocket), and the sync engine deployed to clients.Apple Podcasts | Spotify | Google PodcastsHighlights05:00 - What framework do you use to decide to stop using S3 and store data in your own data centers? (the “Epic Exodus” story)11:00 - Perfect Hashtables and how they are used in production systems to reduce memory use14:00 - What is an OSD? (Object Storage Device). How does it work?20:30 - SMR drives30:00 - The actual data migration - how did it happen, and how does one validate that the data being transferred is correct.33:00 - “S3 being overwhelmed”. That’s a string of words most software developers don’t expect to hear. What kind of overhead do kernels impose on networking, and why?43:00 - What is Kernel Bypass Networking? This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit www.softwareatscale.dev