Build Your Data Lake on AWS: the Flo experience
In this talk Ivan will explain the rationale behind Flo’s descision to move from traditional Data Warehouse to Data Lake and describe a set of tools which will help you to build your own Data Lake on AWS.
1. From Data Warehouse to Data Lake (motivation, drivers, etc.).
2. What is Data Lake and do you really need it?
3. How to ingest and where to store your data.
4. Cataloguing and searching.
5. Processing and serving.
Bonus: few tips from a data engineering standpoint.
AWS Services covered in this talk: Amazon Kinesis, Amazon MSK, Amazon Glue, Amazon EMR, Amazon Athena, PrestoDB.
Software Engineer, Flo Health Inc.