Modern OLAP Database System Design with FDAP (Andrew Lamb) from The GeekNarrator

Modern OLAP Database System Design with FDAP (Andrew Lamb)

56 minutes Posted Jun 5, 2024 at 5:00 am.

Introduction

Understanding Analytics: Transactional vs Analytical Databases

The Genesis and Goals of the FDAP Stack

Decoding FDAP: Flight, Data Fusion, Arrow, and Parquet

Apache Parquet: Revolutionizing Columnar Storage

Apache Arrow: The In-Memory Game Changer

Interoperability and Migration with Apache Arrow

Comparing Apache Parquet and Arrow

Exploring Data Mutability in Analytic Systems

Handling Data Updates and Deletions

The Role of Immutable Storage in Analytics

Optimizing Data Storage and Mutation Strategies

Introducing Flight: Simplifying Data Transfer

Deep Dive into Flight's Benefits and SQL Support

Unpacking Data Fusion's SQL Support and Extensibility

The Interplay of FDAP Components in Analytics

Future Directions and Innovations in Data Analytics

Concluding Thoughts on FDAP and Its Impact

0:00

56:48

Download MP3

Share at current time

Show notes

In this video I speak with Andrew Lamb, Staff Software Engineer @Influxdb. We discuss FDAP (Flight, DataFusion, Arrow, Parquet) stack for modern OLAP database system design. Andrew shared some insights into why the FDAP stack is so powerful in designing and implementing a modern OLAP database.

Chapters:

FDAP Stack: https://www.influxdata.com/glossary/fdap-stack/

FDAP Blog: https://www.influxdata.com/blog/flight-datafusion-arrow-parquet-fdap-architecture-influxdb/

InfluxDB: https://www.influxdata.com/

Follow me on Linkedin and Twitter: https://www.linkedin.com/in/kaivalyaapte/ and https://twitter.com/thegeeknarrator

If you like this episode, please hit the like button and share it with your network.

Also please subscribe if you haven't yet.

Database internals series: https://youtu.be/yV_Zp0Mi3xs

Popular playlists:

Realtime streaming systems: https://www.youtube.com/playlist?list=PLL7QpTxsA4se-mAKKoVOs3VcaP71X_LA-

Software Engineering: https://www.youtube.com/playlist?list=PLL7QpTxsA4sf6By03bot5BhKoMgxDUU17

Distributed systems and databases: https://www.youtube.com/playlist?list=PLL7QpTxsA4sfLDUnjBJXJGFhhz94jDd_d

Modern databases: https://www.youtube.com/playlist?list=PLL7QpTxsA4scSeZAsCUXijtnfW5ARlrsN

Stay Curios! Keep Learning!

#datafusion #parquet #sql #OLAP #apachearrow #database #systemdesign

← See all 110 episodes of The GeekNarrator