Duckdb Internals with Mark Raasveldt from The GeekNarrator

The GeekNarrator

Kaivalya Apte

Episodes

About

Reviews

Promote

Duckdb Internals with Mark Raasveldt

1 hour 4 minutes Posted Dec 6, 2023 at 7:56 am.

Introduction and Guest Introduction

Guest's Journey into Databases

The Birth of DuckDB

Challenges with Existing Databases

Technical Difficulties

Why Existing Databases Fall Short for Data Scientists

The Role of SQLite and Its Limitations

Defining DuckDB

Comparing DuckDB with Other Analytical Databases

Deployment Models for DuckDB

Data ingestion into DuckDB

Data Ingestion in DuckDB

How DuckDB Handles Updates and Mutations

Understanding Column Granularity and Rewrites

Implications of Compression on Data Updates

Trade-offs in Row Group Size

Benefits of Column Storage Model

Row Groups and Parallelism

Choosing Row Group Size: An Experimental Approach

Handling Data Type Changes in Columns

Internal Data Structures in DuckDB

Reading Data: Point Lookups, Aggregations, and Joins

Optimization for Full Table Scans

Understanding ACID Compliance in DuckDB

Multi-Version Concurrency Control (MVCC) in DuckDB

Use Cases and Applications of DuckDB

The Story Behind DuckDB's Name

Future Vision for DuckDB

0:00

1:04:08

Download MP3

Share at current time

Show notes

Deep Dive into DuckDB with CTO Mark Raasveldt

Decode the insights of databases with Geek Narrator podcast. In this episode, host Kaivalya Apte converses with Mark Raasveldt, the CTO of DuckDB labs, discussing his journey from being a database enthusiast to creating DuckDB. They delve into how DuckDB, an analytical database, differs from other databases, the design decisions, its internal mechanisms, and much more. The episode also highlights the advantages of DuckDB in analytics, the motivation behind its ACID compliance, and how DuckDB handles ingestion, transaction isolation, mutations, and queries. Join in to learn how your data workloads can benefit from DuckDB.

References:

DuckDB: https://duckdb.org/

Mark's blog: https://mytherin.github.io/

===============================================================================

For discount on the below courses:

Appsync: https://appsyncmasterclass.com/?affiliateId=41c07a65-24c8-4499-af3c-b853a3495003

Testing serverless: https://testserverlessapps.com/?affiliateId=41c07a65-24c8-4499-af3c-b853a3495003

Production-Ready Serverless: https://productionreadyserverless.com/?affiliateId=41c07a65-24c8-4499-af3c-b853a3495003

Use the button, Add Discount and enter "geeknarrator" discount code to get 20% discount.

===============================================================================

Follow me on Linkedin and Twitter: https://www.linkedin.com/in/kaivalyaapte/ and https://twitter.com/thegeeknarrator

If you like this episode, please hit the like button and share it with your network.

Also please subscribe if you haven't yet.

Database internals series: https://youtu.be/yV_Zp0Mi3xs

Popular playlists:

Realtime streaming systems: https://www.youtube.com/playlist?list=PLL7QpTxsA4se-mAKKoVOs3VcaP71X_LA-

Software Engineering: https://www.youtube.com/playlist?list=PLL7QpTxsA4sf6By03bot5BhKoMgxDUU17

Distributed systems and databases: https://www.youtube.com/playlist?list=PLL7QpTxsA4sfLDUnjBJXJGFhhz94jDd_d

Modern databases: https://www.youtube.com/playlist?list=PLL7QpTxsA4scSeZAsCUXijtnfW5ARlrsN

Stay Curios! Keep Learning!

Cheers,

The GeekNarrator

← See all 110 episodes of The GeekNarrator