The Backend Engineering Show with Hussein Nasser
The Backend Engineering Show with Hussein Nasser
Hussein Nasser
Your SSD lies but that's ok | Postgres fsync
30 minutes Posted May 25, 2023 at 4:00 am.
Intro
A Write doesn’t write
File System Page Cache
Fsync
SSD Cache
SSD ignores the flush
15 Year old Firefox fsync bug
What happens if SSD loses power
What options does Postgres exposes?
open_sync (O_SYNC)
open_datasync (O_DSYNC)
O_DIRECT
fsync
fdatasync
fsync = off
Don’t make your API simple
Database on metal?
0:00
30:04
Download MP3
Show notes
fsync is a linux system call that flushes all pages and metadata for a given file to the disk. It is indeed an expensive operation but required for durability especially for database systems. Regular writes that make it to the disk controller are often placed in the SSD local cache to accumulate more writes before getting flushed to the NAND cells.
However when the disk controller receives this flush command it is required to immediately persist all of the data to the NAND cells.
Some SSDs however don't do that because they don't trust the host and no-op the fsync. In this video I explain this in details and go through details on how postgres provide so many options to fine tune fsync