layout design questions

[HypenZou]

I would like to ask why fixed size blocks are used. Considering that this WAL implementation is used as bitcask storage log, the waste of storage space due to padding + record type is significant. I noticed that RocksDB wal also has a similar structure. Compared to directly appending records, what are the advantages of doing it this way?

[roseduan]

When you read a large wal file from the begining to end, read a fix-sized block is more efficient. Normally the size is equals to the page cache size.

if one record is small, like 1KB, you will read 32 tims to load 32 records.
but if the block size is 32k, we can read them all once, which will reduce the syscall(fread).

[Jack-Kingdom]

there's no index, fix-sized block is easier to extend a binary search for taget data.