The amazing DragonFlyBSD has the fantasticHammer2 filesystem:

[dendrobatus_azureus@mastodon.bsd.cafe]

The amazing DragonFlyBSD has the fantastic
Hammer2 filesystem:

* Block copy-on-write filesystem
* Instant recovery on mount
* Instant snapshots
* Mounted snapshots are writable
* Automatic snapshotting can be enabled at the system level via periodic scripts
* Default Periodic also does daily bulk pass on the meta-data to free space
* Automatic compression (controllable on directory recursion and per-file basis)
* Automatic de-duplication
* Future master/slave mechanism
* Utilizes a dynamic radix tree
* 64-bit hardlink counter
* 2^63 logical file size limit
* Recursive check codes to detect corruption
* Any number of pseudo-filesystem volumes for each physical hammer2 disk image (also used by snapshots).

https://www.dragonflybsd.org/hammer/

#filesystem #programming #BSD #dragonflyBSD #freeBSD #OpenSource #technology #no #FSK

[dendrobatus_azureus@mastodon.bsd.cafe]

Additional Reference Material

HAMMER2 Design v6 (08-Dec-2018)

https://gitweb.dragonflybsd.org/dragonfly.git/blob/57614c517203403e7fe4a34c370c7151ba52c539:/sys/vfs/hammer2/DESIGN

#filesystem #programming #BSD #dragonflyBSD #freeBSD #OpenSource #technology #no #FSK

[dendrobatus_azureus@mastodon.bsd.cafe]

Quote from Hammer2 page

Because HAMMER2 is a block copy-on-write filesystem, the "atime" field is not supported and will typically just reflect local system in-memory caches or mtime.

The radix tree is dynamic in that each entry can dynamically control how many bits it chops off. This allows small files to be contained in just one or two levels regardless of the block seek positions. The depth of the radix tree is increased as needed via a splitting mechanism, and will also be recombined if it grows smaller. All block references are 64-bit aligned-byte-indexed references and thus portable regardless of physical sector size changes between underlying block devices.

Inodes are 1KB of which 512 bytes are used for the top-level radix tree OR 512 bytes of data. Any file less than or equal to 512 bytes stores its data directly in the inode. Files up to 256KB can be accommodated with direct inode block references.

Directory entries are hashed (semi-sorted hash algorithm), and directly embedded in the radix table's blockref structure for maximum performance. Files with very long filenames will contain a dataref, otherwise filenames are embedded in the directory entry itself. Because directory entries are hashed, seeking and lookups are able to use a radix search and no linear scan of the directory is needed.

The inode and directory entry structure is extremely well suited for any file size or directory size, from tiny to huge.

Because of the block-copy-on-write nature of the filesystem, the filesystem is able to create a snapshot trivially simply by copying the volume header's root block table (4 blockref entries). The directory topology actually starts with a SUPERROOT, and volume ROOTs are directory entries under the SUPERROOT. Though the entries are actually special-cased a bit and actually part of the root inode for each filesystem root. And since physical freeing of space is handled via a bulk meta-data scan, destroying a snapshot or volume can be done simply by wiping the inode and ignoring everything under it... the next bulkfree scan will reclaim any reclaimable space. Similarly with file deletions... the top-level data blockrefs can simply be removed. The inode can simply be removed from the radix tree.

Performance is very good. HAMMER2 uses a variable-sized block in powers of two, starting at 1KB, up to 64KB, for the last block of the file (straddling EOF). All earlier blocks in the file, if any, use 64KB blocks. The freemap is organized by domain to cluster various meta-data types together. Indirect blocks can be one of two sizes: 16KB or 64KB, allowing medium-sized files and directories to be optimally allocated. In addition, file data compression of a logical block can result in a smaller physical block. The physical layer always does 64KB I/O and can cluster the I/O on top of that.

https://www.dragonflybsd.org/hammer/

#filesystem #programming #BSD #dragonflyBSD #freeBSD #OpenSource #technology #no #FSK

[clf@mastodon.bsd.cafe]

@Dendrobatus_Azureus I wonder where FreeBSD would be right now if they had accepted Matthew Dillon's non-blocking dreams

[paride5745@mastodon.bsd.cafe]

@Dendrobatus_Azureus my wet dream. HaikuOS rebasing on DragonFlyBSD (a HaikuBSD 😈 ), leveraging Hammer2 and kernel-level message-parsing/LWKT, and a more modern kernel and system design.