From a019e5eba905c349e37eff0c2cb85dbebaeae73f Mon Sep 17 00:00:00 2001 From: askiiart Date: Fri, 25 Oct 2024 00:13:36 -0500 Subject: [PATCH] stuff got updated; benchmarking dwarfs is started --- blog/benchmarking-dwarfs.html | 88 +++++++++++++++++++++++++++++++++++ blog/benchmarking-dwarfs.md | 38 +++++++++++++++ blog/blendos.html | 4 +- blog/building-blendos.html | 8 ++-- blog/building-blendos.md | 2 +- glossary.html | 9 +++- glossary.md | 4 +- style.css | 9 ++-- 8 files changed, 149 insertions(+), 13 deletions(-) create mode 100644 blog/benchmarking-dwarfs.html create mode 100644 blog/benchmarking-dwarfs.md diff --git a/blog/benchmarking-dwarfs.html b/blog/benchmarking-dwarfs.html new file mode 100644 index 0000000..93931ce --- /dev/null +++ b/blog/benchmarking-dwarfs.html @@ -0,0 +1,88 @@ + + + + + Benchmarking and comparing DwarFS + + + + +

Benchmarking and + comparing DwarFS

+

DwarFS is a filesystem developed by the user mhx on GitHub + [1], which is self-described as "A fast high compression + read-only file system for Linux, Windows, and macOS." One of my + ideas for blendOS was to layer different packages, and combined + with its compression and option to be mounted as a FUSE-based + filesystem, it's an appealing option for this use case - blendOS + is immutable, so it might as well have some compression.

+

Methodology

+

The datasets being used for this test will be the + following:

+ +

All this data should cover both latency and read speed + testing for data that compresses differently - extremely + compressible files with null data, decently compressible files, + and random data which can't be compressed well.

+

Sources

+
    +
  1. https://github.com/mhx/dwarfs
    2. +
  2. https://www.kernel.org/
    3. +
+

Footnotes

+
+
+
    +
  1. This data is from a very early version of a math + demonstration program made by a friend. The example below shows + what the data looks like for a 3-sided regular polygon. +
    + + 3-sided regular polygon data + +
    + + +
    + 
    [Vertex { position: Pos([0.5, 0.0, 0.0]), color: Col([0.5310667, 0.7112941, 0.7138775]) }, Vertex { position: Pos([-0.25000003, 0.4330127, 0.0]), color: Col([0.7492257, 0.3142163, 0.49905664]) }, Vertex { position: Pos([0.0, 0.0, 0.0]), color: Col([0.2046682, 0.25598457, 0.72071356]) }, Vertex { position: Pos([-0.25000003, 0.4330127, 0.0]), color: Col([0.6389981, 0.5204368, 0.077735074]) }, Vertex { position: Pos([-0.24999996, -0.43301272, 0.0]), color: Col([0.8869035, 0.30709425, 0.8658899]) }, Vertex { position: Pos([0.0, 0.0, 0.0]), color: Col([0.2046682, 0.25598457, 0.72071356]) }, Vertex { position: Pos([-0.24999996, -0.43301272, 0.0]), color: Col([0.6236294, 0.03584433, 0.7590722]) }, Vertex { position: Pos([0.5, 8.742278e-8, 0.0]), color: Col([0.6105084, 0.3593351, 0.85544324]) }, Vertex { position: Pos([0.0, 0.0, 0.0]), color: Col([0.2046682, 0.25598457, 0.72071356]) }]
    +
    +
    + ↩︎
    2. +

  2. My code can generate up to 25 GB/s. However, it + does random writes to my drive, which is much slower. + So on one hand, you could say my code is so amazingly fast that + current day technologies simply can't keep up. Or you could say + that I have no idea how to code for real world scenarios.↩︎

    3. +
+
+ + + + + diff --git a/blog/benchmarking-dwarfs.md b/blog/benchmarking-dwarfs.md new file mode 100644 index 0000000..1fca903 --- /dev/null +++ b/blog/benchmarking-dwarfs.md @@ -0,0 +1,38 @@ +# Benchmarking and comparing DwarFS + +DwarFS is a filesystem developed by the user mhx on GitHub [1], which is self-described as "A fast high compression read-only file system for Linux, Windows, and macOS." One of my ideas for blendOS was to layer different packages, and combined with its compression and option to be mounted as a FUSE-based filesystem, it's an appealing option for this use case - blendOS is immutable, so it might as well have some compression. + +## Methodology + +The datasets being used for this test will be the following: + +- 25 GB of null data (just `000000000000` in binary) +- 25 GB of random data[^1] +- Data for a 100 million-sided regular polygon; ~29 GB[^2] +- The current Linux longterm release source ([6.6.58](https://cdn.kernel.org/pub/linux/kernel/v6.x/linux-6.6.58.tar.xz) [2]); ~1.5 GB +- For some rough latency testing: + - 1000 4 kilobyte files filled with null data (again, just `0000000` in binary) + - 1000 4 kilobyte files filled with random data + +All this data should cover both latency and read speed testing for data that compresses differently - extremely compressible files with null data, decently compressible files, and random data which can't be compressed well. + +## Sources + +1. +2. + +## Footnotes + +[^1]: This data is from a very early version of a math demonstration program made by a friend. The example below shows what the data looks like for a 3-sided regular polygon. +
+3-sided regular polygon data +
+ + +
+``` +[Vertex { position: Pos([0.5, 0.0, 0.0]), color: Col([0.5310667, 0.7112941, 0.7138775]) }, Vertex { position: Pos([-0.25000003, 0.4330127, 0.0]), color: Col([0.7492257, 0.3142163, 0.49905664]) }, Vertex { position: Pos([0.0, 0.0, 0.0]), color: Col([0.2046682, 0.25598457, 0.72071356]) }, Vertex { position: Pos([-0.25000003, 0.4330127, 0.0]), color: Col([0.6389981, 0.5204368, 0.077735074]) }, Vertex { position: Pos([-0.24999996, -0.43301272, 0.0]), color: Col([0.8869035, 0.30709425, 0.8658899]) }, Vertex { position: Pos([0.0, 0.0, 0.0]), color: Col([0.2046682, 0.25598457, 0.72071356]) }, Vertex { position: Pos([-0.24999996, -0.43301272, 0.0]), color: Col([0.6236294, 0.03584433, 0.7590722]) }, Vertex { position: Pos([0.5, 8.742278e-8, 0.0]), color: Col([0.6105084, 0.3593351, 0.85544324]) }, Vertex { position: Pos([0.0, 0.0, 0.0]), color: Col([0.2046682, 0.25598457, 0.72071356]) }] +``` +
+
+[^2]: My code can generate up to 25 GB/s. However, it does random writes to my drive, which is *much* slower. So on one hand, you could say my code is so amazingly fast that current day technologies simply can't keep up. Or you could say that I have no idea how to code for real world scenarios. diff --git a/blog/blendos.html b/blog/blendos.html index a0ac692..38e2027 100644 --- a/blog/blendos.html +++ b/blog/blendos.html @@ -166,7 +166,7 @@ play in the next post.

Footnotes

- + diff --git a/blog/building-blendos.html b/blog/building-blendos.html index 20ace7a..27d9b05 100644 --- a/blog/building-blendos.html +++ b/blog/building-blendos.html @@ -52,7 +52,7 @@ archiso.

I just added the missing packages needed to build the iso file, plus all the others on blendOS's GitLab, - and rebuilt it. After a bit of troublshooting (nothing + and rebuilt it. After a bit of troubleshooting (nothing interesting, just incompetency, wrong perms, stuff like that), I got it working perfectly.

From the buid log:

@@ -113,9 +113,9 @@ href="https://git.blendos.co/blendOS">git.blendos.co/blendOS. Accessed 2 Oct. 2024.
 Saraswat, Rudra, and Asterisk. “blendOS / image-builder · - GitLab.” GitLab, git.blendos.co/blendOS/image-builder. - Accessed 3 Oct. 2024.
-

+ GitLab.” GitLab, git.blendos.co/blendOS/image-builder. + Accessed 3 Oct. 2024sku.

diff --git a/blog/building-blendos.md b/blog/building-blendos.md index 4f7b814..70e61cf 100644 --- a/blog/building-blendos.md +++ b/blog/building-blendos.md @@ -24,7 +24,7 @@ I then realized that my `iso` file worked, but was not using my repo, and I had Turns out I had added my repo to the container's `pacman` configuration, but not to the `pacman` config for `archiso`, the tool used to generate the `iso` file, or the actual resulting `iso` image's `pacman` config either. So while I would have been able to install anything from my repo in the container, the repo was unused by `archiso`. -I just added the missing packages needed to build the `iso` file, plus all the others on blendOS's GitLab, and rebuilt it. After a bit of troublshooting (nothing interesting, just incompetency, wrong perms, stuff like that), I got it working perfectly. +I just added the missing packages needed to build the `iso` file, plus all the others on blendOS's GitLab, and rebuilt it. After a bit of troubleshooting (nothing interesting, just incompetency, wrong perms, stuff like that), I got it working perfectly. From the buid log: diff --git a/glossary.html b/glossary.html index bf0f78d..25390ae 100644 --- a/glossary.html +++ b/glossary.html @@ -31,7 +31,7 @@
  • Filesystem: The system which keeps track of how data is written to disk, like NTFS, FAT32, or ext4. Some filesystems, like ZFS or btrfs, have extra features like redundancy or - compression.alt text
    • + compression.
  • Git: The most common version control system by far - keeps track of different versions of files, can be used to resolve conflicting changes, etc. @@ -91,6 +91,13 @@
  • iso file: A disk image file, can be "burned" to a USB flash drive (or any other disk) and booted off of, often used for Linux installers and/or live images.
    • +
  • FUSE: A filesystem interface used for running filesystems in + userspace (i.e. not in the kernel)
    • +
  • Kernel: The very core of an operating system, with all its + most essential functions, like filesystems (aside from those + using FUSE). On Linux, device drivers are usually in the kernel + directly, rather than being installed separately like on + Windows.
    • diff --git a/glossary.md b/glossary.md index 551e6af..ff33e4f 100644 --- a/glossary.md +++ b/glossary.md @@ -7,7 +7,7 @@ - Container image: The base filesystem of a container - OCI image: The OCI's standard for container images, used by essentially all Linux container platforms - AUR: Arch User Repository, a repository for Arch Linux packages which are maintained by users. AUR only hosts computer-readable instructions and related files (via `PKGBUILD` files) for creating the packages, not the packages or programs themselves -- Filesystem: The system which keeps track of how data is written to disk, like NTFS, FAT32, or ext4. Some filesystems, like ZFS or btrfs, have extra features like redundancy or compression.![alt text](image.png) +- Filesystem: The system which keeps track of how data is written to disk, like NTFS, FAT32, or ext4. Some filesystems, like ZFS or btrfs, have extra features like redundancy or compression. - Git: The most common version control system by far - keeps track of different versions of files, can be used to resolve conflicting changes, etc. - Forking: Copying a Git repository and optionally and making changes to it. Can be simply to contribute the changes back to the upstream project later, or to use something as a base for your own project. - Branches: Different "chains" of versions of code, which can, well, branch apart, merge back together, and so on. Different branches don't necessarily have to be related whatsoever, but they usually are made from other branches. @@ -29,3 +29,5 @@ - `pacman`: Arch Linux's package manager, also used by blendOS - `PKGBUILD`: A file defining how to build a package which can be install by `pacman`. - `iso` file: A disk image file, can be "burned" to a USB flash drive (or any other disk) and booted off of, often used for Linux installers and/or live images. +- FUSE: A filesystem interface used for running filesystems in userspace (i.e. not in the kernel) +- Kernel: The very core of an operating system, with all its most essential functions, like filesystems (aside from those using FUSE). On Linux, device drivers are usually in the kernel directly, rather than being installed separately like on Windows. \ No newline at end of file diff --git a/style.css b/style.css index 2808d5d..535171c 100644 --- a/style.css +++ b/style.css @@ -79,10 +79,11 @@ code { padding: .125em; } -wrap { - word-wrap: break-word -} - +.force-word-wrap pre code { + white-space: normal; + word-wrap: break-word; + } + @media (max-device-width: 1200px) { h1 { line-height: 1.2;