askiiart/engl-2311-blog
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

stuff got updated; benchmarking dwarfs is started

Browse source
This commit is contained in:
askiiart 2024-10-25 00:13:36 -05:00
parent b2895cb8ef
commit a019e5eba9
Signed by untrusted user who does not match committer: askiiart
GPG key ID: EA85979611654C30
8 changed files with 149 additions and 13 deletions
Download patch file Download diff file Expand all files Collapse all files

88
blog/benchmarking-dwarfs.html Normal file
View file

@ -0,0 +1,88 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta content="width=device-width, initial-scale=1" charset="utf-8" />
<title>Benchmarking and comparing DwarFS</title>
<link href="/style.css" type="text/css" rel="stylesheet" />
<link href="/prism.css" type="text/css" rel="stylesheet" />
</head>
<body class="line-numbers">
<h1 id="benchmarking-and-comparing-dwarfs">Benchmarking and
comparing DwarFS</h1>
<p>DwarFS is a filesystem developed by the user mhx on GitHub
[1], which is self-described as "A fast high compression
read-only file system for Linux, Windows, and macOS." One of my
ideas for blendOS was to layer different packages, and combined
with its compression and option to be mounted as a FUSE-based
filesystem, it's an appealing option for this use case - blendOS
is immutable, so it might as well have some compression.</p>
<h2 id="methodology">Methodology</h2>
<p>The datasets being used for this test will be the
following:</p>
<ul>
<li>25 GB of null data (just <code>000000000000</code> in
binary)</li>
<li>25 GB of random data<a href="#fn1" class="footnote-ref"
id="fnref1" role="doc-noteref"><sup>1</sup></a></li>
<li>Data for a 100 million-sided regular polygon; ~29 GB<a
href="#fn2" class="footnote-ref" id="fnref2"
role="doc-noteref"><sup>2</sup></a></li>
<li>The current Linux longterm release source (<a
href="https://cdn.kernel.org/pub/linux/kernel/v6.x/linux-6.6.58.tar.xz">6.6.58</a>
[2]); ~1.5 GB</li>
<li>For some rough latency testing:
<ul>
<li>1000 4 kilobyte files filled with null data (again, just
<code>0000000</code> in binary)</li>
<li>1000 4 kilobyte files filled with random data</li>
</ul></li>
</ul>
<p>All this data should cover both latency and read speed
testing for data that compresses differently - extremely
compressible files with null data, decently compressible files,
and random data which can't be compressed well.</p>
<h2 id="sources">Sources</h2>
<ol type="1">
<li><a href="https://github.com/mhx/dwarfs"
class="uri">https://github.com/mhx/dwarfs</a></li>
<li><a href="https://www.kernel.org/"
class="uri">https://www.kernel.org/</a></li>
</ol>
<h2 id="footnotes">Footnotes</h2>
<section id="footnotes"
class="footnotes footnotes-end-of-document" role="doc-endnotes">
<hr />
<ol>
<li id="fn1">This data is from a very early version of a math
demonstration program made by a friend. The example below shows
what the data looks like for a 3-sided regular polygon.
<details open>
<summary>
3-sided regular polygon data
</summary>
<br>
<!-- I put it in here just as a `style`, it didn't work. I put it in as a div with that `style`, it didn't work. I put it in as a div of that class which has those properties in style.css, it works -->
<!-- i hate webdev i hate webdev i hate webdev i hate webdev i hate webdev i hate webdev -->
<div class="force-word-wrap">
<pre><code>[Vertex { position: Pos([0.5, 0.0, 0.0]), color: Col([0.5310667, 0.7112941, 0.7138775]) }, Vertex { position: Pos([-0.25000003, 0.4330127, 0.0]), color: Col([0.7492257, 0.3142163, 0.49905664]) }, Vertex { position: Pos([0.0, 0.0, 0.0]), color: Col([0.2046682, 0.25598457, 0.72071356]) }, Vertex { position: Pos([-0.25000003, 0.4330127, 0.0]), color: Col([0.6389981, 0.5204368, 0.077735074]) }, Vertex { position: Pos([-0.24999996, -0.43301272, 0.0]), color: Col([0.8869035, 0.30709425, 0.8658899]) }, Vertex { position: Pos([0.0, 0.0, 0.0]), color: Col([0.2046682, 0.25598457, 0.72071356]) }, Vertex { position: Pos([-0.24999996, -0.43301272, 0.0]), color: Col([0.6236294, 0.03584433, 0.7590722]) }, Vertex { position: Pos([0.5, 8.742278e-8, 0.0]), color: Col([0.6105084, 0.3593351, 0.85544324]) }, Vertex { position: Pos([0.0, 0.0, 0.0]), color: Col([0.2046682, 0.25598457, 0.72071356]) }]</code></pre>
</div>
</details>
<a href="#fnref1" class="footnote-back"
role="doc-backlink">↩︎</a></li>
<li id="fn2"><p>My code can generate up to 25 GB/s. However, it
does random writes to my drive, which is <em>much</em> slower.
So on one hand, you could say my code is so amazingly fast that
current day technologies simply can't keep up. Or you could say
that I have no idea how to code for real world scenarios.<a
href="#fnref2" class="footnote-back"
role="doc-backlink">↩︎</a></p></li>
</ol>
</section>
<iframe src="https://john.citrons.xyz/embed?ref=askiiart.net" style="margin-left:auto;display:block;margin-right:auto;max-width:732px;width:100%;height:94px;border:none;"></iframe>
<script src="/prism.js"></script>
</body>
<footer>
<p><a href="https://git.askiiart.net/askiiart/engl-2311-blog">Source code</a>&ensp;|&ensp;<a href="/feed.xml">RSS</a>&ensp;|&ensp;<a href="/glossary.html">Glossary</a>&ensp;|&ensp;<a href="/about.html">About</a></p>
<small>Image captions are the same as the alt text; assuming you're sighted, you can most likely ignore them.</small>
</footer>
</html>

38
blog/benchmarking-dwarfs.md Normal file
View file

@ -0,0 +1,38 @@
# Benchmarking and comparing DwarFS
DwarFS is a filesystem developed by the user mhx on GitHub [1], which is self-described as "A fast high compression read-only file system for Linux, Windows, and macOS." One of my ideas for blendOS was to layer different packages, and combined with its compression and option to be mounted as a FUSE-based filesystem, it's an appealing option for this use case - blendOS is immutable, so it might as well have some compression.
## Methodology
The datasets being used for this test will be the following:
- 25 GB of null data (just `000000000000` in binary)
- 25 GB of random data[^1]
- Data for a 100 million-sided regular polygon; ~29 GB[^2]
- The current Linux longterm release source ([6.6.58](https://cdn.kernel.org/pub/linux/kernel/v6.x/linux-6.6.58.tar.xz) [2]); ~1.5 GB
- For some rough latency testing:
- 1000 4 kilobyte files filled with null data (again, just `0000000` in binary)
- 1000 4 kilobyte files filled with random data
All this data should cover both latency and read speed testing for data that compresses differently - extremely compressible files with null data, decently compressible files, and random data which can't be compressed well.
## Sources
1. <https://github.com/mhx/dwarfs>
2. <https://www.kernel.org/>
## Footnotes
[^1]: This data is from a very early version of a math demonstration program made by a friend. The example below shows what the data looks like for a 3-sided regular polygon.
<details open>
<summary>3-sided regular polygon data</summary>
<br>
<!-- I put it in here just as a `style`, it didn't work. I put it in as a div with that `style`, it didn't work. I put it in as a div of that class which has those properties in style.css, it works -->
<!-- i hate webdev i hate webdev i hate webdev i hate webdev i hate webdev i hate webdev -->
<div class="force-word-wrap">
```
[Vertex { position: Pos([0.5, 0.0, 0.0]), color: Col([0.5310667, 0.7112941, 0.7138775]) }, Vertex { position: Pos([-0.25000003, 0.4330127, 0.0]), color: Col([0.7492257, 0.3142163, 0.49905664]) }, Vertex { position: Pos([0.0, 0.0, 0.0]), color: Col([0.2046682, 0.25598457, 0.72071356]) }, Vertex { position: Pos([-0.25000003, 0.4330127, 0.0]), color: Col([0.6389981, 0.5204368, 0.077735074]) }, Vertex { position: Pos([-0.24999996, -0.43301272, 0.0]), color: Col([0.8869035, 0.30709425, 0.8658899]) }, Vertex { position: Pos([0.0, 0.0, 0.0]), color: Col([0.2046682, 0.25598457, 0.72071356]) }, Vertex { position: Pos([-0.24999996, -0.43301272, 0.0]), color: Col([0.6236294, 0.03584433, 0.7590722]) }, Vertex { position: Pos([0.5, 8.742278e-8, 0.0]), color: Col([0.6105084, 0.3593351, 0.85544324]) }, Vertex { position: Pos([0.0, 0.0, 0.0]), color: Col([0.2046682, 0.25598457, 0.72071356]) }]
```
</div>
</details>
[^2]: My code can generate up to 25 GB/s. However, it does random writes to my drive, which is *much* slower. So on one hand, you could say my code is so amazingly fast that current day technologies simply can't keep up. Or you could say that I have no idea how to code for real world scenarios.

4
blog/blendos.html
View file

@ -166,7 +166,7 @@
play in the next post.</p>
<h2 id="footnotes">Footnotes</h2>
<!-- Note if you're viewing the Markdown version: [^1] is for footnotes; that's the formatting used by Pandoc, but VS Code, for example, doesn't support it. -->
<aside id="footnotes"
<section id="footnotes"
class="footnotes footnotes-end-of-document" role="doc-endnotes">
<hr />
<ol>
@ -183,7 +183,7 @@
href="#fnref2" class="footnote-back"
role="doc-backlink">↩︎</a></p></li>
</ol>
</aside>
</section>
<iframe src="https://john.citrons.xyz/embed?ref=askiiart.net" style="margin-left:auto;display:block;margin-right:auto;max-width:732px;width:100%;height:94px;border:none;"></iframe>
<script src="/prism.js"></script>
</body>

8
blog/building-blendos.html
View file

@ -52,7 +52,7 @@
<code>archiso</code>.</p>
<p>I just added the missing packages needed to build the
<code>iso</code> file, plus all the others on blendOS's GitLab,
and rebuilt it. After a bit of troublshooting (nothing
and rebuilt it. After a bit of troubleshooting (nothing
interesting, just incompetency, wrong perms, stuff like that), I
got it working perfectly.</p>
<p>From the buid log:</p>
@ -113,9 +113,9 @@
href="https://git.blendos.co/blendOS">git.blendos.co/blendOS</a>.
Accessed 2 Oct. 2024.<br />
 Saraswat, Rudra, and Asterisk. “blendOS / image-builder ·
GitLab.” <em>GitLab</em>, git.blendos.co/blendOS/image-builder.
Accessed 3 Oct. 2024.<br />
</p>
GitLab.” <em>GitLab</em>, <a
href="https://git.blendos.co/blendOS/image-builder">git.blendos.co/blendOS/image-builder</a>.
Accessed 3 Oct. 2024sku.</p>
<iframe src="https://john.citrons.xyz/embed?ref=askiiart.net" style="margin-left:auto;display:block;margin-right:auto;max-width:732px;width:100%;height:94px;border:none;"></iframe>
<script src="/prism.js"></script>
</body>

2
blog/building-blendos.md
View file

@ -24,7 +24,7 @@ I then realized that my `iso` file worked, but was not using my repo, and I had
Turns out I had added my repo to the container's `pacman` configuration, but not to the `pacman` config for `archiso`, the tool used to generate the `iso` file, or the actual resulting `iso` image's `pacman` config either. So while I would have been able to install anything from my repo in the container, the repo was unused by `archiso`.
I just added the missing packages needed to build the `iso` file, plus all the others on blendOS's GitLab, and rebuilt it. After a bit of troublshooting (nothing interesting, just incompetency, wrong perms, stuff like that), I got it working perfectly.
I just added the missing packages needed to build the `iso` file, plus all the others on blendOS's GitLab, and rebuilt it. After a bit of troubleshooting (nothing interesting, just incompetency, wrong perms, stuff like that), I got it working perfectly.
From the buid log: