Rust Ephemeral Encrypted Containers/FS (maybe)
Running through the 'zboxfs' code and these are some notes. Should be able to spin it up & get the general gist from what's written here, enjoy!
What's Up With 'ZBoxFS'?
ZBoxFS is a random project that I had came across a few times prior and logged in my notes somewhere. However, very recently when researching a few solutions for a blockchain-based wallet (agnostic) that I was devising for Librehash members, I stumbled back onto their main repo once again.
You can all find that URL here - https://github.com/zboxfs/zbox
Breaking Down te ZboxFS Project
The project caught my eye for a few reasons:
Smart Design - The way its constructed actually makes sense & it appears to fit into a niche use case / purpose that no other app has fulfilled as of yet.
Strong Cryptographic Algorithms - We'll get to this part soon, but the cryptographic algorithms this project uses are 'strong', for lack of a better term. Truly that better term would be that the algorithms it employs in its orchestration are secure (there are strong cryptographic algorithms / functions that can be vastly insecure when applied incorrectly or used in inappropriate situations).
Continuously Maintained - The development is up to date on this project; as a rule of thumb, I generally don't touch 'dead projects' unless I have some sort of long-term plan to revive them in the future at some point (to be rebranded under Librehash when I can be sure that we have the resources to effecticely maintain the project w/o abandoning it ourselves).
Describing the ZBoxFS Project
In their 'readme', they state, "ZBoxFS is a zero-details, privacy-focused in-app file system. Its goal is to help application[s] store files securely, privately and reliably. By encapsulating files and directories into an encrypted repository, it provides a virtual file system and exclusive access to authorized application."
Curiously, it goes on to state, "Unlike other system-level file systems, such as ext4, XFS and Btrfs, which provide shared access to multiple processes, ZboxFS is a file system that runs in the same memory space as the application. It provides access to only one process at a time."
Finally, the authors state, "By abstracting IO access, ZboxFS supports a variety of underlying storage layers, including memory, OS file system, RDBMS and key-value object store."
This unique property means that not only does this app encrypt on the fly, it also provides some level of isolation from the other processes / facets of the filesystem (on the OS its deployed on).
Listed Features
"Everything is encrypted 🔒, including metadata and directory structure, no knowledge can be leaked to underlying storage"
"State-of-the-art cryptography: AES-256-GCM (hardware), XChaCha20-Poly1305, Argon2 password hashing and etc., powered by libsodium."
"Support varieties of underlying storages, including memory, OS file system, RDBMS, Key-value object store and more"
"Files and directories are packed into same-sized blocks to eliminate metadata leakage"
"Content-based data chunk deduplcation and file-based deduplication."
"Data compression using LZ4 in fast mode, optional."
"Data integrity is guaranteed by authenticated encryption primitives (AEAD crypto)"
"File contents versioning."
"Copy-on-write (COW 🐮) semantics"
"ACID transactional operations."
"Built with Rust."
The very last listed feature is a huge one. Rust is our favorite language (up there with Golang, Python & C++ / C), for its memory-safety features as well as its robust community & surprisingly well-designed apps (which really feel like a 'hook up' everytime you run into one of them; this tool being case-in-point).
What's there not to love here? (we'll get into that soon, which will explain the rabbit hole that we went down just to get to this point).
ZboxFS Architecture
Below is one of the diagrams Zbox provides on their GitHub to help readers visualize the difference between the way their software is instantiated versus other well known file systems and tools.
Which was then followed by a feature matrix.
Below is a feature matrix that shows the different storage backend options that come with Zbox.
Rust-specific documentation can be found here at this link - https://docs.rs/zbox/0.9.2/zbox/
Running Through Some of the Mock Rust Code for ZboxFS
Since there are some easy-to-run examples provided right on their Rust project page, we'll extract those (in code blocks), for those that may be interested in running this code on their own personal machines (as they read).
The following code is said to be designed with the intent of showcasing how to 'create and open a repo using memory as underlying storage' (sort of like tmpfs / ramfs).
use zbox::{init_env, RepoOpener};
// initialise zbox environment, called first
init_env();
// create and open a repository
let mut repo = RepoOpener::new()
.create(true)
.open("mem://my_repo", "your password")
.unwrap();
File content IO using Read and Write traits
use std::io::prelude::*;
use std::io::{Seek, SeekFrom};
use zbox::OpenOptions;
// create and open a file for writing
let mut file = OpenOptions::new()
.create(true)
.open(&mut repo, "/my_file.txt")
.unwrap();
// use std::io::Write trait to write data into it
file.write_all(b"Hello, world!").unwrap();
// finish writting to make a permanent content version
file.finish().unwrap();
// read file content using std::io::Read trait
let mut content = String::new();
file.seek(SeekFrom::Start(0)).unwrap();
file.read_to_string(&mut content).unwrap();
assert_eq!(content, "Hello, world!");
"*Discovery navigation can use
and
```PathBuf```
. The path separator should always be "/", even on Windows.*"
```Rust
let path = Path::new("/foo/bar");
repo.create_dir_all(&path).unwrap();
assert!(repo.is_dir(path.parent().unwrap()).is_ok());
Cryptographic Primitives
of the most ‘relevant’ ones for our endeavors (hint: most of them are going to be related to the memory handling of the k
This is one of the most important features of this app (since we don't have time to play Daniel Bernstein to someone else's project). Information about the cryptographic schemes baked into this project can be found here - https://docs.rs/zbox/0.9.2/zbox/enum.Cipher.html
Among ciphers (encryption), users have the option of either going with AES256-GCM (known to be vulnerable to side-channel cache timing attacks; not good) or XChaCha20-Poly1305, which is considered to be highly secure; specifically, it is resilienit to the same side channel cache timing attacks that AES256-GCM could fall prey to.
Encryption Parameters
Key Size - 256 bits
Nonce Size - 192 bits
Block Size - 512 bits
MAC Size - 128 bits
Up to this point, this write-up has failed to mention that one of the libraries baked into this program is 'libsodium'. For those that don't know, 'libsodium' is one of the best open-source cryptographic libraries available to programmers (hence why you its in use so frequently).
Official GitHub repo for this library can be found here - https://github.com/jedisct1/libsodium
website - https://libsodium.org/
libsodium documentation - https://doc.libsodium.org/
libsodium audit (sponsored by PIA before they were bought by an independent company) - https://www.privateinternetaccess.com/blog/libsodium-audit-results/
Critical libsodium Feature: 'Secure Memory'
There's one portion of the documentation that provides extensive information on the code / syscalls / declarations one would need to use in order to provision 'secure memory'.
Zeroing Memory
void sodium_memzero(void * const pnt, const size_t len);
"After use, sensitive data should be overwritten, but
*and hand-written code can be silently stripped out by an optimizing compiler or by the linker.*" "*The* ```sodium_memzero()``` *function tries to effectively zero* ```len``` *bytes starting at* ```pnt``` , *even if optimizations are being applied to the code*."
Locking Memory
int sodium_mlock(void * const addr, const size_t len);
"The
function locks at least* ```len``` *bytes of memory starting at* ```addr``` . *This can help avoid swapping sensitive data to disk*."
Some additional recommendations given:
- "Totally disable swap partitions on machines processing sensitive data."
or
- "Use encrypted swap partitions"
Disabling Core Dumps When Running Crypto Code (in order to ensure that sensitive data cannot be leaked out during the process itself). "This can be achieved using a shell built-in such as
*or programmatically using*
```setrlimit (RLIMIT_CORE, &(struct rlimit) {0,0})```
.
Its also recommended for folks to disable kernel dump as well if they find themselves in a situation where they're disabling swap partitions.
**More Restrictions to Secure Memory**
Its worth noting that all of these restrictions are ones that can be exercised w/o libsodium since they effectively mirror already existing Linux syscalls.
For those curious, here is a link to all existing Linux syscalls (per the 5.14 kernel release at the time of writing) - https://elixir.bootlin.com/linux/latest/source/include/linux/syscalls.h (ignore the underlying source code for the syscalls here).
Below are a list of the most 'relevant' ones for our endeavors (hint: most of them are going to be related to the memory handling of the kernel itself anyway).
If this were the direction that we were electing to go in, then we would need to find a way to bind that 'C' code to be used with Rust (since this is at the heart of the program).
### One Major Issue With the ZboxFS Setup
Just when you thought everything was perfect! There's a little problem we run into here with the instantiation of this tool (zbox). To understand this problem, one must visit their website (https://zbox.io).
As we can see in the photo above, the site pivots this tool toward a paid model / subscription-based service rather than maintaining its posture as an open source tool / app for us to use.
This is likely the reason for the restricting coding dictating the available backends (should be configurable vs. finite options).
Also, rather than positioning itself as a mountable, custom fs (file system ) created on the fly for the purposes of local on-the-fly encryption, the website conveys that this tool is purposed to facilitate cloud storage (i.e., encrypt the data on one's desktop / locally, then store said data remotely).
If the app fails to provide us with flexibility in the backend tools that we can provision, then all of its advertised 'features' are a mirage.
Further evidence that this app was designed to pivot developers & users alike toward their custom, proprietary storage setup can be found in the example code snippets they give on their main site.
Below are two samples of code they give for creating an instance of 'zbox' and attaching it to a storage (cloud) backend via API (which of course has their site plugged in out-of-the-box, with no documentation on how one would effectively swap out their provided IPs for anther [custom one] provided on the backend).
The first code sample is Javascript, second = Rust.
```javascript
// create a Zbox instance
const zbox = new Zbox();
// initialise Zbox environment and turn on debug logs
await zbox.initEnv({ logLevel: 'debug' });
// open the repo
var repo = await zbox.openRepo({
uri: 'zbox://yb2CCTcmEuxenZVuZhVKMCJD@AWCpPaNkvG6vVW',
pwd: 'secret password',
opts: { create: true }
});
// create a file and write content to it
var file = await repo.createFile('/hello_world.txt');
await file.writeOnce('Hello, World!');
// seek to the beginning of file and read all content
await file.seek({ from: Zbox.SeekFrom.Start, offset: 0 });
const str = await file.readAllString();
// close file, repo and exit Zbox
await file.close();
await repo.close();
await zbox.exit();
// initialise Zbox environment
init_env();
// create and open a repository
let mut repo = RepoOpener::new()
.create(true)
.open("zbox://yb2CCTcmEuxenZVuZhVKMCJD@AWCpPaNkvG6vVW",
"secret password")
.unwrap();
// create a file and write content to it
let mut file = OpenOptions::new()
.create(true)
.open(&mut repo, "/hello_world.txt")
.unwrap();
file.write_once(b"Hello, World!").unwrap();
// seek to the beginning of file
file.seek(SeekFrom::Start(0)).unwrap();
// read all content
let mut content = String::new();
file.read_to_string(&mut content).unwrap();
Of course, the kicker here is when they provide a preview of their architecture (geo-distributed cloud storage); enticing users to create an account / node / storage pool that they host & maintain.
App is Still Usable
Fortunately for us, we can still definitely use the app spec as it is currently (but we're going to need to work at it a bit though first).
Additionally, we can provision this app in a way where we can be sure that the remote server we authenticate with (i.e., not 'zbox.io', but something provisioned by Librehash), will be 100% ignorant of the content being stored on the server (hopefully to the extent of not revealing any metadata about what's being encrypted remotely).
Ultimate Goal - Use this as an application backend for wallet apps & other similar tools where sensitive cryptographic operations are being performed
Good News - API is Swappable
The references to the remote server provided by Zbox in their code can be edited (with trivial ease) to be replaced with our custom backend (alongside some authentication mechanisms to provide significantly enhanced security assurances against a potential MITM attack [in case there were a malicious entity attempting to phish / impersonate our desired target remote server storage]).
API reference can be found here - https://docs.zbox.io/api/
Specifically, there is reference code for Javascript, Rust, and Android (Java). We're going to focus on the Javascript for the time being (despite the fact the app is written in Rust); there's a reason why we're doing this [that will be explained at a latter point].
Diving into the API Reference
From the picture above, we can deduce that the only palpable change that needs to be made here is in the URL where the data will be stored (we'll get to explaining exactly what the replacement backend storage will be shortly).
Example 'Calling' Style
As we can see below, the 'calling' style for this app (API; javascript), is very similar to what we've seen elsewhere (if you're someone that deals with API frequently).
Below is some boilerplate code that was prepared to demonstrate the iterated call style for this app.
// Promise chaining
zbox.openRepo({
uri: 'zbox://access_key@repo_id',
pwd: 'secret password',
opts: { create: true }
})
.then(repo => repo.openFile('/foo/bar.txt'))
.then(file => file.readAll())
.then(data => console.log(data));
// Async/await
async function asyncFunc() {
var repo = await zbox.openRepo({
uri: 'zbox://access_key@repo_id',
pwd: 'secret password',
opts: { create: true }
});
var file = await repo.openFile('/foo/bar.txt');
var data = await file.readAll();
console.log(data);
}
Of course, the app comes with 'error handling' as well (nothing that we're going to spend too much time focusing on anyway).
The following configurations are relevantly important (this provisions the local / client-side encryption & derivation schemes via the provided KDF).
{
opsLimit?: OpsLimit, // default: OpsLimit.Interactive
memLimit?: MemLimit, // default: MemLimit.Interactive
cipher?: Cipher, // default: (see below)
create?: boolean, // default: false
createNew?: boolean, // default: false
compress?: boolean, // default: false
versionLimit?: number, // default: 1
dedupChunk?: boolean, // default: false
readOnly?: boolean, // default: false
force?: boolean // default: false
}
Top comments (0)