Please note

Since this blog was published a lot has happened. The Hooks Amendment is now running on a public testnet and has a public "Builder" where you (devs) can code Hooks and deploy them to the testnet, straight from your browser.

Hooked #2: — Hooks & Security

🔥 We care deeply about the XRPL, and take security, performance, fee structure very seriously. This blog discusses some of the design choices and technical details.

1. Introduction

⚡ This article is very technical in nature and we strongly recommend you first read the previous two articles: here and here.

Hooks are small efficient web assembly modules that run on the XRP Ledger at Layer 1 (on-chain). This is distinct from Codius, Hot Pocket, Flare and other Layer 2 solutions (which are off-chain or side-chain). This means hooks have the ability to control, with atomicity and finality, the logical flow and execution of transactions on the accounts to which the hooks are configured.

The Hooks Amendment invites three primary areas of concern we must address:

• Security — Hooks must be tightly sandboxed with invariant checks and carefully measured constraints on the allowable behaviour.
• Runtime — Hooks must complete execution in a predictable and efficient manner so they do not delay forward progress of the ledger.
• Determinism — Hooks must execute the same way on every validator otherwise the ledger will struggle to make forward progress.

2. Core Security Model

Hooks will be executed in a fast, light and secure "virtual machine", following standards.

Web Assembly is a concise and portable binary executable format capable of being run in any W3C standards-compliant web assembly runtime. One such runtime is running in your browser as you read this. Another is included in our upcoming technology preview of the Hooks Amendment.

Web Assembly does not allow the inclusion of native hardware instructions. All instructions sequences consist of well-defined byte-code format which can only be interpreted by a virtual machine, which then acts only according to the web assembly standard.

The core security model is similar to that of an operating system: the web assembly runtime environment (xrpld) acts as the operating system and the Web Assembly binary (hook) acts as a user-space application. The hook is sandboxed by xrpld. The hook can modify its own memory and make calls to xrpld, but only to functions xrpld has explicitly exposed for this purpose.

• The hook cannot call out to any external or internal process, service or endpoint; it can only call the Hook API.
• The hook cannot modify or read any part of the system outside its own well defined and pre-allocated memory.
• The hook's memory is limited by a validator-votable figure, nominally 64kib.

When calling to ask for something to be done, e.g. blocking a transaction, the hook provides pointers within its own memory-space that xrpld can then read and write to. If the hook provides incorrect pointers it can only trash its own memory and cannot affect the surrounding system.

💡 Learn more about WebAssembly Security here.

3. Runtime Loop Guarding

🛡 Hook developers will have to use "Guards" in their Hook source code, allowing xrpld to monitor, and calculate worst case resource cost, and reject Hooks if they are too inefficient.

Within these constraints the hook may do any arbitrary computation because web assembly is Turing Complete. However Turing Completeness is actually undesirable at layer 1 because it is mathematically impossible to determine when an arbitrary program with arbitrary inputs will end (see: Halting Problem), and hooks need to have predictable maximum execution durations. If they could run forever the ledger might never make forward progress.

Consider the following snippet of code.

for (uint32_t i = 0; i < destination_tag; ++i)
{
    // some expensive code here
}

If we were to include this loop inside a hook then the loop could execute as few as zero times, or as many as 4.29 billion times depending on which value was passed in the originating transaction as a destination tag. If the code inside the loop was very expensive to execute this loop might take minutes or even hours. Alternatively if destination tag was zero this loop would never execute at all.

Enter: guarding. Let's modify this code with some creative use of the comma operator.

for (uint32_t i = 0; GUARD(10), i < destination_tag; ++i)
{
    // some expensive code
}

When compiled to web assembly the loop looks like this (in C pseudo-code):

while(1)
{
    GUARD(10);
    if (i >= destination_tag)
        break;
}

The GUARD() call is the very first meaningful instruction in the loop. In fact, it must occur before any branch instruction in every loop and function according to the guard rules. No branch instruction (e.g. break or continue or call) is allowed before it. And, roughly speaking, it says to xrpld "I promise I will not hit this guard more than 10 times, and if I do then you may terminate and rollback this hook's execution."

The guard rules are enforced on the SetHook transaction. A SetHook transaction containing a hook which fails the guard rules is malformed and will not succeed.

However this level of guarding is not very user friendly so a best practice is to provide soft guards on your hard guards:

if (destination_tag > 10)
    rollback(SBUF("Sorry, this hook only allows destination tags less than 10!"), 1);

for (uint32_t i = 0; GUARD(10), i < destination_tag; ++i)
{
    // some expensive code
}

Now an unsuspecting user will receive an exit reason other than GUARD VIOLATION explaining WHY the hook failed not just that it failed.

3. Hook Execution Fees

🔒 Hook execution will be charged (on ledger fees) based on maximum Hook execution duration, to prevent ledger spam and excessive validator resource consumption. In the worst case, an inefficient Hook can even be rejected.

Preventing spam on the ledger is of paramount importance to the stability and usability of the ledger and is one of the reasons for having a native token (XRP) on the ledger.

Using an analysis of guards in a hook it is possible to compute, ahead-of-time, the longest execution duration the hook could ever take. The fee for running the hook is always paid by the caller (the transaction that activates the hook) and is always proportional to the maximum computed execution duration of the hook. In addition to this validators may vote on a global maximum hook duration: Any hook whose computed maximum execution time exceeds this would be rejected during the SetHook operation, meaning the Hook will never even become activated on an account. For reference the execution times we are talking about are measured on the scale of thousandths to millionths of a second.

4. Determinism

🛬 Hooks can only read & affect on-ledger data, so the output of a Hook executed for the same transaction at the same time on all validators, will always produce the same results to be able to reach consensus.

We have addressed security and runtime however none of this is useful without determinism. In short: when validators run hooks on an account they must run the same hook code, on the same account, with the same inputs and produce the same outputs, at the same time. If they do not then the validators will not agree on the outcome and consensus may become imperilled, leading to halt in the network.

Ensuring deterministic execution is straight forward: all information a hook is able to obtain must come from data that consensus has already been reached on or is about to be reached on. Naturally this includes every object on the ledger and objects in recent history.

At no time should a hook be able to access a (real) random number generator, a timestamp (clock) or any other parameter, function or feature that is not subject to consensus. Enforcing this is easy: we simply do not expose any functions from xrpld to the hook that contain any non-deterministic information.

Sometimes it is necessary for a hook to obtain a nonce to make unique an emitted transaction or other hook execution result. For this we provide a deterministic nonce() function in the Hook API that the hook may call as needed. The nonce is produced by a (deterministic) pseudo random number generator whose seed is the last closed ledger and the hook's account. The PRNG is advanced each time nonce() is called up to a maximum call limit.

5. Isolation Models and Node Types

🔑 Validators will have a range of additional security protections to choose from.

In addition to the core security model (aka Code Level Isolation) as discussed above, in the final Hooks Amendment validators (and other mission-critical xrpld nodes) will have the following isolation options:

1. Process Level Isolation

   xrpld creates a second process on startup, and de-privileges the second process. All web assembly execution occurs in the de-privileged process with Hook API calls routed through interprocess communication.

2. Machine Level Isolation

   The same as Process Level Isolation except the second process runs on a different physical machine with remote procedure calls instead of interprocess communication.

The vast majority of stock nodes will use the default Code Level Isolation with the web assembly runtime running inside xrpld.

Planning ahead, a further level of isolation (and a scaling solution) may be Hardware Level Isolation wherein hooks run on dedicated hardware inside the validator's physical machine (think: FPGA accelerator card).

While there is no way to avoid the escalation in resources required to do the additional computation required by hooks, we can mitigate the effects by setting a reasonable fee structure and enforcing efficiency limits as discussed above.

The crux of this issue pertains to how we think about nodes to begin with: a role exists in the ecosystem for lite nodes... nodes that do not compute the next ledger but do observe validation and record ledgers as they are computed by the validators on the node's UNL. For many use-cases these lite nodes can serve as a low cost entry point to the decentralised network, possibly being light-weight enough to run on payment terminals and IoT devices.

We argue that a homogenous node model does not exist now and would be undesirable if it did. Rather than trying to make xrpld run on commodity hardware we should be thinking about different node types for different use-cases. Validators, full-history nodes and other mission-critical nodes should run on fast highly redundant clusters, and your payment terminal on your electric car charger should be small and embedded and doing the absolute minimum amount of work possible while still being connected to the decentralised ledger. We should not be limiting the technology ledger by trying to make a one-size-fits-all code-base, especially when already that doesn't work.

6. The Hook API

Much has been said about the functions exposed to the hook by xrpld. The Hook API is the topic of the next blog and the upcoming API documentation.

Please check the other blogs, the source & README's and ... just a little more patience as we're on track for a Q1 public testnet release :)

Discussion

Discussions: here »