Unlocking the Power of JSON Patch

What is JSON Patch?

JSON Patch is a standardized format defined in RFC 6902 for describing how to modify a JSON document. It was created to address the need for a simple, efficient, and standardized way to apply partial updates to resources, especially over HTTP. Unlike other partial update formats, JSON Patch is specifically designed to work with JSON documents, which are often used in modern web APIs.

Initially, HTTP methods like PUT and POST were used to update resources. However, this often resulted in sending large amounts of data over the network, even when only a small part of the resource needed modification. The PATCH method was introduced to allow for partial updates, but there was no standardized format for the patch document. JSON Patch fills this gap by providing a clear, concise way to express changes to a JSON document, helping to reduce bandwidth, and improve the performance of web applications.

How JSON Patch Works

JSON Patch operates on a JSON document as a sequence of atomic operations. An operation is an object that describes a single change to the document. Each operation includes an op field, which specifies the type of operation, and a path field, which identifies the location in the document where the operation should be applied.

Here's a basic structure of a JSON Patch:

Let's start with a simple JSON object.

{
  "age": 29
}

[
  { "op": "add", "path": "/name", "value": "Alice" },
  { "op": "replace", "path": "/age", "value": 30 }
]

In this example, the first operation adds a name field with the value Alice to the root of the document, and the second operation replaces the existing age field with a new value.

The resulting JSON

{
  "name": "Alice",
  "age": 30
}

JSON Patch allows the following operations:

• Add a value to an object/array
• Remove a value from an object or array
• Replace a value (essentially Remove + Add)
• Copy a value from one location to another
• Move a value from one location to another (essentially a Copy + Remove)
• Test if a value is set at a particular path in the JSON document

JSON Pointer

In the example above, you might have been confused by how we determined the path to use. JSON Patch relies on another standard called JSON Pointer to define the path to the part of the JSON document being modified. A JSON Pointer is a string of tokens separated by slashes (/), with each token identifying a level in the document's hierarchy.

Here's an example JSON document and corresponding JSON Pointer:

JSON Document:

{
  "user": {
    "name": "Bob",
    "age": 25,
    "friends": ["Alice", "John"]
  }
}

JSON Pointer: /user/name

This pointer identifies the name field within the user object. You can refer to an array's entries by their index (ex. /user/friends/0 is Alice). The last element in an array ca be referenced using - (ex. /user/friends/- is John). If a property name contains ~, then inside the pointer it must be escaped using ~0, or if contains / then it must be escaped using ~1.

For more examples go to the JSON Pointer specification page or this helpful guide.

Strengths and Weaknesses of JSON Patch

While JSON Patch is a flexible and powerful standard, it is not free of pitfalls. Lets dive into the strengths and weaknesses of the JSON Patch format.

Strengths

1. Precision: JSON Patch allows for precise modifications of a JSON
   document. This means you can target specific elements in a complex structure without affecting other parts of the document.

2. Efficiency: Unlike sending the entire updated document, JSON Patch only
   sends the changes. This minimizes data transmission and latency between clients and servers.

3. Atomicity: If a JSON Patch operation fails, the entire operation can be
   rolled back. This ensures data integrity and prevents partial updates.

4. Idempotency: JSON Patch operations can be safely retried without causing
   unintended side effects. This is crucial for reliable distributed systems and APIs.

5. Complex Operations: JSON Patch supports intricate operations, such as
   moving an element from one location to another within the same document. You can also copy elements from one place and paste them elsewhere.

6. Validation: APIs using JSON Patch can validate incoming patches to ensure
   they conform to expected operations and structure, reducing the likelihood of malformed requests.

7. Standards-Based: JSON Patch is based on web standards, making it easier
   to integrate with a wide range of clients and servers.

8. Field-level Access Control: Despite being complex to implement, JSON
   Patch provides the possibility to restrict modifications at a granular level, ensuring only authorized changes are made.

9. Batch Operations: Multiple changes can be bundled into a single JSON
   Patch, efficiently handling concurrent updates in a single request. If you are making calls to an external API that is rate limited, but supports JSON Patch, you can concatenate all of your patches together at the API gateway level and fire off a single request rather than multiple.

Optimized for Updates

JSON Patch only transmits the changes needed, minimizing payload size. This makes it more efficient than sending entire JSON objects, reducing response times and bandwidth.

HTTP/1.1 PATCH
Content-Type: application/json-patch+json

[
  {
    "op": "replace",
    "path": "/id",
    "value": "123456"
  }
]

vs

HTTP/1.1 POST
Content-Type: application/json

{
  "id": "123456",
  "firstName": "John",
  "lastName": "Doe",
  "fullName": "John Doe",
  "age": 21,
  "country": "United States",
  "state": "California",
  "city": "Redwood City",
  "zip": "94063"
}

Notice how much smaller the first request is?

Weaknesses

1. Complexity: JSON Patch can be intricate, especially when dealing with
   complex JSON structures. This typically requires developers to rely on an external library to handle to parsing and application of JSON Patches, which can also introduce security and maintenance issues.

2. Maintenance Costs: As APIs evolve, the paths specified in JSON Patches
   might become obsolete, leading to maintenance overhead.

3. Debugging Difficulty: Tracing changes in JSON Patch can be challenging,
   especially if multiple operations are batched together. Sending over the entire object with a POST gives you a snapshot through the request body, where as a PATCH requires you to have knowledge of the state of the JSON document before it was applied

4. Preservation of Object Order: JSON Patch's move operations don't
   guarantee the order of objects, which might necessitate supplementary operations to reorganize elements.

5. Security Concerns: Improper handling of JSON Patch requests can open up
   vulnerabilities. For example, if an API doesn't properly validate a move or remove operation, it could result in unintended data loss or exposure.

JSON Patch Operations with Examples

Below are examples of each JSON Patch operation, alongside the resulting JSON. Let's start with the following document:

{
  "user": {
    "name": "Bob",
    "age": 25
  }
}

1. Add

JSON Patch:

   { "op": "add", "path": "/email", "value": "bob@example.com" }

After:

   {
     "user": {
       "name": "Bob",
       "age": 25
     },
     "email": "bob@example.com"
   }

1. Remove

JSON Patch:

   { "op": "remove", "path": "/user/age" }

After:

   {
     "user": {
       "name": "Bob"
     }
   }

1. Replace

JSON Patch:

   { "op": "replace", "path": "/user/name", "value": "Alice" }

After:

   {
     "user": {
       "name": "Alice",
       "age": 25
     }
   }

1. Move

JSON Patch:

   { "op": "move", "from": "/user/age", "path": "/user/birthYear" }

After:

   {
     "user": {
       "name": "Alice",
       "birthYear": 25
     }
   }

1. Copy

JSON Patch:

   { "op": "copy", "from": "/user/name", "path": "/displayName" }

After:

   {
     "user": {
       "name": "Alice",
       "birthYear": 25
     },
     "displayName": "Alice"
   }

1. Test

JSON Patch:

   { "op": "test", "path": "/displayName", "value": "Alice" }

If the test operation succeeds, it has no effect on the document. If it fails, the entire patch is considered unsuccessful.

JSON Patch in Tools and Libraries

JSON Patch is widely used in API development and is supported by various libraries in different programming languages. Here are some examples:

Libraries

• fast-json-patch: A Javascript
  node JS package for generating and applying JSON patches.

• python-json-patch: Library
  to apply JSON Patches in Python.

• JsonPatch library in .NET:
  A C# implementation of JSON Patch.

If you use a different languauge, here's a more complete list of libraries.

Tools

A great tool for learning JSON Patch is jsonpatch.me - a free online service for running JSON Patch commands. It even has an API!

Generating a JSON Patch Change Set in TypeScript

If you are working on a client (ex. a React form), you will want to generate a changeset client-side and send it to your API. To generate a JSON Patch change set, a library like fast-json-patch can be used. Here's a simple example:

import * as jsonpatch from "fast-json-patch";

const originalDoc = {
  firstName: "Albert",
  contactDetails: { phoneNumbers: [] },
};
const modifiedDoc = {
  firstName: "Joachim",
  contactDetails: { phoneNumbers: [] },
};

// Generate the JSON Patch
const patch = jsonpatch.compare(originalDoc, modifiedDoc);

console.log(patch);

// Send the patch to the server
const headers = new Headers({
  // This content-type is specified in the spec
  // and is useful for the server to validate
  "content-type": "application/json-patch+json",
});
await fetch("https://your-backend.com/foo", { method: "PATCH", headers });

This code outputs the necessary patch document to convert the original document into the modified document:

[{ "op": "replace", "path": "/firstName", "value": "Joachim" }]

Handling HTTP PATCH Requests with JSON Patch in NodeJS

To handle a PATCH request that includes a JSON Patch document, the fast-json-patch library can be used again. Here's an example using express as your server framework:

const express = require("express");
const { applyPatch } = require("fast-json-patch");
const bodyParser = require("body-parser");

const app = express();
app.use(bodyParser.json());

let document = { firstName: "Albert", contactDetails: { phoneNumbers: [] } };

app.patch("/", (req, res) => {
  try {
    document = applyPatch(document, req.body);
    res.status(200).json(document);
  } catch (error) {
    res.status(400).send("Invalid JSON Patch document");
  }
});

app.listen(3000, () => {
  console.log("Server is running on port 3000");
});

This server listens for PATCH requests and applies the provided JSON Patch document to an existing resource.

Validating JSON Patch Request Bodies

As mentioned in the risks of JSON patch, a malformed or malicious request body sent to your API can cause data loss or other unintended consequences. That's why its important to validate the request body. It's typically best to validate the request body using an API Gateway (ex. Zuplo) so the validation can be applied consistently across all of your Patch endpoints, and be executed in a faster runtime. Here's three ways to do it:

Traditional: Using AJV + JSON Schema

To simply validate a JSON Patch request body, the AJV library and a JSON schema can be used:

const Ajv = require("ajv");
const schema = require("json.schemastore.org/json-patch");

const ajv = new Ajv();
const validate = ajv.compile(schema);

const patch = [{ op: "replace", path: "/firstName", value: "Joachim" }];

if (validate(patch)) {
  console.log("Valid Patch");
} else {
  console.log("Invalid Patch", validate.errors);
  // Return a 400 error
}

This code validates a JSON Patch document against a JSON schema and prints Valid Patch if the document is valid. This does not ensure sensitive data isn't tampered with.

Simpler: Using Zuplo + OpenAPI

A simpler way of implementing the same validation as above is by adding the JSON Schema for JSON Patch directly into your OpenAPI file in your Zuplo API Gateway. Then you can add the Request Validation policy directly to all of your PATCH endpoints. The policy will automatically return an HTTP 400 error using the Problem Details format. Here's a step-by-step guide if you're unfamiliar with Zuplo. Side-benefit is that you now have a fully documented OpenAPI file, complete with schemas.

Advanced: Using fast-json-patch

We can actually reuse the fast-json-patch library in our API to validate the request body, and even add additional test operations within the patch to ensure certain invariants aren't violated.

import * as fastJsonPatch from "fast-json-patch";

const patch = [{ op: "replace", path: "/firstName", value: "Joachim" }];

// Add invariants to safeguard the firstName property
patch.push({
  op: "test",
  path: "/firstName",
  value: originalDocument.firstName,
});
const errors = fastJsonPatch.validate(patch, originalDocument);

if (errors === undefined) {
  // there are no errors!
} else {
  console.log(`Error ${errors.name}: ${errors.message}`);
  console.log("at", errors[0].operation); // ex. {op: 'replace', path: '/firstName', value: 'Joachim'}
}

The above code can be run within a Custom Code Inbound Policy in Zuplo and easily applied to all of your Patch endpoints.

Alternatives to JSON Patch

JSON Merge Patch is a recently proposed alternative to JSON Patch that seeks to simplify the interface for specifying changes. Check out our JSON Patch vs JSON Merge Patch comparison to learn more.

Building Efficient Application Programming Interfaces with JSON Patch

JSON Patch is a powerful and efficient way to perform partial updates on JSON documents. By understanding how it works and how to implement it, developers can build more efficient APIs and applications that send less data over the network and process updates more quickly. If you're looking to implement JSON Patch across your APIs or just want to modernize your APIs more generally, and want advice from an API expert - get in touch.