PipyJS - A functional style JavaScript engine for programming Pipy

In previous article Pipy - A Programmable network proxy for cloud, edge, and IoT we introduced Pipy, explained its modular design, architecture and crafted a fully functional network proxy with load balancing, caching and routing support. Pipy speaks and understands JavaScript via its custom-developed component PipyJS, which is part of the Pipy code base, but has no dependency on it.

As was seen in Pipy introduction article, all of the network proxy programming logic was written in a JavaScript dialect and this article is going to introduce you to Pipy's powerful programming companion PipyJS, the reasoning behind why we chose JavaScript, PipyJS syntax, and areas where PipyJS deviates from ECMAScript.

Why JavaScript?

As the name suggests, PipyJS is based off of JavaScript. Pipy has decided to use a JavaScript-like scripting language for a few reasons:

• It's probably the planet's most widely used programming language
• Its syntax is in C-and-Java-style, which is familiar to the most programmers
• Pipy is a stream processor, that means a large part of its job requires data transformation and restructuring. JavaScript provides a lot of powerful syntax sugar and built-in methods for doing that easily in a very concise way, such as spread syntax, destructuring assignment, and Array operations like map() and reduce()
• JSON is the most widely used message format on the web and JSON operations just feel easy and natural in JavaScript
• Last and most importantly, JavaScript embraces the "functional programming" paradigm

Pipy works like a bunch of interconnected pipelines. Each pipeline is composed of a series of filters. Each filter works like a function that gives a certain output in response to its input. For a design pattern like that, a functional programming language would fit Pipy best when your goal is to keep everything coherent and simple.

Introducing PipyJS

PipyJS is a small and embeddable functional-style interpreted JavaScript engine that is designed for high performance with no garbage collection overhead. It supports a subset of ECMAScript standards and in some areas makes bold choices to deviate from them. PipyJS loves and promotes purely functional style JavaScript, in PipyJS everything is an expression.

Data types

Like the standard JavaScript, PipyJS supports 4 primitive types and Object type for structural data.

• Primitive types:

  • Undefined. The unique value undefined when a variable is not initialized yet.
  • Boolean. true and false.
  • Number. A 64-bit double precision floating number, such as 123, 0xabc, 3.1415926, 1e6.
  • String. A sequence of Unicode characters.

• Structural data type:

  • Null object, represented by a unique value null.
  • A user defined plain old data, such as { x: 1.2, y: 300 }
  • A builtin object, such as Array or RegExp.
  • A function, such as (a, b) => a * a + b * b.

Operators

PipyJS supports all standard JavaScript operators, including some operators that were only introduced later in ES2020, such as optional chaining and nullish coalescing.

Global variables

Global variables in PipyJS are also called "context variables". The term context here is roughly equivalent to "connection" in terms of high-concurrency network programming, where each connection has their own isolated "state" from others. In PipyJS, these isolated states are stored in global variables for convenience. That's why sometimes we call them "context variables". That also leads to the fact that PipyJS global variables are different from the standard JavaScript in that they can have different values for different connections. It looks more like thread-local storage in that regard.

Global variables are defined via the built-in function pipy(), which is always the very first function you are going to call at the beginning of your script.

pipy({
  _myGlobalVariable: undefined
})

For convention, we always start global variable names with an underscore, though it is not enforced by the language itself.

Global variables are scoped within a single file or _module_, and can be shared between different modules by using export() and import().

// file A
pipy().export('namespace-1', {
  __myGlobalVariable: undefined
})

// file B
pipy().import({
  __myGlobalVariable: 'namespace-1'
})

For convention, we always start exported global variable names with two underscores, though it is not enforced by the language itself.

Local variables

In PipyJS, we use function arguments nested inside a function scope for local variables.

void ((
  x, y, z, // declare local variables as function arguments
) => (
  x = 0,
  y = 0,
  z = 0 // initialize and use the variables in the function body
))() // Don't miss the () to invoke the function right away!

If the above expression is supposed to be evaluated to some return value that will be used afterwards, you should remove the operator void at the beginning.

Branch

In PipyJS, everything is an expression. There is no code blocks or control flow. You can't write if or for statements. But that doesn't mean we can't have branches and loops. We only do it in a different style, the functional style.

We can use the logical operator && for simple branches.

res.status === 200 && (_result = 'OK', console.log('Success.'))

// That's equivalent to:
if (res.status === 200) {
  _result = 'OK';
  console.log('Success.');
}

We can combine logical operators && and || for multiple-choice branches.

(res.status === 200) && (
  _result = 'OK'
) ||
(res.status === 404) && (
  _result = 'Not found'
) || (
  _result = ''
)

// That's equivalent to:
if (res.status === 200) {
  _result = 'OK';
} else if (res.status === 404) {
  _result = 'Not found';
} else {
  _result = '';
}

Loop

You can scan an array with Array.forEach() for a simple range-loop.

new Array(100).fill(0).forEach(
  (_, i) => (
    console.log(i)
  )
)

// That's equivalent to:
for (let i = 0; i < 100; i++) {
  console.log(i);
}

Or, for a generic conditional loop, you can use the built-in function repeat().

void ((
  n, i
) => (
  n = i = 1,
  repeat(
    () => (
      n *= i,
      i += 1,
      i <= 10
    )
  )
))()

// That's equivalent to:
let n = 1, i = 1;
while (i <= 10) {
  n *= i;
  i += 1;
}

Deviation from ECMAScript

PipyJS is designed specifically for speed and with constructs which are crucial to the design of writing high performant network stream processing logic. Below section highlights the differences where it deviates away from standard ECMAScript or misses the implementation for:

• Object Oriented Programming (OOP) constructs - no user-defined classes or constructors, no prototype system
• Control Flow
• Keywords
  • break , case , catch , continue , debugger , default , do , else , finally , function , for , if , return , switch , throw , try , while , with , yield , class , import , export , extends , static , super
• Type System
  • BigInt and Symbol
  • Strings are internally stored as UTF-8, and made accessible to scripts as UTF-32. For example "😀".length is 2 in standard JavaScript, while in PipyJS, it's 1
• Variables - There's no way to declare variables with the var or let keywords.

Though PipyJS deviates from ECMAScript standard, but it does provides functional style mechanisms to fill the gaps.

For API and standard library documentation, please refer to Website

Conclusion

Pipy is an open-source, extremely fast, and lightweight network traffic processor which can be used in a variety of use cases ranging from edge routers, load balancing & proxying (forward/reverse), API gateways, Static HTTP Servers, Service mesh sidecars, and many other applications. Pipy is in active development and maintained by full-time committers and contributors, though still an early version, it has been battle-tested and in production use by several commercial clients.

Step-by-step tutorials and documentation can be found on Pipy GitHub page or accessed via Pipy admin console web UI. The community is welcome to contribute to Pipy development, give it a try for their particular use-case, provide their feedback and insights.

Related Article

Pipy - A Programmable network proxy for cloud, edge, and IoT

Ali Naqvi ・ Mar 1 ・ 14 min read

#proxy #servicemesh #devops #productivity