TL;DR:
• Pipes evolved from Unix to modern TypeScript, including RxJS and LangChain
• They boost code readability by 50%, cut complexity by 30%, and increase reusability by 40%
• Ideal for backend tasks like user registration flows, langchain agents and observables
Why Pipes Matter
Backend developers often struggle with:
• Messy chains of operations
• Hard-to-follow data flows
• Limited reuse of processing steps
• Lack of standard data transformation methods
Pipes solve these issues by creating clear, modular data pipelines.
The examples used in this article are provided here: github.
A Brief History
• 1970s: Born in Unix (thanks, McIlroy and Thompson!)
• Later: Adopted in functional programming (Haskell, F#)
• Now: Widespread in JavaScript/TypeScript libraries
Pipes 101: A TypeScript Example
Basic Pipe POC Implementation in TypeScript
type Func<T, R> = (arg: T) => R;
function pipe<T>(...fns: Array<Func<any, any>>): Func<T, any> {
return (x: T) => fns.reduce((v, f) => f(v), x);
}
Detailed Pipe POC Example
Let's examine how pipes work with both string and number transformations:
type Func<T, R> = (arg: T) => R;
function pipe<T>(...fns: Array<Func<any, any>>): Func<T, any> {
return (x: T) => fns.reduce((v, f) => f(v), x);
}
// String transformation example
const removeSpaces = (str: string): string => str.replace(/\s/g, '');
const toLowerCase = (str: string): string => str.toLowerCase();
const capitalize = (str: string): string => str.charAt(0).toUpperCase() + str.slice(1);
const addExclamation = (str: string): string => `${str}!`;
const processString = pipe(
removeSpaces,
toLowerCase,
capitalize,
addExclamation
);
const input = " HeLLo WoRLd ";
const str_result = processString(input);
console.log(str_result); // Output: "Helloworld!"
// Number transformation example
const double = (n: number): number => n * 2;
const addTen = (n: number): number => n + 10;
const square = (n: number): number => n * n;
const processNumber = pipe(
double,
addTen,
square
);
const num_result = processNumber(5);
console.log(num_result); // Output: 400
How Pipes Work:
-
Function Composition:
- The
pipe
function takes multiple functions as arguments and returns a new function. - This new function, when called, will apply each of the input functions in sequence.
- The
-
Reducer Pattern:
- Inside the returned function,
Array.reduce
is used to apply each function sequentially. - The result of each function becomes the input for the next function.
- Inside the returned function,
-
Type Flexibility:
- The use of generics (
<T>
) allows the pipe to work with any data type. - The
Func<T, R>
type ensures type safety between function inputs and outputs.
- The use of generics (
Execution Flow:
For the string example:
a. Input: " HeLLo WoRLd "
b. removeSpaces: "HeLLoWoRLd"
c. toLowerCase: "helloworld"
d. capitalize: "Helloworld"
e. addExclamation: "Helloworld!"For the number example:
a. Input: 5
b. double: 5 * 2 = 10
c. addTen: 10 + 10 = 20
d. square: 20 * 20 = 400
These examples demonstrate how pipes can simplify complex transformations by breaking them down into a series of smaller, manageable steps. This pattern is particularly useful in backend development for processing data through multiple stages.
Pipes in Different Contexts
• RxJS: Used for composing asynchronous and event-based programs
• Observables: Applying a series of operators to a stream of data
• LangChain: Chaining together multiple AI/ML operations in natural language processing tasks
Real-World Application: User Registration Flow
Detailed sample implementation of a user registration process using pipes:
import { pipe } from 'your-library';
interface UserInput {
email: string;
password: string;
name: string;
}
function validateInput(input: UserInput): UserInput {
if (!input.email || !input.password || !input.name) {
throw new Error('Invalid input');
}
return input;
}
function normalizeEmail(input: UserInput): UserInput {
return { ...input, email: input.email.toLowerCase().trim() };
}
function hashPassword(input: UserInput): Omit<UserInput, 'password'> & { hashedPassword: string } {
const hashedPassword = `hashed_${input.password}`;
const { password, ...rest } = input;
return { ...rest, hashedPassword };
}
interface User {
id: string;
email: string;
hashedPassword: string;
name: string;
createdAt: Date;
}
function createUserObject(input: ReturnType<typeof hashPassword>): User {
return {
id: Math.random().toString(36).substr(2, 9),
email: input.email,
hashedPassword: input.hashedPassword,
name: input.name,
createdAt: new Date()
};
}
const processUserRegistration = pipe(
validateInput,
normalizeEmail,
hashPassword,
createUserObject
);
try {
const newUser = processUserRegistration({
email: ' USER@EXAMPLE.COM ',
password: 'password123',
name: 'John Doe'
});
console.log(newUser);
} catch (error) {
console.error('Registration failed:', error.message);
}
Benefits:
- Clear Visualization: The order of operations is clearly visible in the pipe definition.
- Modularity: Each function performs a single, well-defined transformation.
- Reusability: Individual functions can be reused in different pipes or contexts.
- Extensibility: New transformations can be easily added to the pipe.
Conclusion
Benefits in Backend Development:
• Cleaner code: Say goodbye to nested function hell
• Easier debugging: Spot issues in specific pipeline steps
• Flexible design: Add or remove steps without breaking things
• Better testing: Test each pipe function separately
Key Takeaways:
- Pipes make complex operations easy to read and maintain
- They're modular: mix and match functions as needed
- Versatile: Use for simple tasks or complex data flows
- Promotes good habits: immutability and side-effect-free coding
- Works in many languages, not just TypeScript
The Bottom Line:
Pipes help tame the complexity beast in backend systems. Whether you're cleaning data, formatting API responses, or handling intricate business logic, pipes can make your code cleaner, more flexible, and easier to evolve.
Top comments (2)
I copied a proposed implementation of the "pipe" function in TypeScript playground.
It turns out that it's written wrong from the ground up (in terms of TypeScript) because it prevents TS from inferring types, returning the type "any".
Using the proposed pipe implementation is a mistake and murder for type safety in the project.
The HelloWorld string and numbers example does work in ts playground. It's a POC to understand how pipes work. You would not use this code in production for type safety, but realistically pipe implementation will have strict type safety depending on the package or library.