Building in Real-Time with Pushpin

We are excited to announce that Pushpin is now a part of the Fast Forward program — how Fastly empowers everyone to build a good, open internet together! Pushpin is a proxy server that pins client connections open, making it easy to build real-time API endpoints and is the Open Source implementation of Fanout, Fastly’s Edge Messaging System. Pushpin supports HTTP streaming, HTTP long-polling, and WebSockets and since Pushpin is a proxy server, the backend can be written in any language - convenience libraries are available to make integration easy.

Pushpin is extremely versatile and unique making it perfect for use with your existing APIs and microservices.

• Keep your API logic in one place: No need to write a separate backend in a different language just for real-time.
• Perfect for microservices: Forget centralized message brokers. Give each microservice its own Pushpin instance.
• Stateless development: Pushpin keeps track of state so you don't have to. This works great with serverless backends, too.
• Scalable beyond your needs: Pushpin is horizontally scalable. When you need to grow, just add more instances.

How does Pushpin work?

Pushpin communicates with backend web applications using regular, short-lived HTTP requests. This allows backend applications to be written in any language and use any web server. There are two main integration points:

• The backend handles proxied requests. For HTTP, each incoming request is proxied to the backend. For WebSockets, the activity of each connection is translated into a series of HTTP requests sent to the backend. Pushpin’s behavior is determined by how the backend responds to these requests.
• The backend tells Pushpin to push data. Regardless of how clients are connected, data may be pushed to them by making an HTTP POST request to Pushpin’s private control API. Pushpin will inject this data into any client connections listening on that channel.

There are several ways Pushpin can fit into your existing architecture:

Simple Backend:

The most typical use for Pushpin is in front of a typical web service backend, where the backend publishes data directly to Pushpin. The web service itself might publish data in reaction to incoming requests, or there might be some kind of background process/job that publishes data.

API Management:

Pushpin can be combined with an API management system, due to most API management systems working as proxy servers as well, meaning all you need to do is chain the proxies together.

Messaging Queue:

If you have a lot of data to push, you can introduce an intermediate message queue. Backend processes can publish data once to the message queue, and the queue can relay the data via an adapter to one or more Pushpin instances that have subscribers for a given channel.

Microservices:

In a microservice environment, Pushpin makes it easy to listen for instant updates from other microservices without the need for a centralized message broker. Each microservice gets its own Pushpin instance, and microservices communicate with each other via your organization’s own APIs rather than a vendor-specific mechanism.

Community & Resources:

Looking to get started with Pushpin? Here is a robust set of resources to help you get started, including a set of real-time examples on Pushpin.org.

• Live Text Editor - In a world where collaboration is key, learn how you can use operation transformations to create a collaborative text editor
• Leaderboard - Use Pushpin to display a sorted list of scores using HTTP streaming.
• Multi-user chat room - Create a multi-user chat room using HTTP Streaming
• To-Do List - Create an interactive to-do list with updates sent over Pushpin
• Webhook Inbox - Easily create temporary URLs that can receive and store webhooks. The user interface displays new webhook requests in real time as they are received.

If you have questions or want to learn about Pushpin, feel free to check out the project on GitHub, the new developer forum, the Issue tracker, or reach out on Twitter (@pushpinorg)! Commercial support for Pushpin is provided by Fastly Fanout.

Open Source Software:

Pushpin is a proxy server that pins client connections open, making it easy to build real-time API endpoints and is the Open Source implementation of Fanout, Fastly’s pub/sub style message system. Open source is a part of Fastly’s heritage — we’re built on it, contribute to it, and open source our own projects whenever we can. What’s more, we’ve committed $50 million in free services to Fast Forward, to give back to the projects that make the internet, and our products, work. To make Pushpin and Fanout possible, we used:

• Rust - Systems programming language
• ZeroMQ - Universal messaging library
• Qt - Foundational C++ library
• OpenSSL - General-purpose cryptography and secure communication
• Httparse - Push library for parsing HTTP/1.x requests and responses

We did our best to contribute back to the community by submitting issues and bug fixes that we implemented as part of our work. We are sending our deepest gratitude to the people that created these projects. If you’re an open-source maintainer or contributor and would like to explore joining Fast Forward, reach out here.