This blog is based on the video "Build Authorization like Netflix with Open Source"
Netflix has over 220 million active users and is worth over 100 billion dollars. With such an enormous user base, they are responsible for managing a vast amount of personal information. A big part of that is ensuring relevant people have the permissions required to access that information, while others do not.
How does Netflix handle the challenge of managing its authorization? Where does open-source come in? How can you adopt this solution? (Or build something even better) Let’s find out.
With Great Power -
In 1997, Netflix was little more than an upstart DVD rental company. Fast forward two decades, and Netflix has become one of the biggest TV and movie studios in the world.
As a company grows, the responsibility it has towards its customers grows as well, and security becomes increasingly important with every new user joining the platform.
The first challenge is authenticating users when they log into the system - that’s authentication. Once users are in the system, the second step is to decide what they have access to - that’s where authorization comes in.
Why is authorization critical?
Authorization (Not to be confused with authentication) is the process of managing access to resources **based on a user's identity and the permissions **assigned to that identity. This is typically done by comparing a user's credentials against a set of rules (policies) to determine what they are allowed to access.
Authorization is crucial for Netflix - not only to make sure only paying customers have access to shows but also as a means of maximizing potential revenue. How you may ask? By being able to tailor shows based on specific countries or user interests, by offering purchasing power parity - adjusting their prices in accordance with income levels per country, and more.
Authorization is a complicated task
Writing authorization policies is quite a complex task. To address this issue, Netflix chose Open Policy Agent (OPA) - an open-source general-purpose policy engine that unifies policy enforcement across the stack.
OPA provides a high-level declarative language called Rego that lets you write policy as code, along with a simple API to offload policy decision-making from your software (As pairing authorization logic with application logic is a bad idea). OPA can be used to enforce policies in microservices, Kubernetes, CI/CD pipelines, API gateways, and more.
The thing is - Rego is quite hard to master, thus limiting the ability to manage policies to a very small chunk of people. Netflix encountered a problem where very few people in the organization could actually write Rego policies, yet they wanted to distribute the ability to create and manage policies across the organization.
How did Netflix solve its authorization problem?
Netflix built a UI on top of OPA, which allowed them to create Rego policies and simplify the process. That solved the issue, but then another problem emerged:
Once the policy was in place, did they actually capture its intent?
They knew in plain English what they wanted to achieve with the policy, and they proceeded to define it in the UI, but they didn't know if it would actually perform. To solve this issue, Netflix ended up building unit-testing mechanisms for the UI.
You want a policy to be implemented in the system? Write it, write a test for it, and make sure that the test passes. Before you save and the policy change gets pushed, all the tests are run, and then if they all pass, the changes get applied to production. Voila.
This allowed Netflix to create a solution on top of open-source components, saving them much of the effort it would take to build a homebrew authorization layer from scratch. Unfortunately - Netflix kept this solution to themselves, never exposing it to a wider audience.
A great video by the CNCF where Manish Mehta and Torin Sandall from Netflix tell the story of how Netflix solved authorization with OPA in much more technical depth:
How can I implement this solution?
While Netflix never open-sourced their solution, the solution they built on top of OPA inspired another open-source project: OPAL
OPAL (Open Policy Administration Layer) is an open-source administration layer for OPA that allows you to easily keep your authorization layer up-to-date in real time. As you push updates to your application's stores (e.g. Git, DBs, S3, SaaS services) OPAL will make sure your services are always in sync with the authorization data and policy they need.
A similar high-level architecture to Netflix's is expressed within OPAL:
The Aggregator is the OPAL server, the Distributor is the split between a server and a client, and the Updater is the OPAL client.
You can learn more about OPAL’s architecture here
Using this inspired approach allows OPAL to aggregate policy and data from different sources, and integrate them seamlessly into the authorization layer in real-time. The project is free and available to everyone as a public project and is already being used by companies like Tesla, Cisco, Palo Alto Networks, and Walmart.
If you want to go even further, Permit.io provides a no-code UI that allows you to create, manage and enforce Rego policies and is based on a combination of OPA and OPAL. Allowing you to implement complex RBAC and ABAC policies into your application, and manage them with a simple UI anyone in your organization can use.
Help OPAL grow
OPAL is an ongoing open-source project which is already keeping hundreds of policy agents updated in real-time. You can join OPAL’s Slack community to chat with other devs who use OPAL for their projects, contribute to the open-source project, or follow OPAL on Twitter for the latest news and updates.
Top comments (0)