DEV Community

Havoc Pennington for Tidelift

Posted on

If your open source dependencies are a mess, we’ve got you. Introducing catalogs.

Today we’ve added a new feature we are calling catalogs to the Tidelift Subscription. Catalogs bring managed open source to life by providing a mechanism for customers to create and maintain an organization-wide inventory of open source package releases that just work. 

Simultaneously, catalogs provide a mechanism for Tidelift, working with our network of maintainers, to pre-build a data-enriched inventory of known-good, issue free open source packages that feeds into each subscriber’s customized catalogs. Tidelift subscribers receive a feed of data and updates from Tidelift-managed catalogs, helping them keep their own catalogs high-quality and up-to-date.

So that’s the what. Now let us go back to the why.

Over the past few years, we’ve talked to hundreds of organizations about how they manage their open source dependencies. Most of them fall along a spectrum between one of these two extremes.

Screen Shot 2020-07-07 at 2.50.27 PM

1. Distributed approach (aka “move fast”)

Developers in your organization bring in new open source components on their own, without many controls. After all, you don’t want to set roadblocks in the way of your developers being able to deploy as quickly as possible. 

But as you multiply this by hundreds or thousands of applications, each using a large number of open source dependencies, it creates the potential for a maintenance and security nightmare. You often don’t know which dependencies are being used and how they are (or are not) being secured and maintained, and by whom. 

You’ve resisted putting in place too many controls, but the risks are getting higher, and the maintenance headaches are getting worse.

2. Centralized approach (aka “stay safe”)

Your organization can’t tolerate the risk of a maintenance, security, or licensing emergency with an open source dependency. No one wants to be the next Equifax. So you’ve put strict controls in place. Scanning tools flag issues with the components you are using and block builds. Approvals for introducing new dependencies take days, weeks, or even months to weave through the bureaucracy.

The end result: Cranky developers who can’t get much done. Builds blocked at the last minute. A backlog of unresolved issues flagged by scanning tools that no one knows how to fix. Meanwhile, development slows, good developers get discouraged, and no one is happy with the status quo.

It’s become clear: scanning by itself isn’t enough...

We hear from organizations every day that while scanning tools are useful for identifying issues, identification on its own is not enough without a clear way to help resolve those issues. 

Scanning tools take one problem with an open source package (say, a security vulnerability or missing license), and create an issue for every application (and every developer), using that package. The result: work proportional to M packages times N applications. Ouch. Moreover, the issues arise late in the development lifecycle.

So we asked ourselves, what might a better approach look like? How can we help organizations solve the issues that their scanners flag, while getting the benefits of a distributed approach (move fast) AND the benefits of a centralized approach (stay safe) at the same time? 

Staying safe without sacrificing development speed

The biggest and most well-funded Internet giants have identified the need to move fast and stay safe, and have come up with a solution to do both at once proactively. Here is an article that describes Google’s approach, for example.

These large organizations often create a library of pre-vetted, known-good open source package releases. Developers can use these without fear of late-in-the-game deployment blockers. Vulnerabilities and license concerns can be reviewed once, centrally, and addressed for the entire organization at once.

To work in this way, organizations need to solve several problems:

  • A way to tackle the sheer amount of review work, especially for initial adoption when there are thousands of packages already in use;
  • Efficient workflow for developers and reviewers;
  • Accurate data to power workflow automation and policy compliance.

This approach takes a lot of time and people power—which is why only the richest technology companies have been able to afford it—until now.

Finally, good answers to basic questions about open source 

Tidelift catalogs provide a way for any organization to get issue-free open source packages without the expense of vetting them wholly on its own. Instead, the Tidelift Subscription allows you to offload that responsibility to Tidelift and our network of independent maintainers—saving you time and allowing you to focus on building your apps. 

With Tidelift catalogs in place, you can now definitively answer questions like these:

  • "Can I use this package? Just give me a clear yes or no."
  • "What’s the single source of truth for which packages and versions are OK?"
  • "Is there a repository of known-good artifacts that everyone can use?"
  • “Who’s on the hook for maintaining our open source components?”

With the backing of Tidelift and our network of independent open source maintainers, you will have reliable, timely, and often proactive fixes in-hand for the components you rely on.

How catalogs work

Create-catalog-squareCreate your first catalog. Import packages from an artifact manager (such as JFrog Artifactory) or your existing applications’ bill of materials. Subscribe your catalog to one or more Tidelift-managed catalogs, backed by our network of open source maintainers, and then add your own customizations.

Define-standards-squareDefine your standards. Define the security, compliance, and legal standards you’d like your open source dependencies to meet, and then achieve those goals—building on the work Tidelift has done for you.

Catalog-updates-squareTidelift keeps your catalog current. Tidelift, working together with our partnered independent open source maintainers, will continuously provide security updates and track maintenance and licensing data, along with recommended fixes we arrive at for our Tidelift-managed catalogs.

Add-packages-100Add new packages. Your developers have a streamlined workflow to request new additions to the catalog as needed, and your security, licensing, and technology experts have a streamlined workflow to evaluate each concern only once no matter how many applications are affected.

Multiple-catalogs-100Create more catalogs. If desired, the Tidelift service allows you to create specialized catalogs for different teams or deployment scenarios, and to share common work across all your catalogs.

Learn more

Top comments (0)