We paid developers to record themselves using an eSignature API

Introduction

eSignature as a space has been heating up for a while now. Look at Dropbox’s $230 million acquisition of HelloSign in 2019 or PandaDoc’s recent unicorn status for evidence of the growth in interest in the sector. The high valuations reflect the fact that signatures are involved in some of the most pivotal moments in the lives of businesses and consumers: financings, home loans, sales deals, and hiring employees. Who wouldn’t shell out money to make a critical process more convenient and more secure?

In an effort to stand out in this competitive market, the major players have all released API versions of their product. For API customers, the value proposition is that they can request legally binding signatures without requiring customers to leave their platforms. For example, loanDepot, a DocuSign customer, can have a mortgage applicant sign something without ever sending the customer away from the loanDepot website. For e-signature companies, having an API builds defensibility and drives usage, among other advantages that we discuss at greater length elsewhere.

How’s a developer to choose among these great API products? The product teams would all have you believe they’re the easiest to get started with, but at Usabl, we believe the proof is in the pudding. So we paid developers to spend 30 minutes on a task that seems deceptively simple, at first: sign up for an eSignature API and email this sample document to yourself. Developers quickly realized that this was a challenging task requiring navigating some combination of the graphical UI, Postman collections, APIs to exchange auth keys for tokens, and API reference documentation. And only one of these four eSignature companies stood out. Let’s take a deeper look.

Methodology

For each API, we paid 2 or 3 developers to do roughly the same steps:

1. Sign up for an API key
2. Upload a PDF using the user interface
3. Send this PDF to yourself for signing using the API
4. Sign the document

These were unmoderated tests, meaning that the developers were alone in a Zoom meeting while completing the task. We did, however, use Usabl’s built-in support for timed steps to prompt testers with the correct answer if they spent more than 5 minutes on steps 1, 2, and 4, or more than 10 minutes on step 3.

DocuSign – the brand-name

Founded in 2003, and having gone public in 2018, DocuSign has a $25 billion market capitalization, almost 3x that of HelloSign parent company Dropbox. This is the company businesses and consumers think of when they think of e-signatures. So, how usable was their API?

• 3 out of 3 developers could not successfully complete onboarding
• On average testers spent 43 minutes before giving up
• All testers were blocked by the fact that DocuSign only allows OAuth flows for authentication, so they couldn’t figure out how to set up a redirect URL to grab a token for authentication, even when shown a Loom with the correct answer. One tester used the C# SDK, one used the Python SDK, and one hit the REST API directly.

Overall grade: F

PandaDoc – the upstart

Founded in 2012 and still privately held, PandaDoc is the youngest of the companies we looked at. It also appears to have the newest API, with this announcement coming in September 2021.

• 2 out of 2 developers gave up after an average of 24.5 minutes
• Both developers encountered flakiness. One actually did get the document despite getting an error response. The other kept getting 503s without any error message explaining why
• Explanation: while in many ways PandaDoc also deserved a failing grade, the fact that one tester did actually succeed (despite not realizing it) means we can’t give them the F we gave to DocuSign

Overall grade: D

HelloSign – the challenger

Founded in 2011 and acquired by Dropbox in 2019, HelloSign has the nimbleness of a startup with the backing of a publicly-traded tech company. Has that enabled them to be more customer-centric?

• Both developers succeeded in under 25 minutes
• Explanation: The 100% success rate with HelloSign is far better than what we saw in competitors. The main reason we didn’t give HelloSign an A is that they had some usability quirks, for example the sample cURL request they provide comes pre-populated with values (for example “NDA.pdf”) that are not usually correct and they don’t clearly document that an “@” symbol is required before the filename

Overall grade: B

signNow – the hot potato

Founded in 2011 and acquired twice, once by Barracuda Networks in 2013, then by airSlate, signNow has more of a checkered past than its competitors. It looks like the API is a way to drive usage of airSlate’s broader no-code document management portfolio.

• 2 out of 2 of developers could not successfully complete onboarding
• Both testers struggled a lot with the fact that signNow hides their free sandbox environment behind multiple clicks. One tester did finally find the sandbox after 12 minutes, but the other never found it. Getting a bearer token was complex. One tester spent 51 minutes but never succeeded at this step while the other spent 23 minutes before giving up and asking for help. Once this tester got help they were able to get a 200 response, but only when using a document that had no actual signature fields on it.
• signNow makes signing up without a credit card mind-numbingly difficult. But if you do figure out how to sign up, and you’re willing to take the path of least resistance (authenticating with a username/password, probably not appropriate for production scenarios), you can can eventually figure out how to send a signature request to yourself

Overall grade: D

Overall verdict

While none of these 4 eSignature APIs was completely without friction, the one that clearly stood out was HelloSign. All of the other ones made developers jump through convoluted hoops without clear guidance, including:

• Hiding free trials in non-obvious places (signNow)
• Exchanging an API key for an ephemeral token via OAuth (Docusign) or password auth (signNow)
• Flakiness (PandaDoc)
• Providing incorrect or partially incorrect sample API code (all 4)

Despite the storied pedigrees of the companies in this space, almost all seem to have significant work to do to improve the experience of developers working with their APIs. This space seems ripe for disruption for a company that is willing to invest in improving the usability of a developer-first eSignature experience