DEV Community

Cover image for Why we built Propel Data
Nico Acosta for Propel

Posted on • Originally published at propeldata.com

Why we built Propel Data

Today, we are thrilled to announce Propel Data – an API Platform for developers to easily build in-product analytics with large-scale data. Propel Data enables developers to quickly build usage metering, insights dashboards, product metrics, and advanced reporting without the overhead of managing complex data infrastructure. We are excited to launch our API platform to the market and also to announce a $4.6M funding round led by Matrix Partners with notable angels including founders and executives from Twilio, Databricks, RapidAPI, FaunaDB, Labelbox, and Calixa.

Building in-product analytics should be fast and easy for developers and that’s why we built Propel Data. Empowering your customers with data shouldn’t require a multi-month or year-long project.

In-product analytics are a frequent feature request businesses make to their SaaS vendors. It’s easy to understand why – running a software product without analytics is like running a website without Google Analytics. You wouldn't know how many people visit your site, where the traffic is coming from, and if visitors are converting to customers – you’d be flying blind. Yet, most SaaS products provide no analytics and they are often pushed off and delayed on product roadmaps – despite being a top feature request – leaving their customers flying blind. That’s because building data products is really HARD, until Propel Data.

Why? What's so hard about it? 🤔

It’s taken leading companies like Twilio, LinkedIn, Shopify, and Uber years to build their customer-facing analytic products despite having huge teams behind them. Why is it so hard? Why is it so common for analytics to get punted in product roadmaps or take forever to build? Can’t you just query the database?

There are good reasons for this:

  • Data chaos - Data is often siloed in different data stores and transactional databases don’t perform well with analytical queries.

  • Scarce engineering resources - Data engineering is a very scarce resource and analytics infrastructure and pipelines require dedicated teams to stand up and operate.

  • Inadequate tooling - Business intelligence dashboards that work well for internal use cases lack the customizability required to feel fully native when applied to a customer-facing data analytics use case.

My co-founders and I faced these challenges at Twilio where we were early product and engineering leaders. Customers loved our APIs but needed insight into usage, spend, and deliverability in order to troubleshoot with autonomy and give them the confidence to accelerate adoption. In the early days, we had grand ideas to visualize ‘Google Analytics’-like features in the Twilio Console. Unfortunately, these fell flat when we realized that we needed new data infrastructure that we quickly learned was out of our reach.

It wasn’t until years later that Tyler Wells and Mark Roberts (my co-founders) built the first Voice Insights product for WebRTC at Twilio. Voice Insights is a call quality analytics product that gives customers a complete picture of call performance as calls move through the customer's networks, Twilio's infrastructure, and carrier networks. It started by collecting telemetry from the WebRTC endpoints, processed it in real-time, and gave customers metric dashboards to understand what networks or individual agents had connectivity issues. Today, you’d be irresponsible to run a sizable Twilio Voice deployment without Voice Insights. But the biggest learning - it turned customer frustration into customer empowerment.

Even for the best companies like Twilio, building data products that use large-scale analytical data requires a significant engineering investment and takes years to build. When we left Twilio at the end of 2020, hundreds of engineers were working on the data products across the company. This is not unique to Twilio.

We asked ourselves – does it have to be this hard? Could the complexity be delivered as an API so that every company can deliver in-product analytics without fleets of engineers and investment?

Introducing Propel Data! 🎉

Propel Data is an Analytics API Platform for developers to easily build data products powered by large-scale analytical data. It enables developers to quickly build usage metering, insights dashboards, product metrics, and advanced reporting without the overhead of managing additional complex data infrastructure or creating new data silos.

Propel's GraphQL API and serverless infrastructure make building and running data products seamless. It leverages the data you already have in your data warehouse, operationalizes it, and serves it via highly available low-latency APIs.

Building large-scale data analytics into the product features requires sophisticated data infrastructure because customer-facing products have unique requirements:

  • ⚡️ Low latency → Customers expect fast, snappy product experiences. While internal employees may be okay waiting 45 seconds or even minutes for a query to run, this latency is a no-go for customer-facing products where the data analysis is part of the core workflow that the product enables.

  • 🌊 High concurrency → Products need to support thousands or millions of users. Because data products ultimately serve customers rather than employees, they must support a dramatically higher number of concurrent requests than what internal data tools are designed to handle. And they must do so seamlessly and cost-effectively.

  • 📜 APIs → A contract in code between product and data teams. Shipping a modern data application requires collaboration between data and the product development teams (frontend & backend). APIs codify the contract and SLAs that allow the separation of concerns that keep the product development teams focused on solving customer problems.

  • 🔐 App-centric security layer → Each end-user can only see their own data. In SaaS and consumer products, end-users access their own data from the web and mobile applications.
    This requires an app-centric security layer for multi-tenant or consumer environments, not just the employee-centric role-based access control.

With Propel you can build in days what has taken leading companies years and hundreds of engineers to build.

What are customers saying? ❤️

Over the last year, we have been working with around a dozen design partners to build Propel. We are thrilled to share that Courier, the leading developer notifications platform, will be powering their notifications insights with Propel.

“Analytics was one of the top feature requests that always got punted due to the complexity it involved. With Propel, we were able to ship the analytics we always wanted without having to staff new teams or manage additional infrastructure, all in a couple of weeks.”

  • Seth Carney, CTO at Courier

How do I get started? 🚀

Join our waitlist! We are onboarding customers as fast as we can. We are starting with Snowflake customers and will be adding support for AWS Redshift, GPC BigQuery, and Databricks in the coming quarters.

Stay tuned! Follow us on Twitter or subscribe to our product updates.

Stay tuned! Follow us on Twitter or subscribe to our product updates.

Top comments (0)