August 2021 Update: Here is part 2 of this post that gets into full detail on DevRel KPIs: youtube.com/watch?v=KrqfqyN2FTs
Purpose
I noticed when the term “Measuring Success/KPIs” comes up, it’s a hard discussion to have as not everyone has clearly defined these metrics or they may be struggling to define KPIs that quantifiably prove the value a Developer Relations (DevRel) organization contributes to the bottom line. . So as a group, we came up with a generic list that a DevRel team out there may find very helpful.
This is intended more as a discussion starter here on dev.to as every item is worthy of its own thread. Or if you have any information that would be valuable, please include it in the comments below, and I can add you as a contributor. (include your twitter handle).
The Seattle DevRel community came together to meet at the Microsoft Reactor to listen to a few talks, network, and learn from each other. On top of that, we did a group project where we compiled lists of the ways we measure success.
“What do you measure that benefits your team?” means the things you track that are valuable to your team, but not necessarily data that the higher ups need to see consistently. Then there’s the KPIs that align with company-wide KPIs. Although there were two questions asked, we realized that some metrics a DevRel team would track wouldn’t work for other teams in the company but they still worked for measuring their individual team's goals and contributions.
The Question:
What do you measure that benefits your team? What do you measure that is valuable to higher ups? How does this align with company wide KPIs?
- Internally evangelizing and getting developers from other teams involved in speaking engagements.
- Defining what success looks like in different social media channels: impressions, likes, reposts, shares, etc. and assigning a value to each action.
- Partner networking, blogs (Sales Qualified Leads) grow usage down the road and turn into paying customers
- MQLs (Marketing qualified leads)
- Engagements for developer focused websites specific to language
- Traffic to your DevRel-managed websites.
- Number of speaking engagements and number of attendees in your talks
- Number of sponsored events / attendance. Leads per event.
- # of champions and engagement with their activities, growth of champions.
- # of partners or developers that are engaging with our content
- Code contributions by external developers and what people are actually doing.
- External growth and feedback
- What "Thought leadership" means for your product and how you measure it.
- Decrease the number of support requests while increasing the numbers of self-served users via the docs
- Customer sentiment favorability survey and what is the score/target for 16. onboarding experience (NPS)
- Growth/number of 3rd party applications/extensions (marketplace, developer activity)
- # of unique visitors to content/engagement of content (sample code, PRs, Tutorials, Bug submissions, live-stream content)
- Decrease time to activation: end of onboarding, time to deploying first project
- Increase time to deactivation
- Measuring post sign-up activities/engagement, usage beyond simply downloading
- Stackoverflow posts, forum activities, Slack forum, any collaboration tool. How many users there? How many questions there?
- Feedback into product roadmap, number of feature requests into product
- Number of user/paid user signups
- Number of 3rd party blog posts/tutorials about product (organic press mentions)
- Awareness of your company to developers you’re looking to hire
- Decrease customer churn
- Attributed SQLs/MQLs (Ex: Visited docs before becoming MQL)
- Organic social media mentions
- Number of Downloads
- Think of where DevRel sits in your organization. It may affect (OKR) 31. objectives down the road.
- PR Analyst involvement
- # of events planned / tracking industry engagements
- Number of partnerships with technologies to create integrations between both products.
- Influencers sharing/writing/mentioning about your product to their developer following
- DevRel solved recognized problems.
- Weekly SDK downloads
- Making sure what you're measuring aligns with company goals
- Awareness of what you are measuring and why
- Landing features that meet established customer needs (and getting the data to establish them)
Discussion
* Discuss an item on the list and add your feedback
* Add your input that isn't on the list to share with the #DevRel community
Group Project Contributors:
Tessa Mero
Jessica West
Carter Rabasa
Melissa Thorne
Paul Oliver
Mike Mast
Aeva
Chris Griffing
Lauren Lee
Ishan Anand
Jon Peck
Geraldine Zanolli
Dawn Parzych
Ann Spahr
Jaime Lopez Jr
Kevin McIntosh
Content Editor Contributor:
Dev.To Contributors:
TBD
Top comments (10)
I love this @tessamero! It's so fun to measure all the things related to DevRel. My personal hard lesson learned is that I find it valuable to separate out the data available for me to reflect on from the metric I'm communicating to the business to celebrate (and justify) the team's work.
I go through a few options in a recent talk at ATO: mbbroberg.fun/talks/#all-things-open
Happy measuring everybody 👋
This is great, but what about companies that already have most of that being measured from their marketing team? Then what is left for DevRel to measure? :-) (In addition to items listed in the original post above).
May I include your information and mention you as a contributor in the post?
thanks for sharing Tessa. Definitely an interesting topic and a very tricky one to get right. I'd be interested in reading a follow-up as to how some of the KPIs can be measured as it isn't always an easy thing to do like looking at GA for a website, other things are trickier.
It's a really difficult area to measure and with GA, it depends if you have a marketing team that manages that. Or if they can separate the non-developer's pages analytics from the developer portal's analytics. If you have an option for self-service, how do we know if these new users came from DevRel's advocating/awareness/reach over time?
The REAL KPIs cannot be measured over time, but that isn't an option for most teams as the higher level people need to see data and tie it to the change in users/revenue in some way. I do know that getting rid of DevRel causes numbers to decrease over a period of time (which is where we want to avoid happening for any of us, which is one of the inspirations behind this post).
yep that's part of the struggle :)
Thanks @tessamero -- we're defining ours across all our open source projects....the question I am posing is how do we filter to the ones that most matter? Do you have a sense of how to find the 80/20 (I'm sure it is situational, but any feel?)
The one's who matter most? What is your payment structure? Is it a free OSS project? Do you have contributors that make up most of your community? Is it a subscription payment? Also, check out Matt Broberg's slidedeck he posted above in the comments that includes how they measure success for their open source project!
Thanks for this article Tessa. Given the strange times we are in during this COVID-19 phase, I'd be interested in learning about the modifications in KPIs reflecting the post COVID-19 world. Keep sharing
Thanks for this comment. I haven't visited this post in awhile. There will be a Part 2 coming up soon once I put it together! I have a lot to say and share with others based on the experience since this post. :)
If anyone would like an update, I've done a lot of research and studying on DevRel KPIs in the last 2 years and sharing my knowledge here:
youtube.com/watch?v=KrqfqyN2FTs