Reflections on Stack Overflow: Building Successful Communities

Today is my last day at Stack Overflow. For most of this year, I was the lead product manager for their flagship product, public Q&A. In my last couple of weeks there, I spent some time reflecting on what I have learned and how my perspectives as a product leader matured. As someone deeply invested in building open developer communities, I continue to refine my opinions about how to approach building community-facing products.

Based on these reflections, the following is my own personal perspective on what a product leader at a place like Stack Overflow can do to be successful (and hence make community products successful). Although they are pretty specific to Stack Overflow, I am taking what I’ve learned with me to my next endeavor and you may find them applicable to your world, too.

• Make Stack Overflow work for everyone.

• Be ambitious and a driver of positive change in the face of inertia.

• The community is an asset. Experiment, iterate, and listen.

• Create simple defaults and tear down configuration and customization.

• Keep the mission and impact in perspective always.

Building a successful community

Make Stack Overflow work for everyone. Do not indulge “us versus them” dynamics.

There are many types of people who use Stack Overflow. The main segments for Public Q&A are: Askers (including people who just look up answers to existing questions), Answerers, Curators, and Moderators. The interesting thing about Stack Overflow’s user base though is that all of these segments (of real people!) interact with each other in feedback loops:

• People ask questions and other people answer those questions

• Curators identify quality issues and other people receive that feedback

• People flag content that doesn’t belong on the site and moderators handle those flags

Any initiatives that only serve one segment put the community and product into imbalance. In reality you’re making changes in a complex, dynamical system of real people in a community. Emphasizing one type of user also makes things unnecessarily personal. By “personal” I mean about people, not about the system/software. I observed that this triggers an “us versus them” backlash which is not necessary and causes users to suffer. Instead, focus on the system itself and how all different kinds of users participate in it to interact with each other. I believe that this is an approach that will inherently make Stack Overflow more welcoming.

Here are some things I kept in mind when thinking about how to make Stack Overflow work for everyone:

• Always ask “How does this impact users?”. In specs/briefs I write, I always explicitly consider all participants in the relevant feedback loop. Going through this motion can uncover unanswered questions, highlights your thought process for readers, develops a muscle for user-focus.

• Treat everyone as a valuable person as part of the system. Even if I’m working on something mainly designed to help one type of user, it can go a LONG ways both internally and externally to demonstrate care for everyone. Other people will pick sides or favorites, but you as the chief advocate for your users cannot succumb to this.

• Think holistically. I tend to like to think of user journeys in product work. For a given feature, how did users arrive there? What’s the next step? Considering all participants at each touchpoint in a user journey is a natural extension of this thinking.

Be ambitious and a driver of positive change in the face of inertia. Exude optimism.

The team focused on public Q&A is shockingly small: four IC developers. And the surface area of responsibility is enormous: all of Stack Overflow and the Stack Exchange network serving millions of people every month. Stack Overflow is a top 50 site in Alexa rankings. Having the impact the user base needs and deserves requires efficient use of resources paired with a healthy openness to risk. Otherwise it’s too easy to slip into conservative bug-fixing mode to just keep the status quo afloat.

The good news is that the team is super capable when pushed to dream big. When I joined in early 2019, we made 7 internal commitments of which only 28% were hit. In October, we made 31 internal commitments and hit an 84% completion. This was possible despite receiving no additional developer resources during this timeframe.

Here are some tips for keeping functional teams productive and engaged:

• Constantly clarify ambiguities. The biggest thing a product manager can do to help team productivity is clarify ambiguities. In the face of chaos or shifting priorities, the best thing you can do is make sure to shield the team as much as possible so they can stay focused on productive project work. In the day to day, stay engaged with what they’re working on and answer their questions.

• Accentuate the positive. This team has historically been subjected to a lot of strife. Be empathetic, but do your best to sincerely highlight successes and positive motivators.

• Connect the team with end users. Making the team clearly see how their work has impact is a morale boost and motivator. Some ways to do this include inviting developers to join user interviews, reviewing and discussing research with a broad group of people, and holding project retrospectives that include preliminary data on feature performance. In a small example, I always included a user quote in the team’s weekly company-wide report.

The community is an asset. Experiment, iterate, and listen to your users the whole time.

Rapid experimentation, incremental releases, and iteration is an opportunity to keep a dialogue with your users open. It’s a venue for you to not only involve them in the conversation, but to shape it by sharing and aligning on goals. Like I once told a coworker, “We’re both on different vectors. I share mine with you and you share yours with me and we both get a little closer and amplify our efforts.”

A number of examples of this approach took place as part of our bigger efforts to improve experiences related to question closing. In one recent project, we worked to ship the new post notices feature early and we were responsive to feedback from users. This led to making lots of small improvements before rolling the change out more broadly. In another example, we engaged the community closely in testing different thresholds for voting to close and reopen questions, including transparently sharing the results of our experiments.

Stack Overflow is a website with a huge user base. Not taking advantage of the ability to run tests and get feedback from enthusiastic community members is a missed opportunity. Do it in complement with other research methods. At the same time, these feedback loops threaten to slow things down and can be emotionally draining. Your mileage may vary (and a lot seems to depend on individual tastes), but here are some quick tips:

• Don’t take things personally. You’ll get a lot of critique, positive and negative. As a product manager you have to emotionally separate yourself from the features you work on as much as possible. Instead, invest your energy into focusing on the objectives you share with end users and other stakeholders.

• Delegate communication. It can be exhausting and extremely time consuming to interact deeply with a critical audience. Lean on people who are experts at this. For example, you can try strategies like ghostwriting blog posts or asking others to synthesize feedback for you in a report.

• Take a principled stance. Decision-making is easier when you already have clear objectives and guiding product principles to use as heuristics. So form these early and lean on them aggressively to quickly resolve feedback rather than debating minutiae at length.

Where appropriate, experimenting and iterating on ideas in public is also a great way to battle harden new features before introducing them to Teams customers, Stack Overflow’s Private Q&A product. If something is successful at Stack Overflow’s scale, it may work for a company’s internal knowledge management, too.

Create simple defaults and tear down configuration and customization. Do not try to please everyone.

Stack Overflow is too complex. It’s hard to use for people who are not power users, it’s difficult to maintain for engineers, and it’s a challenge to design and build changes that work around tons of variants. This is not sustainable for such a small team with a huge mandate. For example, take user profiles. They’re a mess of information overload, they’re bogged down with tons of settings and togglable preferences (some only used by a tiny number of people), and users can maintain unique profiles for each and every network site that they’re a member of. Then there’s the overall network profile. And chat profiles. Then profiles for Teams (private Q&A), too. Oh, and they’re not mobile responsive. Yuck.

Taking things away by simplifying them will be hard, but it’s necessary for the future of Stack Overflow. When I first joined Stack Overflow in March, the team launched the new Ask Question Wizard. This created a whole new default question-asking flow for users with less than 111 reputation while other users had the old (10 years old) form intact.

The UX of the wizard was clunky. We heard this in feedback from a random sample of people who responded to our new monthly site satisfaction survey. And maintaining it AND the existing form was an immediate timesuck as we battled bugs and strange edge cases arising from technical workarounds required to support this complex feature. It wouldn’t surprise me if its measurable benefits actually stemmed from how hard it was to use. Therefore, we went back to the drawing board, taking what we learned from the wizard plus additional research, and launched a redesign of the question-asking form for everyone. The data (quantitative and qualitative) and increased simplicity led us to consider it a huge improvement. Plus, it’s now way easier to iterate on (since of course it’s still not perfect!).

Do more to reduce forking paths and little used customizations. It’s not realistic to continue supporting these given extremely constrained resources and minimal benefit to users.

“This web of time – the strands of which approach one another, bifurcate, intersect or ignore each other through the centuries – embraces every possibility. We do not exist in most of them. In some you exist and not I, while in others I do, and you do not.”

― Jorge Luis Borges, The Garden of Forking Paths

Here are some tips for approaching this:

• Make data-driven decisions. Use data (quantitative and qualitative) at every level to inform how you solve this problem. Use data to prioritize which things to tackle first and then use data to decide what configurations are most important, for instance.

• Involve all stakeholders in decision-making. You really, really have to know who is dependent on something and how when you’re planning to take it away. Make sure to involve the right people in the conversation and, as much as possible, share the same data you have access to (and invite them to ask new questions, too).

• Have empathy. Loss aversion is a very real thing. Even if simplifying something is the best thing for users by all other accounts, taking something away still hurts. And this impacts not just end users, but the people who originally worked on a feature. You can have empathy by understanding how they use the feature and asking about the historical context around its original creation.

Keep the mission and impact in perspective always.

With respect to mission, a lot of people are passionate about Stack Overflow and its community. There are a lot of loud voices and strong opinions pulling in many directions. Do what you can to keep focus on the bigger picture, both inside the team and externally. In this period of transition for the company, the mission is still being honed, but here are some things I personally believe to be important:

• Stack Overflow is a source of high quality question and answer pairs. Do things that continue to grow this content, enrich it, and make it easier to access and use for all developers. But, Stack Overflow is not equivalent to Wikipedia. It’s not made up of facts frozen in time. More than a traditional encyclopedia, relevant information and practices in Stack Overflow’s community change as the developer landscape evolves. Make Stack Overflow a living, evolving library. It will never be “done”.

• Make Stack Overflow for all developers. It started as a place to serve programming enthusiasts and this was a successful way to catalyze the creation of a community and corpus of technical knowledge. Over time, it has clearly shown potential to make lives better for anyone who writes code. Figure out how to make Stack Overflow a place where anyone in the world who codes has a place to participate and learn.

With respect to impact, this is where it’s all too easy to fall into a trap of listening only to a small minority of people who enthusiastically offer their voices. These voices are super important to be sure, but they need to be balanced. In my time at Stack Overflow, we put a lot of effort into filling gaps in who we listened to. The vast majority of people who use Stack Overflow don’t have accounts (or aren’t logged-in) yet we had virtually no way to hear from them until we launched a satisfaction tracking survey last summer. At the same time, we also strengthened ties between Community Management and product as a way to more scalably incorporate feedback from highly engaged users into the work we prioritized. They need to have a seat at the table in all of major feature work in order to continue to have a big impact on the solutions the team designs. They are experts on different types of user segments as well as the system and are critical to building Stack Overflow to better serve everyone.

Continue keeping impact in perspective and strive to better serve the needs of all types of users. It’s difficult, painful work because limited resources means important people may feel ignored. But I have confidence that progress will continue to be made on this front.

In conclusion

These beliefs are a reflection of my own personal values as a growing product leader who’s passionate about building great experiences for global developer communities, and as someone who has a great deal of optimism about the future of Stack Overflow. If I had to sum them up, I’d say I believe in unrelenting user focus and optimizing for global, rather than local, minima. I’m excited to take these values, along with everything I learned at Stack Overflow, with me in my next adventure: rejoining the Kaggle team at Google.

Thank you to Shog9, Donna Choi, and Julia Silge for their feedback on drafts of this post.

–

Follow the conversation and discuss on Twitter.

Megan Risdal 👾

@meganrisdal

Read my reflections on Stack Overflow and how my product thinking around building for communities has matured mrisdal.github.io/blog/posts/ref…

17:18 PM - 06 Dec 2019