There's been some Twitter back-and-forth about the way we've been running the State of CSS survey, and since this raised valid concerns I thought I...
For further actions, you may consider blocking this person and/or reporting abuse
For a post that positions itself as trying to take responsibility for a mistake, there are certainly a lot of excuses provided here. Also, the "make things better" section really seems to put the onus of making things better on readers of this post. What are you all planning to do to make things better here?
So here are the actual steps we're taking:
Do you have any other suggestions?
(And by the way we've been talking about women right here but I'm fully conscious the same conversation could take place around BIPOCs; or even something like accessibility issues.)
I think these next steps sound really strong and I appreciate you messaging me back here to share this.
Reading your points above and responses to others, I do believe you are actively trying to correct the issues with inclusivity in the survey.
I'll continue to think on this and will be in touch if I have advice.
I... don't understand the problem. I deeply appreciate the post, and I am sad that the percentage of 'female' participants isn't higher, I just haven't experienced any different working in the web development industry for 20 years. I used to be the only female I knew in my immediate area, then one of a handful - and that hasn't changed. How can anyone be expected to increase 'female' participation when there just aren't as many working in the industry? I do not think I use any different resources than my male counterparts, so you could not have reached me any other way. I feel like I am missing something: everyone thinks the percentages are off compared to real life? Where's the statistics to support that?
My humble advice would be the following:
Use more diverse platforms: for example, Twitter is used by more Americans than Colombians, and it's possible that gender issues/state of CSS are different for Colombians than for Americans. Also, the second biggest group is Japan and I think your survey isn't in Japanese... I THINK (o x o;;)
Connect with female development/coding groups and organizations: Epic Queen, Woman in Web Development, Vue Vixens and Black Girls CODE come to mind, try to get yourselves known out there in those types of circles c:
Promote your survey to female dev groups, use female development hashtags, connect with important women in the development industry (in different countries if possible, you need that diversity!) that can help you reach girls like me: not popular but have insight to provide.
You need to find your market segment, it won't come finding you!!
Hopefully this small advice helps :3
It's good that you're aware of this pitfall, but I don't see this as a reason to continue on as you have. This strikes me like a too-convenient excuse. Many women in CSS would probably see it as an opportunity, not a burden. There's no harm in actively involving more women while agreeing to listen to them if they say they need more from the rest of the team.
To be clear, we absolutely welcome feedback and involvement from women, sorry if I didn't make that clear.
It's just that from my own discussions with women on this very topic, it's become clear that a common response to raising any kind of representation issue is to put the onus back on the victimized community to fix it ("oh our conference is not diverse enough? well why don't you send us a list of women who want to participate and we'll consider it!").
I wanted to communicate the fact that I'm aware of this, and for that reason am not expecting others to fix the issues with our own product. Sorry if that didn't come across properly.
Honestly, the implication here sounds a bit like "Women just aren't on the internet, so we don't reach them", which I find odd.
Twitter, Reddit and Hacker News are, at their core, very accessible platforms. If you have something to say, you can create an account and participate, or just lurk and read what others post.
So the assumption seems to be that there is a sort of shadow CSS community with a higher percentage of women, which the survey just doesn't reach because they're mostly inactive online?
With such a web-centered technology as CSS, where would such a community be hiding, and is there any realistic way to actually reach them for future surveys?
Or did I just misunderstand what the post is trying to explain?
I'm not in a minoritized demographic so that's also how I've always naively perceived these platforms. Just create a Hacker News account and leave a comment, what could be simpler!
But if not only minoritized individuals, but also the data itself, are telling us that there are biases on these platforms I think we should listen even if that doesn't match our own subjective experience.
So maybe we'll succeed in reaching out to a "shadow CSS community of women" or maybe we won't, but that doesn't change the fact that these biases exist and that we should take steps to fight them.
The thing is, that's not just a perception. If we're considering the technical side, then that's just a fact, so we can safely assume that on the technical side there's no problem.
The data certainly does indicate that there's factors we don't account for and I'm very curious what those factors are.
[edited: my comment is a response not to the post itself but to another comment, by a popular community member, that ridiculed the criticism the survey got]
I think I understand your sentiment that whenever there's a survey, there's a group of folks voicing their criticism for the sake of criticizing. However, I'd like to just point out that:
I'm not writing this response because I take an issue with how Sacha addressed the criticisms — I was just taken aback by your emotional comment, in which you seemed to mix research factors that do matter and that perhaps don't (thus downplaying the stuff that is important) just because you “have very little patience”. I appreciate how Sacha is open and honest about the limitations of the survey. Having worked as a researcher, I know what a horrible and unreliable tool surveys are and I usually just accept it as a fact and move on. As long as the survey is clear about methodology (who/how was prompted to respond, what the question were, etc.), I'm fine with the results.
In this particular case, I also don't know how much of a factor gender is in CSS-related subjects or how the gender of a respondent influences the response. Maybe it does when it comes to salary, for instance — but then again, this survey is not about salaries in tech and anyway, the salary data is also here all represented equally across countries even though only 21% respondents are from the US, which makes it irrelevant ($1k in Poland would be okay vs $1k in New York would be very little for this role). So, beyond just curiosity factor, I'd first ask if we even need the data on gender/age and whether the criticism is about the survey or rather the general feeling of not being included/considered in tech — these are two different problems.
However, if it should be kept, perhaps also comparing/representing the answers of women/men/nb/trans/other folks could tell us how the CSS experience differs and why.
If I can try and put myself in the shoes of the critics for a second, I think the issue is compounded by the fact that the survey (being named "The State of CSS") tries to speak for the CSS community – and our goal is definitely to make something the CSS community can embrace and be proud of.
So beyond the purely methodological criticisms, I also understand the frustration if you feel that something that's aiming to represent you does not in fact correspond to your own reality.
Thank you for this comment — just to be clear, I was responding not to your post directly but to another comment that ridiculed the criticism.
I am not trying to downplay the frustration people feel with not being represented in tech (I myself am rolling my eyes and speaking up often because of this). Still, I wonder whether gender does play any part in the results — seeing the differences and learning about reasons for them could tell us a lot about the community and CSS alike.
I really appreciate the transparency in your methodology and your openness about the limitations and imperfections of this survey. I’d be happy to help out in the next iterations even if by distributing it in my networks.
I'm missing possible action items for next steps.
And I think reflecting on your bias would also mean to reflect not only on the things you mentioned but having it even broader:
My feedback for 2021 would be to start reaching out to people, do some research and ask many more people (Woman, PoC) while developing this survey.
Some feedback related to what you write:
What is then your plan to increase awareness of women and non-binary people to support this? And not only that, this survey has a high range of responded being White (76%). There is definitely a huge community out there of PoC you are currently not able to reach out.
One way could be to reach out to different folks in these areas, even better reach out to organizations (Black Female Coders, Blacks In Tech,...).
The truth is, doing this survey every year is time-consuming for you (and the rest) and because of that you might accepted existing exclusivity inside developer communities.
For 2021 one major focus should be to a) get more people (Women, PoC) into your processes b) reach out to many of these PoC organizations / thought leaders and ask them What you could do to increase reach. Even better get some of them into your project.
This isn't easy, but it is important and necessary.
I doubt that this is true. It's true that they have a big reach, but I believe your impression of that itself is biased. In general, in 2021 I wouldn't put any person there or make sure to have a better (diverse) range of sites, podcasts etc. (just to name some: Emma Bastion, Sara Soueidan, Lea Verou, Sara Vieira, Shruti Kapoor, Shruti Kapoor, Tracy Lee | ladyleet,...)
Thanks Sacha, good job identifying some of the preexisting bias. It can be difficult to acknowledge this, and especially your own preexisting bias (which we all have). Harder still when I know how much work you team puts into the survey. It can feel very personal and uncomfortable to acknowledge.
If keep working on it it, that won't be uncomfortable and we can start to eliminate some of that bias and reach a wider community - good luck with it. As I've said before I'm willing to help if I can.