Ashley: 0:00

Today we are gonna talk about something that we care a lot about and that we've touched on before in this podcast, which is how gender interacts with our fields of study or computing and software development. And before we do, we wanted to say a couple of things. So first of all, gender is not a binary. As a podcast crew of gender complicated folks, we fully recognize this and embrace this. This is just how a lot of the studies that we talk about today are actually gonna address gender. And the second thing is that if you're a woman or a non-binary person in computing, we love you. We see you. We want you here, and we hope that nothing you hear today is disparaging in any way. And if nothing is just further proof about how resilient and amazing you are.

Cat: 0:43

I'm not sure how to describe this, like"Community Note" from Cat maybe. But I have been thinking about achievement and school and people's cognitive abilities and how we measure human beings for so long, and it sometimes overwhelms me to talk about it. But I also think it's really important to talk about. So I'm gonna try to do my best to give you like a 30,000 foot view of some of the research as I understand it, and especially to probe into like the ways that we misunderstand evidence and misuse research.'cause that's a real hot. You know, topic for this podcast and something close to my heart, and I think a lot of the slippery nature of these biases and these stereotypes and these assumptions, it makes it really, really hard to talk about. In fact, when we were prepping this podcast, I was thinking about how somebody can say a really flippant thing about how girls can't do math, and then it takes you like four months to read all the research and process it and put together like a water tight case. You know, it takes so much effort to talk about these topics, but we deserve to talk about them and to talk about the really good, strong evidence that puts us towards more equity for more people. So I'm gonna try my best. And what we don't get to, we'll link in the show notes.

Ashley: 2:09

Let me set you up. Lemme set you up.

Cat: 2:12

I'm super excited to be here and talk about something not controversial in any way.

Ashley: 2:17

We never talk about things that are controversial on this show, we're gonna start in 2019 in Scotland on a bus.

Cat: 2:28

So I was going to the Scottish Highlands and my Scottish aunt had booked me on a bus that was a student bus. It took off from the University of Glasgow, I wanna say. And so I'm not a student. I was significantly older than almost everybody on this bus. But they take you up and you go stay on the Isle of Skye, you see some sites, and I was doing this to prepare for like a bigger hiking trip that I was gonna have. And I very quickly noticed that I was much older than like all the little college students who were on this bus, but I was having a great time. I was just staring out the window, looking at mountains, excited to be here. I felt really far away from everything. I met a couple of people, the only other people in this group who were also not students, uh, a man and a woman. And they were so lovely to me. We all landed at this bed and breakfast on Skye. And that night they said, Hey Cat, let's all go on a walk. So the three of us, new friends, middle of the Highlands, um, out walking. We felt far away from everything

Ashley: 3:34

Okay, so you're in the middle of the Highlands and you've just met these kind of strangers, but then you share some, you know, common history, let's say, or some, some overlapping interests. Us In a, in a sort of crossing of fates in the middle of Isle of Skye.

Cat: 3:50

We had both worked at Google, me and this guy, and I am a psychologist, and he was a software engineer. And you have to understand. 2019, I was really trying to figure out what to do next. I was kind of in between things. My startup had closed down. I was consulting a little bit. And honestly, the last thing in the world I wanted to do was talk about tech or work or

Ashley: 4:17

You were on vacation,

Cat: 4:18

I was, I was so on vacation, I was so anonymous. I was wearing, you know, jeans and a little hoodie, um, which, which ultimately was maybe too tech. But, um.

Ashley: 4:30

spotted you from a mile away.

Cat: 4:32

There was this moment we're out there in this beautiful landscape and I can just, you know how you can just tell when somebody wants to say something

Ashley: 4:40

They're kinda like making more eye contact than you expect them to make with you.

Cat: 4:46

And he'd like fallen silent, you know, for a while. And, and truly, let me tell you what was going through my head at the time was like, what a nice rock. I'm gonna do a handstand. I wonder if my new friends would like take another picture of me doing a handstand, like on this rock. You know, and this guy turns to me, he says, Hey, listen. And he sort of like, you know, plants his feet and looks at me. And I, I wanna emphasize he was doing this with such a pleasant tone. So non-confrontational. Like it was clear that this was a moment that he had kind of been worrying about, you know, and wondering if he could like, take this chance. And there was also the woman, his friend with him sort of. Looking at him like, do not do this,

Ashley: 5:34

and. you're just like, dear God, what is gonna come out of this guy's mouth?

Cat: 5:37

yeah, it's like the international language of two women looking at each other and she's like, you know what, I'm here, whatever. And I, I have no idea what he's about to say. And he turns to me and said, listen, I just feel like I have to ask you this. You are a psychologist, but you're also someone who's worked at Google and you do math and you do statistics. And I had told him like just a little bit about, you know, what I worked on. Um, and he said. You know, you work on learning and achievement and all of these things and, and you are gay

Ashley: 6:13

These are a lot of very unpredictable factors.

Cat: 6:17

Well, they're correlating into something, right? But at that moment, he said, like, and you're gay. You know, the, the woman who was with us she blurted out something. I don't even remember what, It was like frozen in time, like I just knew he was gonna ask me about women and tech and math and coding and Google. He said, I read the memo about why there aren't women at Google. And at this point I was like. It's okay, you can talk to me and I think I said it's okay. Like to the woman who was with him and I remember thinking. I can't believe this is finally happening to me. I have avoided talking about this for the years since it came out. I remember the day that that memo came out, you know, and here it is. I went all the way around the world and I'm out in the wilderness and here we are. It came for me, the question.

Ashley: 7:13

So for the people who are listening who don't know what the memo is, can we talk about what it is? Because I know, because I'm your wife and, uh, you were in tech when the memo came out. Um, so this is a memo from a guy, I don't even, let's not say his name because I don't even wanna give him credit. This douche bag at Google wrote a big memo about like, what, why women aren't in computing and why they suck at math or something. Tell me, tell us, tell us what the memo said.

Cat: 7:41

So every once in a while a big comment comes out about how there is a large observable gender difference in, uh, pick any number of things. In this case, how many women are in computer science adjacent roles um, how many women become programmers? How many women enter STEM is another version. How many women become scientists? How many women are, math faculty members? Um, you will see the exact same arguments brought up in many, many domains. And, and the memo was a version of this where, um, you know, a document had been put together that said, basically there is a lot of evidence to explain these gaps. And maybe for reasons that we can't intervene on and shouldn't intervene on. And so I think the core of the argument is that, you know, there are fixed, innate, immutable gender differences that should lead us to expect women to be worse at math, women to be worse at the skills of computing. Women perhaps to have less interest in those forms of work, women to have less ability in those forms of problem solving. So these are all like slightly different versions of the same thing. Another famous example is what Larry Summers said. That gender disparities across career pathways in science and engineering are attributable to biological differences, as well as differences in interest and motivation, which usually in these kinds of arguments is assumed to be innate, um, and fixed and also immutable.

Ashley: 9:18

Okay, so this is a memo that drops in 2017. Basically makes the claim that there are fixed differences between men and women, and this explains why there's not that many women in computing and, and a huge piece of this is based in some research about women in math, right? And, and math ability, which is one of the big things we wanna talk about today.

Cat: 9:41

Yeah, so, so it's based in, I think, old research in many cases, warped research or um, or incomplete research. It's also sometimes based in slippery arguments that actually are not justified. So kind of like what you might call like a Motte and Bailey argument where you say, okay, here's one sort of. Thing that we observe in the world. But I'm going to use it to, to prop up this way more radical conclusion. And you'll, you'll see like these sort of very radical conclusions smuggled in. And you know, the thing that I. Had a conversation about in Scotland is just how hard this stuff is and just how, how pervasive these really shaky, weak arguments are and all the deep interesting, um, complicated counter-evidence that people have surfaced. So the first thing that I started talking to this guy about was, you know, let's talk about. A particular day of work as a software engineer, and let's talk about all the different skills and abilities that go into that kind of day of work. And then I will tell you about what I know about, you know, the achievement research on it, and then all of the different effects that like drive, the kind of differences we might see and who ends up getting that job in the first place. So there's like just a massive, massive mountain of things to unpack here.

Ashley: 11:11

Hmm. So I love that this guy is like on this walk with you. Drops this like bombshell of a question and the first thing you do is you respond with a question where you're like, okay, interesting. Very interesting. You would think about this, like tell me about what a typical day at work looks like for you. Like you go into an informational interview with this guy, and I love this so much because this is like so much what of what we try to do in like learning science where the first thing you do is you invite someone to like reflect on their own experience and sort of think about you know, where they are and sort of just. Basically surface some ideas about the topic. So, so you get this guy to say, okay, well I'm an engineer. Yeah. Okay. I spend like two hours of my day coding, but I spend a couple days in meetings and maybe like, I'm doing project management. Who knows?

Cat: 11:58

Mm-hmm.

Ashley: 11:59

Presumably he tells you something like, like this, and then, you know, what, what do you say in response to all of that?

Cat: 12:06

Yeah, so if you can get grounded in your real life and the real complex problem solving of a knowledge work job, suddenly you're out of the realm of stereotypes and you're much more into the realm of like, let's talk about the tangible facts. And I like to start there because I also think, you know. This was a very scary moment for me. Like I felt stereotype threat. I felt like the bottom of my stomach drop out. You know? I felt like I had to represent like all of psychology again. Can I just say, I was hiking in Scotland looking messy, like little goober did not expect to suddenly be the one voice of psychology for the one time. This man felt safe to ask this question though,

Ashley: 12:52

Yeah. How many times have you been in this situation where someone is like, hello, psychologist? Yeah, and I love too, I mean it's like it's very much directly stereotype threat. He's like, Hey, woman in tech who does statistics and actually happens to be queer? Like, let me just frame how I see you in my head and then ask you about. Big ass question about what you think.

Cat: 13:15

I got. I gotta tell you though, the amount of compassion I felt for this guy was. Not trivial was like very present and the amount of compassion I felt for his friend, this woman who was with him, and he said something to me that really made the difference between like whether I was gonna have this conversation or not. He said, I really want to understand the evidence. He was clearly a very smart guy. They were both very smart people, both incredibly well intended, and just that there was genuine real curiosity that was like, I've seen this stuff. And it felt like a lot. I think he even said to me, I don't want to believe that this is true. But it's like pages and pages of evidence. And to me, of course, I was like, oh my gosh, you think that's pages of evidence. I have evidence from millions of students and years of work comparing culture and years of education that says the opposite. But of course you don't know that. And to you this small, tiny, sort of distorted weird study that's actually crappy evidence looks super impressive, but you don't know. How rigorous and great all the research is on our side and on the side of, you know, appreciating the diversity of human potential. So that's what I wanted to get into with him. And I tried to.

Ashley: 14:40

Yeah, so I think what you're summarizing is the fact that there's been. Not just the memo at Google, but there was a former Harvard president who had some statements about, you know, why women don't go into STEM? And many people could name those two people, but they couldn't name the abundance of researchers that are out there tackling these ideas really firsthand. And so we wanna put those names in your brain actually.

Cat: 15:06

Yeah, so let's talk about, one of them is Janet Hyde. So Janet Hyde is a researcher who authored this thing called the Gender Similarities Hypothesis, and Janet Hyde in 2005 went out to get a grant. And she directly says in a piece about this, uh, I decided to put data behind these claims and this Larry Summers, you know, speech that went out and all the cultural conversation that was happening. Well, I went out and I got a grant, and the results of their work is in this paper where they looked at data from. 10 different states because at the time all of this interesting different math achievement data started to be available. Um, I think because of things around no Child Left Behind and the kind of increased availability of education data. So what was interesting about this was Janet Hyde. You know, does this approach that we call a meta-analysis, so if you think about a single study in science as producing like one effect, you know, or one finding, there's gonna be like an estimate, um, of the size of that. So let's say, okay, there's a gender difference in math where we think boys are performing better than girls at math. That the size of that difference, you know, is actually really important. Like, is it a really small difference? Is it a really big difference? And to get that kind of. Knowledge, we actually need to look across many, many studies and many, many instances of performance. So Janet Hyde's really great at doing this kind of meta-analysis, and when you do this, you actually start to see that the math achievement gap between girls and boys is either closing or in many cases has vanished altogether, is close to zero, um, in a lot of grades.

Ashley: 16:55

so one sentence from a 2008 science paper. Janet Hyde is the first author says. Our analysis shows that for grades two to 11, the general population no longer shows a gender difference in math skills.

Cat: 17:09

Mm-hmm.

Ashley: 17:10

That's a big statement, right? So it's even like a straw man hypothesis to be like, well, there are math differences early on in, in boys and girls, right? And that's like one of the linchpin arguments people use when they're trying to argue for gender differences.

Cat: 17:27

Yeah, absolutely. Like we shouldn't see it shift if it was actually a biologically rooted, immutable characteristic. It also shifts when you look across cultures, so that is a really powerful piece of the evidence against this as well. If you look at cross national analyses of gender differences in math for instance, there's a paper that has half a million students. You look at things like the actual variance between boys and girls on math performance is really different for different countries, and it tracks along with gender equity. Improvements. So the less gender equitable a country is, the more there's this stratification that happens between boys and girls. And so that is also very strong observation. You can look at it this way. There are countries where girls outperform the boys of another culture that have a different math education, right? So again, you can see this strong argument for an education effect a social effect.

Ashley: 18:31

So two things came up as you were talking about this. One is an argument I've never been able to prove because I don't think there's enough data to prove this, but I have long argued that the success of a women's soccer team is very much predicted by how much gender equity is in that country. And I think this is really true. I mean, it's, it's hard because like gender equity is also a proxy for wealth of the country in a lot of ways. Um, but I think women's soccer teams are better in places where there's more gender equity.

Cat: 19:03

Well, you can, yeah, you can get at that kind of question by looking perhaps sometimes within a culture and looking at gender equity variation. So you know this, a great way to look at gender equity variation for STEM is to look at fields that are actually more gender equitable and compare them to fields that are less gender equitable. But that. Generally draw on the same skill sets and cognitive abilities. So a great example of this is biology has diversified on gender and like chemistry and math haven't, and people think that that's because of the culture of those fields. Um,

Ashley: 19:40

Yeah. Lots of things going on there.

Cat: 19:42

Another piece of evidence that's really, really, kind of blows people's minds. I think it's, it's, I'm sorry. I got excited and I sound really positive about negative evidence, but that's the, that's the psychology nerd in me. This is so cool. Right. So like a different level of achievement will predict greater benefits for men than for women. And this is so hard to wrap your mind around, but it's actually so important, like. Um, if you're trying to think about who gets to go into engineering, lower scoring, men get to go into engineering at like the same rates as as higher scoring women. So we essentially are bleeding out these high achieving women, and this is one reason that looking at the number of people in these fields who have these jobs is. So complicated. You get an A in math as a woman, it's gonna basically benefit you as much as a man. Getting a C in math benefits them, which is pretty extraordinary. Like if engineering fields required such a strong in math ability, why are we letting so many mediocre at math men into

Ashley: 20:53

Hmm.

Cat: 20:53

I There's an interesting way of framing this, which is if what we are interested in is actually trying to get the people that we need into engineering. We wanna examine multiple pieces of the assumptions we have. So like one assumption we have is that math ability is really required to succeed at engineering and technical careers. However many men succeed at engineering, they're admitted into these pathways at a lower level of math achievement.

Ashley: 21:22

I think another thing that's crazy about that half a million children study is that that is so many students. Right. And it always blows my mind when we are so willing to believe the voice of one or two men over, you know, in their little anecdotal experience with however, like what, 10, 2 50 people they've interacted with in their careers really deeply enough to know versus like these studies of half a million people. I mean, it reminds me of. Anti-vaxxers, right? And the fact that we actually have an overwhelming amount of evidence in millions and millions of children that vaccines are safe. And yet people will always point at the small, at this point, basically anecdotal and flawed evidence from other sources because it reinforces their worldview.

Cat: 22:14

Yeah, people act like social evidence is just not as strong or convincing or objective. The biggest pieces of evidence we have are about social factors. Actually, you know, in that same study, so, you know, the one I mentioned is half a million. The other meta-analysis that Hyde did is 7 million students in the United States, you know, across states. Yeah. But the half a million one, they talk about how the gender equity in school enrollment is a predictor. You know, all of these things that point back to structural, socioeconomic, um, and gender equity explanations.

Ashley: 22:54

So go ahead and unpack with that for me. So school enrollment, like if there's more boys in the school, boys will do better in math. Is that the

Cat: 23:03

Yeah, so this is an, this is potentially an equity variable, right? Because in a situation where, you know, people are treated less equally in a country, you know, girls, girls' education is not valued as highly by society. So girls are more likely to be, let's say, at home taking care of siblings or dropping out, you know, sooner to work jobs and things like that. So you could think about. Enrollment in school is also a variable that's culturally responsive. It's not, it's not like boys and girls are coming into the world and actually getting, uh, what we call a treatment effect. You know, if it was a medical study, right? They're not getting an equal treatment effect all the time. The fact that it's a really good predictor of. Achievement. You know, again, points to social explanations. So it's not necessarily a causal argument, but it's just saying, there's so many structural pieces we need to fix. We can't even begin to measure innate ability until we fix those pieces. Every one of these achievement measures is capturing. Cultural factors and when we use those cultural factors, they provide us with high predictive validity. So this is actually part of the conversation I had with my engineer friend in Scotland, was if we care about what the data let us do and what the data let us predict, these social cultural variables provide us with far more prediction of the variance that we see in the world. So

Ashley: 24:29

And probably an intervention knob as well, right? Like a place we could jump in and actually make change if we really wanted gender equity in these fields.

Cat: 24:39

Absolutely. There's another piece I want to mention, which is people often talk about. Differences in terms of really extreme ends of the distribution. So they'll focus on like people who score at the top 5% of math or something like that. Um, immediately that data gets very difficult statistically, because not a lot of people do that. It's highly variable, you know, compared to looking at those like 7 million learners across the whole distribution. But we have such a bias to like care about. Geniuses and care about like extreme results that people will kind of extrapolate from that. And the gender similarities hypothesis, like some of this research that Hyde has put out says a huge amount of the time, men and women are more similar than they are different. And yet we fixate on the small moments where we observe a difference and then we repeat that over and over again and we publish about it over and over again. But the evidence does not hold it up. Like think about how much. There's a similarity between, you know, a, a single woman and a single man might actually be more similar to each other than two women are to each other, who are like further apart on certain characteristics. So this is like a dangerous thing about looking at our world and just splitting it into two categories, which, you know, we know they're not even two actual clean categories themselves, but that's what we use.

Ashley: 26:08

Yeah, I mean, I hear you saying a few things, which is one, we overweight individual factors and things that we think are like fixed. Like maybe we overweight the idea that, um. Boys and girls when they're young, gravitate towards like different things if that's even true, right? We, we overweight the individual factors versus the structural factors. And then the second thing which you just raised is that we overweight the differences rather than the similarities, right? We are more interested in talking about the things that differentiate us as individuals than talking about the things that unite us and or structural factors that impact. In this case or in these cases, like millions of people simultaneously.

Cat: 26:52

let me just poke holes at a few more dearly held beliefs, okay? A lot of people think, oh, we know that men are better at spatial tasks. No, we don't really, that's actually pretty controversial. I'll link a critical review of this that talks about how the evidence for some of that st some of the stuff that people pull up as like the quote unquote strongest evidence, it, you can challenge it and pick it apart. There's a claim that that boys are more interested in objects and girls aren't. That's really disputed. Some of the early work on that was like very glib about what they said about objects or not. Um, there's a claim that boy infants direct their attention to objects. Um, I love this thing that they point out in this critical review, like people are holding the newborn infants in some of these old studies and like positioning them like toward the truck, you know, or something, or toward the doll. So, you know, I'm, I am. Not at my heart. A cognitive scientist. I'm an environmental effects scientist. I'm a scientist who studies beliefs. You know, there's many pieces to this picture, but I wanna point out that we just are so reductive and so ensorcelled and enchanted all the time. If something's a biological argument and some of this. Evidence is just like not, you know, very strong evidence even for the things that we kind of take as gospel about gender differences.

Ashley: 28:20

Yeah, I think about this a lot, which is like, when do we need a biological explanation versus when is it either not helpful or even harmful? And you know, I think that to be clear, there are. Genetic and environmental differences between people and these change your preferences in the way you operate in the world. Like not, uh, not all women are the same. Not all men are the same. Everybody is a unique snowflake in their own right. And so, you know, like it's not as if the claim is that everybody is the same. And there may be some kind of interesting biases that the brain sets up early on, like sure. But I think like the more interesting question as you said, like, you know, rather than latching onto some biological explanation, which might have some value in some particular context, but probably not here, like rather than latching onto that, what if we pivoted and we said. You know, what are the other things that are much bigger levers in the world that are clearly the levers or the situations that are changing whether or not women go into math or whether or not they go into computing, of which we can point at so many so blatantly, including the memo that we started this conversation talking about. Right.

Cat: 29:41

Yeah. So this is where too, like the work of, of Sapna Cheryan and you know, Meltzoff and other people who've written about how interest is itself a culturally responsive variable. There's not some magical world in which we are observing girls making choices that don't include some amount of their experience and knowledge. You know, that the world is biased against them. So that has been a huge counter argument to the people who say. Well, you know, women just aren't interested in these fields. Women are of course, interested in any huge number of things. Women are also interested in like avoiding being attacked, you know, and in being safe and the choices that they face are different. I wanna pick up the biology piece again though, because I, I think you're right. I understand like, you know, biology is. A real thing that exists. It's your field, but at the, at the same time, there are just so many pernicious like dangerous ways that we think about it that are worth maybe bringing to the surface here. Like even if there are average differences between, you know, two groups of people that we might divide in the world, um, that doesn't mean that we actually talk about those differences in a way that's accurate to biology. And of course, as like two queer people,

Ashley: 31:04

Mm-hmm.

Cat: 31:04

know, we think about this and I wanna point us to this paper. Another paper that is one that I really love called The Future of Sex and Gender in Psychology, five Challenges to the Gender Binary. And it acknowledges, first of all that the gender binary is itself very flawed, um, and also has all this interesting stuff in it about how we assume, you know, that just because we can pick up on an average pattern sometimes that it's, you know. Um, fixed in a certain way or that everybody in a group will have a prototypical like version of that brain. And I think that that's something that people miss a lot and don't understand, like the new science of.

Ashley: 31:45

Yeah. Yeah. I think it's pretty clear like we, you know, you have XX or XY chromosomes, but how that plays out, or say most of us do, but that's not even, that's not even universally true. Right. Um, but then the way this actually plays out in terms of like your brain, which I suppose is a relevant organ, I'm biased, but the relevant organ in this case is like it. It actually could be more of a mosaic of these different areas being, you know, more or less quote unquote like male. And yeah, that's a whole thing we can unpack in a separate episode, which is really fascinating. But yeah, like the, the cutting edge science of this is not even like, you know, that there are like two different kinds of brains in the world or

Cat: 32:29

Yeah, I'll read a couple quotes from this paper maybe'cause I have it open and I, you know, as not a biologist, I love to read this stuff because it really like affirms a more nuanced view of the world. And so they write like, you know, complex sex by environment interactions. Produce a brain structure that's multim, morphic rather than dimorphic. You know, you have. This like very interesting situation where even if we can detect sort of average group differences, it doesn't actually mean any individual you select from that group will be some kind of prototypical female brain or male brain or something like that. And, and reasoning that way can be super misleading. I also think they make a super great point that the causal direction of this stuff. It goes in both ways. So we often think, okay, biology causes behavior. You know, I hate to inform everybody. Behavior shapes. Your biology behavior can change

Ashley: 33:25

else would we go to the gym?

Cat: 33:27

Behavior shapes what part of your brain you're investing in. So we have like domain specific cognition. So Hyde makes this point too, I think, and, and collaborators that, you know, we're never observing a brain that grew up outside of these cultural effects as well. And so, it's kind of just a false dichotomy where we're sort of saying, you know. Is it biology or is it social? I mean, and there's been so many important challenges in psychology where we realized that we had developed ways of testing people and testing for their abilities that had never considered the possibility that someone has the same. Capability, the same aptitude, but they just learn to use it in a different area. And huge advances have come from recognizing that fact and saying, well, what if we took like the same task, but we kind of put it in the context of the problems you actually learned to solve. And we see people excel at it, you know? So that's why I think it's like so dangerous to say, we know how to measure ability

Ashley: 34:30

Yeah, so I think you're I think you're sort of flirting with the idea that like standardized tests are biased because the sort of like context in which they ask questions are often biased towards like, you know, wealthy people or white people. Is that kind of what you're hinting at here?

Cat: 34:47

Um, I would say like assessments more generally, you know, like ability assessments more generally. So like, if you wanna say, does this five-year-old have good attention? Does this five-year-old have good executive functioning? You know, do they have a. Strong working memory, even those kinds of things that people outside of psychology might hear and just think, oh, that sounds very, very objective. Like there's a little box in your brain that says working memory and here's the task for it, and scientists did it and it was in a lab, so it must be, this is the way we measure it. Like there's just kind of a lot of, thinking that it's very, very objective and pure. Some of my favorite recent cognitive science has actually shown that we have been under measuring the cognitive performance of so many children in the world because we never thought to translate these tasks into something that they thought was relevant. Or it made sense, you know, or wasn't silly, silly and boring to them. Um, making it culturally literate. So yes, that shows up in standardized tests as well, like maybe the SAT um, people might be familiar with, but I just want to bring it home that it shows up in, in all these things that people act like are very, like, pure measures of cognition or something. And those are still things that we designed.

Ashley: 36:04

I think this is interesting in the context of like in neuroscience, a lot of the times what we do is we, you know, develop some task for like a mouse to do, for example. And you know, we have to make sure that the task is like something one the mouse can do, and two, sort of just like. It makes sense to it, you know, developmentally and evolutionarily. Right. Um, and, and tasks are more or less what we might call like ecological, and I feel like what you're pointing at is the same idea. It's like ecological validity. Like does this task that we're giving a child actually map onto like what they might normally do in the world and how they might normally exercise their cognition.

Cat: 36:42

Oh, it's completely ecological validity. There was like a devastating review that just came out from a, an excellent psychologist who studies, um, I believe like equity in the assessment of black children in the United States and like no surprise is basically shows that all the standard measures of things like executive function. They're like penalizing these students in a way that essentially just makes them look like they have less ability than they do. We find this again and again and again, like achievement tracks equity, and even like the way that we measure and conceptualize of achievement itself is loaded, you know, with all these. Features, all these implicit choices we made that make them less ecologically valid for some people. So, you know, this happens for women with math. The ability of women in math is not recognized by teachers. So for instance, teachers interpret overconfidence from boys in math classes as math ability, and they interpret it as a social and emotional problem, um, from non-white students and from women.

Ashley: 37:50

But little, Billy had the answer. He really knew the

Cat: 37:53

You know, little Billy was interrupting everybody and you know, little Billy gets to, little, Billy gets to roll the dice on giving his answer and try to give 20 answers. And out of them five are gonna be correct. And then people say, little, little, Billy's a math genius. Whereas. Little Sally like tries to answer twice and gets 50% of her answers correct and she's seen as, you know, average or below average or something like that. So all of these social effects are layered into like our evaluation. Actually a really fun paper that I remember on this. And the title is Just Overconfident Boys. That's the title. And I think they looked at, I'm gonna maybe get this wrong, we'll see, I'll link it in the show notes. But I think they looked even at sibling pairs and their self-evaluations of their own math scores, um, and their own math ability. There's just a persistent. Constant effect where girls who are good at math rate themselves as less good at math than they actually are. Um, and then there's also this sort of. You know, issue where like as you get in and you, you demonstrate high ability in these kinds of domains, we get a whole other area of effects that we call backlash effects and stereotyping congruence effects. So the more that you achieve. At solving math problems as a girl, um, or someone perceived to be female, um, or non-male, the more people penalize you in other ways for that. So they say, okay, well, you're good at math, but gosh, you're a jerk. You know, you could be nice about it and that kind of thing. Um, is is a stereotype in congruence effect so that there's a great paper I link all the time on this called Hard Won, easily lost. And it's about how you have to fight a lot harder to get into the thing. And then once you're there, people are always watching, waiting for any reason to say, um, your ability doesn't actually, make up for the fact that you're a woman, in fact. So you're, you're asked to actually do a lot more to even hold on to your position and hold onto other people's judgments of your competency.

Ashley: 40:10

Yeah, so just covered two ideas there. So the first one is like, boys are overconfident, which often gets them. Recognized by teachers, and we know that that kind of, recognition from other people is something that directly contributes to the development of identity, your own self-identity and your sort of self-efficacy in these fields. And then two, you're pointing out this like stereotype in congruence, which is that when girls, you know, in, in school do better, then they're actually like, called out for that as sort of being outliers basically.

Cat: 40:42

Yeah. So one thing that happens to you if you're minoritized in a field in any way on a social dimension is that you get lower quality feedback all the time. So you get noisy signals all the time. You get people reacting with hostility to you when you actually succeed because you triggered them. You get them policing your behavior because you're not normative in some way. There's a lot of people projecting onto you. Right? That's the, that's how stereotype operates and how bias operates. A piece of research I was looking at talks about, let's get a little bit away from just the performance scores themselves and talk about what strategies boys and girls are allowed to use when they're problem solving in school. and I think this is a super exciting kind of newer area. this is something that points to some paradox as we still see with math achievement. So girls do a lot better than boys do academically. They're catching up or. Gender gap is closed, um, on math achievement in lower grades, but boys are still outpacing girls in higher grade math. And sometimes people point to this, as a reason to say, well, maybe boys just have this innate math gift. But again, we have some interesting social explanations. So girls are kind of punished for more risk taking, creative problem solving strategies. Boys are not so boys. Again, little Billy is gonna be able to kind of iterate and experiment and think outside the box a lot more and have that be seen as a sign of ability, whereas girls really, really learn to be teacher pleasers and being a teacher pleaser. Works out for you as a strategy for a while, but it kind of puts you in a narrow box. And you see this with women in the workplace too. You are constantly told to be a doormat or a people pleaser and you face big consequences if you're not. To be clear. I'm not blaming anybody for trying to survive that situation. Like this is very much like put on you by the environment, but it can help us understand how, like, you know, something that gives you a good grade in a certain context will start to make you struggle in another context because you never got the room to expand your entire portfolio of problem solving strategies.

Ashley: 43:08

Mm. Yeah. So our, our individual identity and our development of problem solving strategies doesn't develop. In isolation from all of society, all of the stereotypes that exist, what our teachers think of us, what our parents think of us, and all of these things actually shape whether or not someone might choose to go into computing or math or whatever it might be. So on your hike in Scotland. Did you lay out all of this evidence for, for this? Was this like, was this a, uh, a 30 minute, let me tell you everything I know.

Cat: 43:45

I mean, we were walking around for over an hour. I honestly think we talked about a lot of stuff, and I don't know how, I can't remember at this point in time how deep we went into it, but what I tried to get across was human ability is so diverse and it's actually so hard for us to predict what we are gonna need in life. When we look at this at, you know, we look at like the empirical evidence at scale, we look at cultures over time. What we need are highly effective, supportive human groups. Like no single individual is solving the world's problems alone. And so I hope, like what, what I succeeded at doing in this conversation was basically showing this guy that he already knew why he felt uncomfortable with these reductive framings and these kind of arguments that we're trying to get him to see all these other people as like less than him. And I think that, even if we didn't cover all the evidence we got to things like, you know, you have seen how hard it is to predict. Who's gonna be a good teammate, or you have seen that one time you went and did a test in school, and that test in school did not give us everything we might need to know about who you could be for the rest of your life. Even people who have a lot of biases and stereotypes that they've learned. I think they have more complicated versions of the world inside of them, you know, that we can try to pull out and try to say, I see that in you and I see how hard this is, and you're allowed to think about it. And there's actually a lot of science that backs you up.

Ashley: 45:24

Mm. And I feel like this intersects with some other. Misconceptions that people have about work, especially in computing or software development, which is that like it's all individual and there's like one person who is the superstar coder on the team or something. Right. And I think, you know, going back to your initial response to his question, asking him to think about what he did on the daily or how his job was carried out is a recognition of the fact that. He doesn't work in isolation and in fact, a lot of the skills that he needs at work are in community with the people around him. And so does it even make sense to ask about something like math ability in the context of, of any of this, right? It's so far distilled, it's abstract. It doesn't make sense in the real world.

Cat: 46:11

I, I would say maybe it's useful in some context, but man, is it not useful for most of our questions. Like software engineering culture has a tight grip on this individualistic model and that model has low predictive value for the things that we actually care about. So we feel this as this kind of torture, and I hear people. You know, try to throw out all these explanations for it. Like, okay, you know, um, my boss is bad. No one's as smart as me. You know, if we put the real engineers in charge, we would fix this. But I think it's really, really important to go to the heart of how deeply we are trying to associate worth with human ability and not acknowledge that all of us are gonna age. All of us are gonna have something happen in life where we get sick or our partner's sick, or something's happening. That changes the amount of resources we bring to the table, and that doesn't make us less worthy as a human. Sometimes it doesn't even have actually the relationship to problem solving you might think it has, you know, someone who suffers and goes through something incredibly terrible can find wisdom in it that becomes the solution to like climate change in some way. We didn't know, you know, like we, we are incredible beings I think who, who solve problems in incredible ways and we just can't predict all of it. And there's like this foundational theme, which is. What is most useful for us to do if what we want to do is build a workforce of problem solving people who are getting the most done in the best way, for the best number of people. What value have these models provided to us? And they have fallen very, very short. I think, you know, in providing that value, whereas integrating this rich, interesting, important evidence that actually. Explains a lot more of our own lives and the paradoxes we see in achievement, it's really useful. You know, it tells us what to change.

Ashley: 48:19

Hmm. Do you feel like you changed this guy's mind at the end of this conversation?

Cat: 48:26

That's such a good question. I wonder about it. I don't know.

Ashley: 48:31

Yeah. You haven't talked to him since?

Cat: 48:33

I haven't talked to him since. We were Facebook friends for a while, but I'm not on Facebook anymore. I will

Ashley: 48:38

Yeah. Hmm.

Cat: 48:41

We had this lovely moment and I, I felt really proud of myself afterwards because, know, I feel very overwhelmed when this topic comes up. I care so much about. Like these women in engineering who reach out to me and send me messages and say, you know, thank you for saying something about this evidence. Every time I post something or I say, I saw people use this paper, and I don't think the evidence is good. I hope that what I do is help give people permission to say that you are allowed. You don't just have to be a passive recipient of what you hear. You don't just have to be a passive recipient of like a cultural stereotype. You know, I think this guy said to me something like, I am a curious person. I want to learn. And that was why I said, okay, I'm here. You know, I'm here to talk about this. Even if it's actually quite scary to me in this moment where we're like isolated in Scotland. But I think as long as I can still feel that feeling. Than an, I can still be a psychologist in software because I do feel it's my responsibility to be honest about what I think and try to talk about these really hard things and try to interpret it for people. So, you know, I can at least tell you the effect it had on me.

Ashley: 50:08

Hmm.

Cat: 50:09

Because I went out, out of the bed and breakfast that we were staying at the next morning, and I was still thinking about this conversation. And I think you, you never know. You don't know what's gonna happen. You know, you tried to provide someone with a counter narrative. Then they go live their life and you're like, I wonder, I wonder if you're gonna work with someone and you're gonna treat them with more respect. Now I wonder if you're, you know, gonna turn on me. I wonder if you're gonna attack me. I don't know. But all I can do is say, I tried to show up and show you that I was a human. And I remember going out and you know, I'm a psychologist, so I went out and did like my little self-compassion, self-affirmation exercise. I sat on a rock alone and I looked out, you know, at the water and I just said I am here. I'm here. Like I deserve to be here.