What has it been like in your life and work to have authored a category-defining book?4:16 how the conversation has changed6:57 career arc7:06 theater!09:09 influences10:55 audience question: when you’re teaching on this, what activities resonate with your students16:36 “what the humanities and social sciences do is they give you a really great vocabulary for talking about the things you care about and for you know looking at them closely”17:36 algorithms offline?19:38 what is the Center for Critical Internet Inquiry at UCLA doing? (site: c2i2.ucla.edu)20:17 big announcement!29:07 the challenges for companies want to address the oppression in their own tech47:56 what makes you hopeful? (BEAUTIFUL answer)

all right hi everyone02:08i hope we’ve got some photo we do have02:10some folks already turning out02:12uh give me a little comment or thumbs up02:14and make sure we can hear the audio02:16since we have had some02:18recurring issues with audio02:22let’s get some some comments go in from02:25folks say hi02:26i want to hear who’s out there and make02:29sure we’re02:30we’re coming across so um we’ll get02:33rolling in just a moment here02:36as soon as oh we’re getting more folks02:39some signals starting to come in02:44glad to see folks turning up i know02:46you’re all excited02:47i’m excited too i’m about to introduce02:51the guest here in just a moment but um02:54yeah just make sure let me know you can02:56hear me02:57little thumbs up that thumbs up03:03ah thank you john frater thumbs up all03:05right yay03:07hi sebastian thank you for being on i’m03:09glad that you all are hey03:11melton all right everybody’s turning up03:13hey we got you guys are coming in from03:14linkedin live i’m so glad that um we03:16just got that set up today03:18so it’s exciting that got a new channel03:20added to the live stream03:22all right and mark bernhardt says he’s03:24tuning in from wisconsin all right we’ve03:26got we’re everywhere right now03:27chris mclean thanks for the thumbs up03:29we’re gonna go ahead and get started03:30then03:31i’m glad you guys can hear me i’m glad03:33that sounds coming through and03:35we’re good to go so03:39pull up my notes here03:43all right i know that you all are03:44excited because the moment the03:46announcement went out03:47about today’s guest there was03:50so much uh joy and so much excitement03:53and everybody was03:54retweeting and sharing it like crazy so03:56i i know you all are excited to hear03:58from03:59dr sophia umojan noble who is an04:01associate professor04:02at the university of california los04:04angeles ucla04:06as better known as in the department of04:08information studies where she serves as04:10the co-director of ucla center for04:13critical04:13internet inquiry she’s the author of a04:16best-selling book on racist and sexist04:18algorithmic bias in commercial search04:20engines04:20algorithms of oppression how search04:22engines reinforce04:24racism that’s from nyu press and we have04:27Safiya with us04:29right now hi sophia you’re on thank you04:31so much for being here04:33hi kate it’s so great to see you and i’m04:35so happy to be here04:37wow it’s such a thrill and it’s such a04:39fun thing that everybody got real04:41excited you know i think it’s just04:42everybody knows that these are important04:44conversations and then of course to have04:47people04:47you know who i i’m able to bring some04:49folks into the conversation uh04:52in these last few weeks who are leading04:54this conversation truly and you have04:55been leading this conversation04:57you came out with this book what is it a04:59couple years ago now right05:03what was it 2018 uh05:06yeah yeah but you know it has been05:10uh it was a work in progress for a long05:12time before that05:13yeah i know it must have been because05:15you even say in the book that you know05:17you were tracking these issues05:18for a few years uh and so it’s it’s one05:22of the first i think that comes to mind05:23for people when they think05:24about algorithmic bias so that’s really05:28cool what’s that how’s that been like05:29for you in your own life since you wrote05:31sort of the best-selling category05:33defining book on a topic like what’s05:35that done for you in your in your life05:37and your work05:38it’s really been uh shocking to be05:41honest05:42because when i was writing the05:45dissertation05:46which eventually a version of that05:48turned into this book05:50i there was very little agreement in the05:52world uh05:53or with me that technologies could be05:56racist and sexist i mean there were05:58people like06:00wendy chun who is you know hero of mine06:03she’s at06:03simon fraser university who were writing06:06about the implications of software06:08and um you know there were like a06:11handful of women of color06:13really lisa nakamura um06:17anna everett kind of talking about all06:19these different cultural dimensions of06:21technology06:22but when when it got to talking about06:25that the code06:26could in fact be implicated in06:28structuring06:29racism and sexism there was really very06:32little agreement that that was even06:33plausible and so it’s06:35it’s strange to go from uh06:38writing about it a decade ago in you06:41know in the face of no agreement and06:43really having a hard time06:44getting people to be in a conversation06:48with me about that to now06:51you know i mean i’m at the beauty shop06:54and i meet somebody and they ask me what06:56i work on and i say oh06:57you know i do i study and research07:00racist and sexist07:01algorithmic bias and discrimination and07:03they’re like oh yeah let me tell you07:04about this07:06[Applause]07:07okay show me about it so07:10in that way it’s really it’s joyous to07:13to know that07:15there are a lot of people who want to07:17have this conversation now and i’m07:19really07:20um grateful because there’s the07:22implications of07:23of what’s happening is so dangerous and07:26it’s so important07:27that many many many people need to be07:29thinking about it and talking about it07:31yeah i mean i’d argue everyone who07:33especially who works in technology but07:36beyond technology too because we’re all07:37affected by it07:39certainly yeah and so what have you seen07:41change in the industry since then since07:432018 your book comes out07:46you know literally the conversation’s07:48changing you helped change that07:49conversation07:50so now how do you feel like that07:54discourse07:54exists what’s changed about that well07:57this is the interesting part07:59that i think the conversation08:03has become more mainstream in the tech08:06sector let’s say and also maybe a little08:08bit more08:09in academia which is of course deeply08:11tied to the sector08:13uh but i’m noticing that08:16the real uh critique of08:19the implications of the work is getting08:22kind of defanged and de-politicized08:24and so how you see that now is that08:26people are talk instead of talking about08:29algorithmic discrimination or oppression08:31which are08:32what kinds of words i use people are08:34talking about things like bias08:36um and i think one of the things that08:39that does08:40is it really um08:43you know it neutralizes the power of the08:46critique08:47by kind of devolving it into a08:50set of arguments that you know everybody08:53is biased everything is biased08:56and that’s not helpful when we’re08:58talking about08:59the implications of like life or death09:01technologies09:02and also these things are structural09:05they’re not just09:07um you know living at the individual09:10level of how a coder09:12thinks right or how a programming team09:14thinks or09:15how an engineering team is oriented09:18and i think that is one of the things09:20that’s really changed in the last 1009:22years is that now09:24people talk about ai and ethics but they09:27want to talk about it09:29i think sometimes in a very thin09:31register and09:32so you know we have a lot of work to do09:35and this is like many09:36many kinds of movements in the world09:38where um09:40you know you’re trying to argue09:43about how health disparities09:47impact women differently and more09:49powerful how09:51women are more likely to be let’s say09:53you know um09:54victims of um cancer reproductive09:57cancers09:58and then next thing you know like a pink10:00ribbon is on10:01an iced tea bottle and you’re like10:03that’s that’s actually not what we’re10:05trying to talk about that’s all we’re10:07gonna do10:08that’s not it so i think that’s the10:11thing to be watching for right now10:13is the way in which these conversations10:16are increasingly being10:17de-politicized that’s such an important10:19insight and you so you had a journey10:21yourself getting to10:22this level of awareness right you came10:24from advertising10:26and to come into academia and to be so10:29fully entrenched and aware10:30of what’s going on within that space how10:33did you10:33make that transition how did you get10:35from where you know you were working in10:37that advertising world to where10:39you know you’re on the front lines of10:40making sure that people10:42recognize the weaponization and the the10:44oppression that’s happening there10:46yeah well you know when i was an10:48undergrad i studied10:50sociology and i was really into kind of10:52the social sciences and humanities i did10:55theater10:55i was like a theater nerd too and i knew10:58we were going to be good friends10:59[Laughter]11:02i’m like i you’re getting me to tell all11:04my secrets now11:07okay so i i understood the power of11:10things like art11:12and also statistics to stories11:16and i was um i went into corporate11:18america really11:19idealistic about the kind of change i11:22felt that people who were11:24activists and um really like coming out11:27of the fields of ethnic studies women’s11:29studies were the people who could really11:30make the difference in corporate america11:32and i was motivated and oriented that11:34way and11:35you know um yeah11:38corporate america can really dull your11:40knife you know if you11:42aren’t careful and so by the time i was11:45leaving the industry of course i was11:47um i had been on the internet since11:49probably i don’t know 88 or 8911:52and i was seeing all the changes that11:55were happening11:56inside the industry and when i went back11:59to school which is where i really felt12:00like i belonged and kind of the place12:02i’ve always felt at home12:06i just saw what a disconnect there is12:08between12:09how people are making in corporate12:11america12:12and how products and services come to12:14market and how12:16academia is sometimes lagging woefully12:19behind12:20um in in an assessment and so that’s12:23kind12:24of it was just like i don’t know two12:26worlds kind of colliding and i felt like12:28in academia though i would12:29have the space to interrogate12:33and maybe make a difference about some12:35of the things i was seeing in a way that12:36i12:36felt i couldn’t do when i was in my12:39corporate job12:40but it’s such an interesting background12:41to bring to it because you are able to12:43bring12:44uh such a truth-telling you know clarity12:47to to the work that you’re doing where12:49you can cut through and say12:51this is not just a search engine this is12:52not just a social platform it is an12:54advertising platform and that’s12:56fundamentally what’s what’s happening on12:58what’s underlying13:00you know the the complications of the13:02matter right and that seems like that’s13:03part of what makes13:04your work so so potent is that you have13:07that perspective13:08yeah well you know there were so many13:10people who influenced me when i was in13:13graduate school that i was reading i13:14mean i read this13:15book you know siva’s the googlization of13:18everything13:19really had a huge impact on me because i13:21felt like he was13:23writing what was happening what i was13:25witnessing and of course his you know13:27his new book um anti-social media you13:30know i i felt like there were people who13:32were writing i mean13:33frank pasquale there were a lot of men13:36quite frankly who were writing13:37um in ways that touched me and13:41yet i felt that there was this dimension13:43of like race and gender that was kind of13:45missing13:46or you know the way in which scholars13:48sometimes write13:49in a universal kind of paradigm about13:52you know what happens to society13:54but of course my own life living in13:57society you know i know that there are13:58many worlds that are very different many14:00realities that are different14:02i occupy different worlds and realities14:04from14:05from people who aren’t black and people14:07who aren’t women14:08and so that was the part i thought i14:11could contribute14:12um kind of in dialogue with all of these14:15other amazing brilliant people who14:17laid the groundwork but i think you know14:21uh again sometimes when women and people14:25of color when we’re pushing a boulder up14:27a mountain which is what it feels like14:30you know you’re not sure if that14:31boulder’s going to roll back on you and14:32crush you14:33or if you’re actually going to get it14:34over and kind of you know14:37get some like momentum and you know14:39velocity and so14:40i’m really grateful that i could kind of14:42get this conversation14:44up over the mountain yeah i think a lot14:46of us are grateful too14:48first of all uh nicole radziwill says14:49nice shout out to siva14:52so yay everybody’s appreciating that i14:54have a question from14:56dr siv brown who says when you’re14:57teaching on this topic what activities14:59resonate most with your students and15:01audience15:03well that’s such a great question okay15:06so15:07on a basic level around search i’ll15:09often have my students15:11do their own searches on google15:14and you know other large commercial15:19search engines and i’ll ask them to look15:22for identities that matter to them that15:24are kind of15:25like either their own identities or15:27identities they care about15:29um and it’s interesting to see the kinds15:31of searches they do so15:32i say you know go do these searches come15:34back next class session we’re gonna15:36discuss everybody’s gonna get a chance15:37to talk about what they found and what15:38it means to them15:40and you know for some people they you15:42know i will never forget what i was15:44teaching at the university of illinois15:46at urbana champaign15:47i had just like a disproportionately15:50high number15:52of white women in predominantly white15:55sororities15:56and i think almost all of them had done15:58a search on16:00sorority girl and were pissed16:04[Applause]16:05so where i don’t think like necessarily16:08that like searches on black girls and16:10latino girls and asian girls that would16:12surface porn16:14um at the in those years um16:17meant that much to them or like they16:19couldn’t think that they were16:19sympathetic but not empathetic when they16:22looked16:22for their own identity they were16:24disgusted16:25so you know often it’s just those kinds16:28of16:29uh experiences that are really helpful16:32to16:33help students feel the impact16:36of the the work that you’re talking16:38about but you know i also do things like16:40i teach my students a lot about museums16:43and libraries i’ll have them go look for16:46the same16:47uh people and communities that they care16:49about in the library16:50many of my students believe it or not16:52have never even walked the stacks16:54in a major research library um16:57so i said go do that and this is also a17:00place where they17:01see the subjectivity of knowledge17:04where they start to realize oh you know17:07i was looking17:08up like um my sexuality17:12and it was in all like in a17:15cluster of books about sexual deviance17:19and i’m not feeling that right yeah it’s17:22right17:22and so then they start to see that like17:25knowledge17:26is subjective and it’s political and17:28it’s meaningful17:30and this destabilizes their trust in17:33just like getting an answer in 0.0317:35seconds that’s so17:36great to create an experience for people17:39where they can truly dimensionalize and17:42create the empathy that they may not17:44have been able to connect with uh17:46otherwise and i feel like i it’s my17:49experience and17:49tell me if it’s yours that once you kind17:52of can once you can connect that empathy17:54it stays with you and you can you can17:56extend it in different ways in ways that17:58you may not have been able to before18:02i think so you know i i had a colleague18:05say to me once18:06you know if you can just like touch one18:08student in a class18:10that and inspire them to like maybe go18:12on to graduate school or18:14to like really care about the things18:15you’re teaching them about18:17that’s you know like you’ve done your18:18job and i and i18:20think more than one like i need to18:22return greater than one18:23but i could be meant by it um but i i18:27definitely see18:28that you know getting to know your18:30students18:31is a great way to then pull in resources18:35that are relevant to them so i always18:37try to know18:38all the majors for example that are18:40represented in my class18:42and i will adjust the reading schedule18:44right after the first day of class18:46to start to match up with things that18:49they care about so that they can see18:51like oh you you want to do cognitive18:54science18:56why don’t you take a look at these18:57things um that are always kind of18:59bringing19:00them back to a critical interrogation of19:03their own work and then the things they19:05care about but also of these systems19:08of control surveillance and power in our19:10society19:11because in many ways those things are19:13going to have a huge impact19:15on their work and on their own personal19:16lives and i i feel they have to leave19:19the university knowledgeable about that19:21that’s so smart and we’ve talked about19:23this a couple times on this show before19:24about how you can almost19:26matrix out all the different facets of19:28life and then put it alongside19:29technology and say19:31you need to know about how technology is19:33going to impact economics across19:35different sectors of different segments19:37of society you need to know how it’s19:38going to impact politics across19:40different segments of society so it’s19:42it’s brilliant to take people’s majors19:43and what they’ve already sort of19:45declared an interest in and say now19:47here’s how this is going to play out or19:49allow them to do the discovery about how19:51it’s going to play out in their field19:53that’s right i mean i always for example19:55with with their final papers in my class19:58ask them to take up the readings that20:01we’ve looked at and the kind of critical20:03thinking skills that i’ve taught them20:04about this20:06this domain and apply it to20:10something that’s in their field of study20:11or their expertise so that the computer20:13science students20:14and the engineering students get to20:16write about the things they care about20:19but interrogate them differently in a20:20way that i know they’re not doing in20:22their computer science courses20:26i know hopefully they will though i mean20:28i feel like that’s the change that20:29you know your work and and other folks20:32in the space that are creating that20:33momentum that20:35that kind of conversation will be part20:37of the technology20:38uh majors the the computer science20:41majors and all that20:42yeah well you know what the humanities20:44and social sciences do is they give you20:46a really20:47great vocabulary for talking about the20:50things20:51you care about and for you know looking20:54at them closely20:55having a close reading really being able20:57to articulate and20:58storytell what matters in21:02in the moment right or on the project21:04and i feel like21:06students and and people not just people21:09in the university but21:10all of us need to be able to better21:12storytell21:13why things work and why things don’t21:17work21:17so that we can influence and persuade21:19each other21:20to to move in directions that are21:23that are better you know better for for21:26everyone21:27just for a small sector and um that’s21:30hard work21:31but i um you know i i feel really lucky21:35to get to try to21:37do that kind of work yeah and so do i i21:40think that’s21:40that’s the most powerful work you know21:43one thing i wanted to ask you though it21:44seems like21:45when you wrote the book and you know in21:47in the time that you were doing the21:48research on it21:49it seems like we were mostly21:51concentrating on algorithms online and21:53you know kind of the21:54the experience of being at a browser or21:57whatever and typing in a search and21:59getting the query back22:00of course now we know that all our22:02experiences are so much more hybridized22:05so i wonder you know are you doing new22:07research that looks at that inner22:08integrated space and how22:10how you know that bias and the22:12oppression that comes into those22:14through those algorithms it’s affecting22:15people in their everyday lives22:18yes and i i really appreciate you22:20bringing that up you know22:21when i was writing algorithms of22:23oppression it was at the same time cathy22:25o’neil was writing her book offensive22:27mass destruction22:28and i met her at this convening that22:31meredith22:32whitaker um and kate crawford did22:35that was in partnership with the white22:37house this was during the obama22:39administration22:40and you know she was talking about um22:43how algorithms and ai are so deeply22:46embedded in like banking and finance and22:48all kinds of you know predictive22:50technologies and i was like22:52i’m writing about that too and so you22:54know we both of us our books were about22:57to come out i think hers came out22:58a few months before mine and um wow the23:02landscape has changed so much i mean now23:03there are so many more people23:05writing brilliant books um23:08that just are amazing you know i think23:11of rujob benjamin’s you know23:13new book race after technology that is23:16so23:16beautifully accessible and um and and23:19people23:20are studying virginia eubanks you know23:22the word this work on23:24social welf welfare systems so yeah23:27there’s no shortage23:28of predictive analytics that are working23:31across every sector of our society23:33and um from the time that algorithms of23:37repression came out23:38you know until now uh that the speed23:43by which that has happened has been uh23:46um remarkable um23:50astounding so i’m thinking about um i23:53think people are doing really good23:55you know jobs with that uh at the center23:57for critical internet inquiry at ucla24:00you know i’m i’ve been looking at things24:02like24:03how the tech sector um24:06undermines democracy by doing things24:08like not paying taxes24:10by you know pulling the cream of the24:12crop of the best students in the country24:14into their projects how it abuses24:16workers um24:18you know just the uh how it unleashes24:20products24:21on society with no oversight24:24by anyone um other than24:27you know themselves and and what the24:30implications of that will24:32be long term so those are kind of the24:34things that i’m i’m working on and24:36writing about right now24:37so you just had a big announcement today24:39relating to your work there24:41yeah so we did so we are really grateful24:45um24:46we’ve been working with julia powell’s24:47dr julia pals and bianca wiley24:50who are um at the mindaroo foundation24:53who have um established a global network24:58of critical scholars who are kind of25:00taking on big tech or the implications25:02of big tech let’s say25:04and uh we’re so at ucla we are going to25:07be one of the nodes in the network and25:08we25:09were just um gifted 2.9 million dollars25:13to over five years to stand up this25:16initiative25:17and one of the things we’ll be doing is25:19working on policy25:21around these kinds of things that i’m25:22talking about but also25:24culture culture making counterculture25:26making activities because we feel like25:28you know the sector has really um25:32created a culture that fetishizes itself25:35right and we’d like to introduce25:39you know other ways and other kind of25:41cultural takes hot takes on what on what25:44the implications of these25:45projects are so um i hope that people25:49who are interested in this will like go25:51to our site at c2i2.ucla.edu25:54subscribe to our mailing list follow us25:57on twitter25:58and just um be in conversation with us26:00because we really want to26:03do culture jamming and and policy26:06jamming kinds of work and i think it’s26:08going to be a very experimental26:11and exciting time to to be working on on26:14these things26:15for sure and congratulations by the way26:17we have uh dr26:18again pops up to say awesome26:21congratulations26:22in all cats so uh thank you for sharing26:25your congratulations dr sid26:26uh yeah i noticed that your agenda as26:30stated on26:30on the site is so tackle lawlessness26:33right26:34empower workers and reimagine tech and26:37those are26:37three powerful declarations of what you26:40plan to focus on yeah i mean my26:44focus in all of that is i’m thinking26:47about things like26:48the lawlessness of tech uh in26:51again kind of tax evasion um26:55you know undermining democracy26:59not just in the united states but in27:00many modern democracies around the world27:03what does the sector owe back to the27:06public27:07in terms of repair and restoration27:11and so i’ll be working on things like27:13restoration27:14and reparations and imagining27:18um and you know again i will put it like27:20in this paradigm27:21um and kind of where i’m writing right27:23now um there was a time when27:26people couldn’t imagine the american27:28economy without big cotton27:30and the labor relations of enslaved27:32africans and27:34of um occupation of indigenous lands27:37like that was the model27:38and it was um no one could imagine27:41beyond it but there was a small group of27:43people who were abolitionists and27:45i certainly probably would characterize27:48myself as a27:50i fancy myself a tech abolitionist and27:52an abolitionist27:54i mean i love it and that you know27:56there’s just a small group of people who27:58are always saying like this28:00the morality of this isn’t right the28:03immorality28:04we have to reimagine the american28:08economy28:08and you know i i’ve been um laughing you28:11know i’ve been telling the story28:13that i’m sure my when my mom was28:15delivering me28:16in the hospital in fresno where i grew28:18up where i was born28:20um had like a cigarette hanging from his28:22mouth you know i mean there’s like28:23no question that um you know28:27the 17’s were just all about that and um28:30in the 80s and28:32and then we had a paradigm shift about28:34big tobacco28:36and the people who had been doing the28:38the activist work and the researchers28:40who had been saying28:41big tobacco is a public health creating28:43a public health crisis28:44we can’t afford this it’s too extractive28:48it’s it we’re paying too high a price28:50they shifted the paradigm and you know28:52my students now they can’t imagine28:54that people were like chain smoking in28:57the hospitals28:58right you know in all kinds of places so29:01i think29:02about big tech you know in in that model29:05that29:05um how could we look at other historical29:08moments and take a longer view29:10on this era that we’re in and maybe29:12reimagine29:13something uh far less harmful and maybe29:16even29:17helpful wonderful we have a question29:19from the audience29:20she says laura laura says i’m curious if29:24you think that predictive analytics29:26can be used ethically emphasis on can29:28our company has shied away from using29:30them but it can make it challenging to29:31compete with companies that do use them29:33heavily29:35this is the challenge with predictive29:37analytics okay of all cat29:38all technologies and systems that rely29:42upon classification29:43and categorization systems and this to29:45me i mean this is where29:47my library science nerd is really29:50uh has me kind of anchored to29:54the fact that all forms of29:56classification and categorization29:59um have implications so the question30:03is what are the implications of the30:05kinds of classification systems that30:07you’re using30:08in order to make your predictive30:10analytics most of the ways in which the30:12technology is oriented30:14is really around like binary30:16classification systems30:18um you know binary code and that it30:22already um is a problem30:25it’s certainly a problem around gender30:27it’s a problem around30:28um uh race um the30:31the things that these kinds of30:33classification systems do to reify30:36you know power imbalances um and30:39exploitation30:40are very important so i think the30:42question is is it possible30:44to make classification systems that are30:47not30:47um harmful and that is actually30:52probably a more important you know30:54question we have to ask before we can30:55get to the deployment of the predictive30:57analytic30:58yeah it really is an interesting uh31:00question there because so i’ve said for31:01years that my favorite book title is31:03george laykov’s31:04women fire and dangerous things what31:07categories reveal about the mind31:09and that taxonomies are not neutral like31:10there’s nothing neutral about any kind31:12of categorization or classification you31:14can ever do you’re imposing some sort of31:16opinion31:17judgment whatever into the31:19categorization31:20so yeah it’s an important point you make31:23that’s really31:23it and you’re also creating social31:26structure31:27through those categories and what we31:30know is that those categories31:32have always existed at least in the31:34western context31:35as hierarchical so if your31:38categorization system31:39puts you know it has you have a racial31:42classification system like we have the31:43united states and many other parts of31:45the world31:46where white is the highest valued31:49and most um resourced and most powerful31:53and black is the antithesis of that and31:55the binary and31:56everything in between is vying for its31:59relationship32:00to power or powerlessness32:03um that those systems become real32:06so the question is um you know how do we32:10create systems that aren’t hierarchical32:13and where power is not distributed32:16along those lines of classification or32:18categorization32:19and we have not solved that um instead32:22we are reinforcing those systems of32:24power over and over and over again32:26right right so i know a lot of companies32:29want to try to do something32:31about this within their own systems or32:33at least there’s32:34lip service given to wanting to try to32:36do something about32:37the bias and the the oppression that32:39happens within their systems but32:41i i saw that in 2019 according to the32:43artificial intelligence index report32:46uh put out by the stanford university’s32:49human centered ai institute32:50it said only 19 of large companies32:53surveyed said their organizations are32:55taking steps32:56to mitigate risks associated with the32:57explainability of their algorithms32:59and 13 are mitigating risks to equity33:02and fairness such as algorithmic bias33:04and discrimination so clearly33:06even if there’s a lot of lip service to33:08it there isn’t a lot of action and i33:10wonder if you33:11have um concrete steps or33:14recommendations that you’re able to33:16offer and33:16academia doesn’t typically offer you33:19know concrete steps into33:20corporations but i wonder if you have33:23recommendations for33:24for companies that want to you know do33:26some mitigation and make sure that33:28they’re33:28they’re taking steps yeah i it’s33:32it’s difficult because the algorithms33:34and the ai and the predictive analytics33:36that are coming out of33:37industry are optimized for profit so33:41that’s one of the challenges here is33:43that33:44let’s say capitalism and33:48multiracial democracy might be at odds33:51with one another33:52um so we have to figure out the degree33:55to which33:56industry feels um it must be responsible33:59for in which it’s shaping our societies34:02um and we haven’t done so well there34:08so i think you know if companies are34:10serious about this34:11then they also have to be serious about34:13the role that their companies are34:15playing34:15in the world um you know should it be34:19profit at all costs34:20um when is enough enough um uh you know34:24when is being profitable enough34:26enough and i think that of course there34:29are many great scholars34:30out here i think for example of a a34:34group of scholars uh that i work with34:36out of nyu34:38um called the center for um critical34:41race and digital studies34:43and um you can find us34:46with a kind of simple query on nyu’s34:50uh center for critical race and i’ll34:51send this out and digital studies34:53we have a lot of readings there um34:56where you can get educated people who34:59work in industry35:00and tech leaders can get deeply educated35:03about the implications of their work35:05um at c2i2 on our resource page we also35:08have kind of 15 plus books35:10at the intersection of race and35:11technology that we think people in the35:13industry35:14a variety of industries should be35:15reading i know they’re in35:17you know since the week of june 8th and35:19um you know in this35:20moment where we’re calling for um35:23justice for george floyd and brianna35:25taylor35:26and organizing uh in this new civil35:29rights you know extended let’s say35:31civil rights movement um of black lives35:34matter35:35that many tech companies in particular35:37but also other companies financial tech35:39and others35:40are reaching out to scholars and35:43bringing us in and asking us to educate35:45them35:46so i think you know it takes that and it35:48also takes a real35:49you know a longer term concerted effort35:52to figure out35:53um you know the ethical35:56um framework uh that a company is35:59is working in and um those are36:02complicated36:02long term conversations yeah that makes36:05me it makes me think36:06makes me wonder how much of the36:08underlying issues and the reluctance to36:10mitigate36:10those issues have to do with either36:13incentives36:14or the transparency that it requires or36:16the equity that isn’t shared36:18at the table as opposed to say the36:20technical difficulty36:22that it would take to go through and36:23sort of clean up that system yeah i36:26think it’s both36:27i mean i think the fact that we have no36:29meaningful regulatory framework right36:31now in the united states is a huge36:33issue because we know for example that36:36most companies36:37that have historically been implicated36:39in um36:41uh discrimination all the way to the36:44technical level which i would say would36:46be by36:47banking finance insurance right as like36:50right out36:50right off the bat um those companies36:54those industries didn’t really36:55start to um address redlining um in36:58their products and services37:00until it was against the law so we know37:03we have to have more than just kind of37:05the fox guarding the hen house we need37:08regulators to get serious about the37:10discriminatory effects37:12of many of these technologies and that37:13will be one way that companies will be37:16forced to kind of comply with the law37:19i mean facebook you know itself never it37:22you know we know that like they hadn’t37:24even considered things like eeoc37:26or civil rights act um or housing37:28discrimination37:29law in their own products and services37:32and it wasn’t until37:33um you know congress got serious about37:34calling to them to the carpet or the37:36federal trade commission37:37got serious about that that they um37:40started to37:41um examine their products at a technical37:43level37:44and of course at a technical level the37:46challenge now is that for the big tech37:48companies37:49they um they don’t know how to fix37:52some of these problems um they they37:56think or maybe let’s say they um profess37:59that they will solve these things with38:01ai that they will kind of38:02automate the fixes but we know that in38:05fact their automation38:06are human beings and this is where the38:08work of people like my colleague and38:10collaborator sarah roberts and her work38:12on38:12you know helping us understand this38:14these armies of content moderators for38:16example38:17around the world but it’s human beings38:19who are implementing these decisions38:22and um the real policy decisions are38:24getting made by the lowest paid most38:27vulnerable workers who touch content38:29that might be discriminatory38:31these cannot be fit necessarily at a38:33technical level38:34and um this is where i think you know38:38we’ll have to figure out are these38:40companies in fact just38:41too big um to fix38:44their the problems of their own making38:47yeah38:48that’s a huge conversation we got into38:49it uh the last few episodes too about38:52the human moderators and the work that38:53casey newton and other38:55uh investigative journalists have done38:57in exposing some of the conditions that38:58some of the facebook moderators work in39:00and39:01for example but it seems like i mean it39:03is going to be a real thorny39:05thing to try to figure out how to create39:08the right kinds of regulations or what39:09what is39:10what how much of it is going to be about39:12breaking up pieces of the companies how39:14much of it is going to be about39:15creating the right incentives or39:17disincentives how much is going to be39:18about requiring the right kinds of39:21uh transparency in the algorithms and39:24the ai39:25uh and and and more i’m sure39:28more than what i’m i’m thinking of yeah39:30i mean the challenge here39:32is that really in the in the39:35social media space in the search space39:38um39:38you know we’re talking about uh and in39:41the hardware i would say39:42space you know we’re talking about39:44monopolies39:45and um so the very um39:49you know ground upon which39:52internet-based companies39:53came to the fore was in the wake of39:56breaking up39:57big um telecom monopolies right40:00and so i think it’s interesting now that40:03um40:03you know that in internet-based um40:06companies are40:07in fact monopolies and this means that40:12consumers have very little choice it’s40:14very difficult40:15to have harms addressed or redressed40:19because we don’t have40:20a legislative apparatus that is40:24literate enough quite frankly to even40:26understand what these technologies are40:28and what their harms are and i think40:30that you know that could change40:32um so you know we have a lot of work to40:35do40:36uh to not only help the public40:38understand what’s at stake but40:40you know when when social media40:43companies and and big tech companies40:45like google40:46say you know youtube you know say that40:48they’re not media companies40:49and try to skirt responsibility for the40:51content that moves through their40:52platforms you know where40:54you know we’re in dangerous territory40:55because um40:57they indeed are responsible for the41:01content that moves through their41:02platforms and um41:04much of their content i mean one of the41:05most dangerous things we can see right41:07now41:07is the flood of disinformation that’s41:10moving through41:11these platforms as we uh uh rapidly41:15uh you know pummel toward the41:17presidential election41:19and all the down ballot elections and41:22um uh a lot is at stake41:26if we aren’t serious about looking at41:28this sector41:29yeah and so i’ve been uh advising41:33groups in other countries as well as41:35they kind of hurtle toward regulations i41:37know brazil has been41:39uh dancing around different kinds of41:41regulations and they’re having a lot of41:42the same issues around misinformation41:44and coming up on41:45on various political elections that are41:47are critical to41:48to those countries so uh he comment from41:52bruce celery says crazy to think that41:53yesterday’s disruptors are today’s41:55monopolies but that’s exactly right41:57it was a great point sophia42:00yes that is the situation and you know42:03it’s uh42:04when i think about brazil and the united42:06states and other places42:07around the world you know the uk um and42:10the disruption to kind of the way42:13democracy works and42:14in all of our countries um42:18you know people thought that when the42:20cambridge analytica42:22um i mean i wouldn’t even call it a42:25scandal because it’s just like the42:27the business operations of cambridge42:30analytica42:31when that came to the fore people were42:33you know like stunned42:35and i thought i remember first reading42:37about cambridge analytica before it42:39became a big story42:40and i remember kind of watching them and42:44i was like the whole internet is42:46cambridge analytica42:49internet is brokering and selling42:52and making data profiles about us and42:54micro targeting42:56us and making digital profiles about us42:59that we’ll never know about43:00we’ll never be able to see that we can’t43:02intervene upon and that are43:04um making again opportunities and43:07foreclosing others43:09and this is where you know shashana43:11zuboff’s you know book the age of43:12surveillance capitalism43:14is so um important you know i always43:16thought if my that my book might be a43:17book that you shouldn’t read at night43:19before you go to bed because it might43:20give you nightmares but then i read43:21shawna’s book and i was like well43:23well well wow that category43:28and you know again the thing that i43:30would add to43:31her critiques about how these predictive43:34technologies43:35are so um opaque and embedded in in43:39every move all the way down to like43:41you know our ovulation uh you know43:45tracking our ovulation tracking you know43:48every pimple43:48i don’t know tracking every question you43:50answer right or wrong in the learning43:52management system when you’re43:54nine years old in the fourth grade um43:56and and what those will43:58amount to is really really um44:00frightening but then when you overlay to44:02me44:03the racialized and gendered power44:06systems in our society and you know then44:08that this will have44:10far more severe consequence for people44:13who are already poor44:14who are already marginalized who are44:16already living um44:18you know under uh you know uh threats of44:21civil and human rights44:22just by virtue of who they are in the44:25world44:26um then i think you know to me there’s44:28nothing more interesting to talk about44:30and to to work on and um44:33and of course this includes working with44:36artists44:37and people like you and you know smart44:39people just to have these conversations44:41i mean these are the kinds of44:42conversations everybody should be able44:44to have44:45at the dinner table um because they44:47really affect all of us44:48uh you know imagine what it’s like for44:50your your four-year-old44:52whose life is being documented on the44:54web in every possible way44:56and you know that you’re unknowingly44:58creating digital profile for that child45:01you know imagine what it’s like that you45:04will inherit45:05the digital legacy of your family45:06members and that that will45:08affect your ability to get a mortgage or45:10go to college or45:11or do things that you want to do in the45:13world because somebody in your social45:14network45:15said something against the government45:17and of course we already see these45:18things happening45:20so it’s not uh the dystopian future45:22we’re just talking about right now45:25yeah yeah i think what’s really45:26important is that you have brought such45:28clear language to it because as you say45:30you know it’s the sort of you have the45:32opportunity to talk about things like45:35bias or you have the opportunity to talk45:36about them in terms that actually call45:38them out as45:39oppression and i what i fear is that45:42what it takes to really understand what45:44happens at that whole the whole internet45:46is cambridge analytica level45:49is that you have to be able to think45:50about meta systems upon meta systems and45:53understand not just how you know the45:56internet is connected and how45:57data collection and monetization systems46:00happen but46:01obviously of course underneath that the46:03the societal structures of systemic46:04inequities and everything that happens46:07at that meta system level and so to have46:09those kinds of conversations in46:11an articulate disciplined way46:14you need to be able to call them out as46:16the truths they are and i think you know46:18using words like oppression certainly46:20help bring that clarity46:22to to it yeah i really agree46:25that um all of these are interlocking46:29systems of oppression this is one of the46:32reasons why46:33i think people who are like46:35intersectional you know who use46:37intersectionality46:38as theory to understand or feminism or46:41black feminism or critical race46:43you know that’s what we study and write46:46about46:47our interlocking systems of oppression46:49and46:50it’s very important that we have um46:53clear vocabulary words so we all know46:55what we’re talking about46:58and of course our words are in dialogue47:01with other people’s words47:03like technology is liberatory47:07or that it engenders more freedom or47:09more connectivity47:11i mean it was astounding to watch mark47:13zuckerberg47:14in the anti-trust hearings um uh a47:17couple weeks ago47:18where while the senator you know that47:20this the members are saying you know47:23could you talk to us about the way in47:25which your products have collapsed47:26democracy47:28and you know the answer is like more47:29connectivity47:31i mean it’s like could you talk to us47:33about you know47:35um discrimination that happens on your47:37platform47:38and the ways in which um people have47:40been threatened with genocide47:42through your your project and he’s like47:44we’re connecting more people47:47so it’s like the discourse that’s coming47:49out of these spaces are really really47:51powerful47:52and amplified so much uh more powerfully47:55than you know what i’m saying so i think47:58we have to be really clear and get our48:00vocabulary words um straight48:02as we do this and it also seems like48:04because when you start48:06evaluating discussions around things48:08like how are we going to tackle the48:10issue of content moderation and you48:12talked about human content moderators48:14but then you think about48:16when there really are ai solutions which48:19you know48:19mostly there are not yet but but as48:22there start to be48:23uh more and more machine learning uh48:26solutions deployed against this48:28i worry about that too i worry about the48:30encoding of48:32you know biases there and what’s going48:34to happen when48:36uh you know using gans for example to to48:38try to simulate bad behavior as48:40just came out a few weeks ago that48:42facebook’s doing uh to try to create48:45some rules that they’re able to process48:47against48:48which in theory sounds great on one48:50level of abstraction and then when you48:52think about what are you really48:53codifying and what are you bringing into48:55scale48:56that’s what i worry about so yeah48:58there’s these systems upon systems49:00yeah yeah i mean you’re right to worry49:02about that because49:04at the level right now of a business49:08operation49:08used you know across many of these49:10platforms youtube49:12i mean all kind of many many different49:14major platforms49:17the human beings can’t get it right49:20so if you already have you know a49:24a corpus of decisions that have been49:26made49:27by moderators and those have not been49:30good decisions49:31well those are also the decisions or the49:33actions that that are informing your49:35machine learning model49:37so of course we know and you know this49:39one of the things i really try to49:40impress49:42just like make this point in the in my49:44book is49:46if you are a part of an oppressed class49:50let’s say you’re black in america but49:52you’re only 1349:54of the population you actually will49:56never be able to impact49:58the algorithm even if all 13 percent of50:00the population50:01of the black population was in agreement50:04which we are not50:05about adjudicating harm racist50:08propaganda disinformation this kind of50:10thing50:11still can’t even impact the broader50:16whole so even using these like so-called50:19democratic methods50:21are really um unfair um50:24so we have to be more complex uh about50:26what we’re talking about and i’ll tell50:28you50:28the thing that’s been interesting about50:30watching countries like germany and50:32france50:32as they’re working through regulating50:34things like hate speech or propaganda50:37and these big platforms is that they50:39have a much closer50:41relationship in history to the holocaust50:43and understanding for example the role50:45that50:47anti-semitic anti-gay um50:50anti-black propaganda played in50:54the relationship like the the the50:56legitimation50:58of cost right in the in the dulling of51:01the senses of the public51:03that to the point that when people would51:05stand on train platforms51:07going to work in in um51:10germany jewish people would be51:14being loaded onto train cars to be sent51:17to51:17concentration camps while other germans51:19were getting on train cars and going to51:21work51:22imagine that level of desensitization51:25that would have to happen and that51:27happens through51:29fascist propaganda racist anti-semitic51:32these kinds of violent subtle forms of51:36of discourse in a society and that’s51:39what’s happening51:40in these platforms and it’s being fully51:43normalized and of course it’s having a51:44tremendous impact51:46i was just gonna say it’s the51:46normalization of that speech and you51:48just hit it there at the end yeah51:50it’s it’s having that just be51:51surrounding you all the time and it’d be51:53normalized into how can anybody have the51:56stamina51:57to keep up with fighting against that51:59level of disinformation and52:00misinformation52:02yeah so it’s it’s all that’s all very52:06bleak and i know52:07we we’re working very hard to try to52:09keep the worst from happening but52:11what do you think that we have to be52:13optimistic about52:14what what do you look at in this space52:17and what makes you feel hopeful52:19well i feel so hopeful because i am52:24uh you know part of a community52:27of people in the united states who have52:31um you know gone from52:34uh you know lynching um public lynchings52:38um you know being shadow slavery52:42not being considered human to at least a52:45greater52:46possibility for our humanity to be52:48realized and so52:49that is always for me so in the52:52forefront of my own52:53lived experience and my family52:55experience and so52:56how can i not be hopeful because i feel52:59like53:00black people and black women in53:02particular have been at the forefront of53:05so many important movements for justice53:07in this world and i feel really53:09grateful i was born into this package53:11even though it’s53:12also been difficult and um and that53:15makes me feel i i feel hopeful that53:18there has been hope for hundreds of53:21years in this country53:23to make change and to realize change53:27and that’s the kind of legacy that i53:29feel like my work is53:30is in or tradition that my work is in53:33and so how can again i53:34i feel um grateful53:38and and um and um53:42open-hearted that the world53:46will be better that it can be better53:49that it has already become better53:51in some ways and i also know that these53:53struggles53:54are not um static it’s not like you win53:57your civil rights53:58and you keep them forever um you know54:01african-americans had tremendous54:04civil rights the the moment in our54:06history where we had the most54:08rights was the period following54:10reconstruction54:11after the civil war a lot of people54:13don’t even realize that54:14um and um then those rights were rolled54:17back through jim crow54:18um laws and um knight riders and the54:22clan and54:23and terrors terrorizing our communities54:25and burning down54:26our cities and our our banking54:28institutions and stealing our wealth and54:31we had another civil rights movement in54:32the 1960s54:34um and we’re undergoing another civil54:36rights movement right now after the54:38rollbacks of the civil rights54:39that’s it that have happened over quite54:41frankly the last four years most54:43intensively54:44so um you know we should never take our54:47rights for granted54:48and i think we should read and learn54:50from history and54:52that is a source of strength and power54:54for all of us54:55that’s beautiful i think it’s also such54:57an important point that you made earlier54:59that progress and freedom for55:02everyone is not going to happen as a55:04result of you know55:06majority rules like the the people who55:08need the power55:10because they need you know equity aren’t55:13going to necessarily be able to take55:14that55:15by by power it needs to be something55:17that everyone55:18kind of awakens to and that there’s uh55:22an evolution of we we55:25evolve to our higher selves that we we55:27think about uh you know our roles in55:29society and culture55:31in in that more evolved way and i55:34have to uh bring up a comment that just55:36got posted55:37never stop talking sophia never ever55:40please and thank you55:42yeah thank you it’s really hard and i’m55:45really grateful55:46that um i i’ll just say every people55:49don’t even realize55:50every little tweet at me and every nice55:53thing55:53is very encouraging because um you know55:57you never know people’s private55:58struggles and what they’re you know the56:00interior of their life is like and56:02i will tell you that there was a time56:04when i was so56:06painfully powerfully insecure to speak56:08my thoughts56:09and um it’s really been um56:12a journey and i’ll just say that anybody56:15who’s ever felt56:17small um should try to let that go56:21and just speak your heart because um56:24that’s that’s what we need in the world56:26and i’m really glad that56:28uh you know hugs by uh like a thousand56:31hugs and and and comments and56:33encouragements56:34really as is what has gotten me here56:37that’s wonderful well thank you so much56:39for that transparency and that56:41that uh that kind open-heartedness too56:44uh we’re getting such love from the56:47comments by the way uh dr siv says this56:50is such a necessary conversation can’t56:52we56:52wait to share with belmont students and56:54to that end she also asked earlier56:56uh can you recommend websites for56:58educators to use to teach about57:00algorithms of bias and oppression i use57:02the digital divide filter bubble and57:04facial recognition resources greatly57:06appreciated so57:06if you think of some right now that you57:08can share that’d be great if you just57:09want to send me some and i can include57:11that in show notes uh that would be57:12awesome too anything come to mind right57:14away57:15yeah so the two we have some syllabi um57:19at the um critical race and digital57:21studies um57:22website for edit for nyu and i think57:25that the url57:26is critical race and digitalstudies.com57:28and um57:30and there’s some great syllabi there and57:32of course people have been57:34um organizing um many syllabi through57:36twitter57:37um and so i would say uh we also have57:41resources up57:42on our resource page at c2i2.ucla.edu57:46um and you know we’ll keep populating57:49that57:49um in the weeks and months to come57:51because uh57:54it’s fun to teach students these things57:57it does seem like doom and gloom but it57:59really isn’t because with knowledge58:01comes power58:02and people do feel empowered when they58:04know more58:05i tell my students i just try to i want58:06you to be the most interesting person at58:08a cocktail party58:09i know you’re 18 and you’re not at58:11cocktail parties yet but give it a58:12minute58:13um you know you need to know things and58:15to be an interesting person and so58:17i i would say um those two places are58:20good places to look for58:21articles and books and videos and things58:23that we think um58:24are important yeah i feel like i talk58:26about this a lot too that58:27it’s not necessarily that being i talk58:30about being an optimist and it’s not58:31that58:32being an optimist means you don’t58:33acknowledge the bad things that happen58:36or can happen or are happening right58:38like i think it’s really important58:40to be fully realistic and conscious of58:43everything that’s happening but the work58:45of optimism is to recognize the good58:47that can happen and then steer our work58:50in that direction58:51and i feel like there is no one who does58:53that more clearly and with more eyes58:55wide open58:56uh goodness than you and i appreciate so58:59much you coming on and talking to us on59:01the show59:01thank you sophia i’m happy to be here59:04anytime59:05i really appreciate that this59:06conversation and59:08i’m going to have a great day i’m so59:10glad can you let our viewers know where59:12they can find more about your programs i59:14know you’ve given a few urls already but59:15just59:16where can they find you and your work59:18yeah you can find me59:19um i like to hang out on the internet uh59:22at sophia noble on twitter where i59:25always try to retweet59:27and send out good things that i think59:29are happening and if you catch me on a59:31late night tip i might be59:33being a little bit of a smart mouth so59:34just just leave it59:36and i’m on instagram safiya.noble.phd59:42and i try to post some things there too59:44and share out and59:46of course you can always email me center59:49staff59:50ucla edu and um59:54uh i think wait go to our twitter59:58because i could be lying about that59:59that’s a brand new60:01we’ll see c2it.ucla.edu60:04is the the centers contact us60:08and you’ll get right to me awesome thank60:10you again thank you so much60:11thanks to all of you for tuning in and60:13thanks to our listeners out there60:15on the podcast when that comes out60:18sophia60:19have a beautiful rest of the day we’re60:21going to disconnect thank you everyone60:23thanks