how Rumman’s background in political science shapes her thinking in AI3:28 “quantitative social science is math with context”3:58 “often when we talk about technologies like artificial intelligence… we’ve started to talk about the technology as if it supersedes the human”4:11 Rumman mentions her article “The pitfalls of a ‘retrofit human’ in AI systems”: https://venturebeat.com/2019/11/11/the-pitfalls-of-a-retrofit-human-in-ai-systems/4:56 What is the core human concept that shapes your work?5:25 “I recognize and want a world in which people make decisions that I disagree with, but they are making those decisions fully informed and fully capable.”5:49 A DOG ALMOST APPEARS!7:18 transparency and explainability in Responsible AI8:17 on the cake trend: “reality is already turned upside on its head — I want to be able to trust that the shoe is a shoe and not really a cake” 9:04 on the critiques of Responsible AI, “cancel culture,” and anthropomorphizing machines11:11 Responsible AI is not about having politically correct answers; her role leading Responsible AI is part of core business functions12:00 Responsible AI is about serving the customers, the people; credit lending discrimination example12:40 need for discussion that’s bigger than profitability and efficiency; humanity and human flourishing13:27 “human flourishing — creating something with positive impact — is not at odds with good business”15:21 “I think sometimes people can get overly focused on value as revenue generation; value comes from many, many different things”17:05 a political science view on human agency relative to machine outcomes19:22 AI governance20:34 “constructive dissent”21:13 the “human in the loop” problem25:14 algorithmic bias29:20 “building products with the future in mind”29:44 are there applications of AI that fill you with hope for the good they could potentially do?

how can we promote humanity and human flourishing with AI and emerging technologies?1:16 what can businesses do to enable Responsible AI1:22 “I have a paper out… where we interview people who work in Responsible AI and Ethical AI… on what companies can do” (see: https://arxiv.org/abs/2006.12358)6:22 what can the average human being do8:40 where can people find you?on Twitter: https://twitter.com/ruchowdhon the web: http://www.rummanchowdhury.com/

there we go uh hopefully you’re seeing both of us00:19now hi there it’s great to have you on Rumman yeah00:24thank you for having me on I am so happy to be talking00:27to you in the person-ish pandemic person00:29in person-ish that’s probably the best uh best term to describe conditions right00:36it’s uh it’s been a strange time I’m sure for you as well00:40yeah absolutely um although i must say it is nice to not continually00:45be on the road yeah yeah you know usually i’m away from home like 60 to 7000:49percent of the time so you know it’s nice to be not jet00:53lagged and in one location i do miss the travel myself so i have00:57also a pretty aggressive travel schedule for my work and uh it’s01:02it’s a little bit of a bummer not to have you know kind of a new country to01:06check out every couple of weeks that is very true01:09that is very true and by the way i will add that we’ll01:12probably get some sort of accompaniment from01:14my cat and/or my dog my dog is sitting right right down here and the cat’s kind01:18of behind the computer right now so i love that so my cat is in hiding01:23somewhere and she’s done really well so far at not01:26making an appearance in this show but I feel like one of these days she’s01:29going to demand her cameo so and my pets are attention hog so the01:35cat makes it a point to be vocal I I’ve just joined this um01:39this like Oxford commission we’re actually going to be announcing it01:42pretty soon and we’ve decided that she is our01:44unofficial mascot because she’s very vocal during all of our commission01:48at all so that’s perfect so what’s what’s really01:51cool about getting a chance for us to finally01:54sit down and talk is that you know as you mentioned you’re traveling a lot and01:58i’m traveling a lot and i know that we have been following02:00each other on Twitter for a while and it seems like02:03our our paths came crossing we’ll be like02:06in the same city uh either on the same day02:09or like yeah ships passing in the night and we just haven’t had a chance02:13to uh overlap enough to sit down so this is it this is our first chance to do02:17that and that’s really exciting um to me it’s02:20exciting because uh from the moment I I first kind of interacted with your02:25profiling and with you online I got the sense that first of all I love02:30how you put come across online but that your area of focus is so um02:35it’s so relatable to me you know this this intersection of course of AI and02:38humanity is very parallel to mine in tech and humanity02:41but also I noticed um that you have degrees in political science02:46and so I thought it’s your your phd’s in political science02:50even isn’t it yeah yeah so i to me that is incredibly intriguing because it’s02:56sort of I can relate to the idea not that you02:58know I have a background in political science mine is in languages but just03:01this idea that that that education and that framework03:05probably shapes you know your thinking and your mindset03:08about things right and the idea of systems and and the public03:13good and that sort of thing how does that how does that shape your work and03:16your thinking yeah um so the thing that really drew me03:19to political science so this was actually even03:21as an undergrad at MIT was the idea of essentially like distilled two it’s03:26basic it’s like quantitative social sciences math with context03:30and i really like massive context right or maybe another way to put it03:34would be uh i think it’s really fascinating to understand03:38at a high level uh patterns of human behavior using data03:42but the way I framed all of those sentences03:46well especially the second one you know centralizes the human it centralizes03:50society and what i find intriguing frustrating depending on what03:55you know my mood at the moment is that often when03:58we talk about technology like artificial intelligence04:01with technologies in general but especially with AI we’ve started to04:04kind of talk about the like the technology as if04:08it supersedes the human and this whole article i wrote called the04:12retrofit human where i raised that concern like why is04:16it we build technology and assume the human being fits in afterwards and we04:19really should be doing it the other way we need to be designing our tools04:23because these things are tools need to be designing our tools to help04:26us we shouldn’t be we like you know recreating how we04:30naturally are or what to be to fit someone’s notion of how society04:34ought to be so in your mind what is that kind of04:38core human concept because to me i i also mentioned um you know that that a04:43lot of the ideas felt parallel in our work and one of the things that04:46that i find i keep coming back to in my work04:49at the core of human experience feels like meaning and the04:52making meaning and the quest for meaning and so that’s one04:56theme that just over and over again i keep finding myself04:59returning to is there a similar concept for you that you find yourself returning05:03to in your work yeah and i think like it’s05:06very parallel unsurprisingly i would say it’s either05:09something like human self-determination or human agency05:12but ultimately it’s just the right to make an informed decision05:16the ability to uh you know have all the information for05:20yourself and make that choice and and i very carefully say that because i05:24recognize and want a world in which people make decisions that i disagree05:28with but you know but they are making those05:30decisions fully informed fully capable so to your point on05:34on meaning whether it’s you know being able to derive good05:37meaning from the systems we’ve created to make the decisions05:40or understanding what our meaning is or what our purpose is as05:45a human being and not having that be shaped or guided by05:48other forces unknowingly that’s my dog yeah is the dog gonna make a cameo is05:55that i mean i’m sure he wants to come here do05:58you want to say hi to everybody yeah he’s not tech canaanist show06:03that’s fantastic at the door so i apologize for the flying at the06:08door oh no the the pawing at the door just makes me06:12feel sad kind of get a little cameo actually the06:15thing is if i like open the door he’ll go out and he’ll father06:18come back06:21well i love how you put it and i think i think06:25you know the agency and the self-determination is is a really06:29solid piece of of what always kind of comes back to me too i’ve lately started06:33thinking about you know how we talk in in06:36literature and culture about the human condition06:39and exactly yeah and i feel like that when you break down what the elements06:43you know what we’re typically talking about when we talk about the human06:45condition it does seem like you know agency and you know sort of06:49control over your own destiny at some level06:52is is part of that right absolutely absolutely um and i think what’s really06:57great about it is you know it’s it’s not normative or judgmental like i07:01said i’m not trying to enforce my values on someone else my point is07:05that we should all make informed decisions yeah and we have transparency07:10into the systems that are maybe shaping us or07:13guiding us or giving us opening doors or closing others07:17so when it comes to ai then it seems like07:20the where that carries over is into the idea of07:24you know transparency or explainability and07:27is that what generally when you talk about responsible ai07:30as the scope of the work that you do is it generally focused on those07:34attributes or are there other attributes that are even maybe more07:38pertinent to that consideration yeah i mean so07:42certainly responsible ai covers those fields i think those schools are07:45incredibly important i think that you know of course any07:48conversation about responsibility would be remiss not to talk about07:51uh fairness and accountability um particularly when we think about07:55biases and biases and the technology that’s being built07:58and i know this has been kind of a contentious topic lately especially on08:02twitter that what isn’t different just topic on twitter right uh violence08:06is talking about worms has become a topic of contention if08:10you’ve seen don’t bring up cake that’s all we just08:13don’t need to talk about cake right now that’s a lot i mean look08:16like reality is already turned upside in its head i want to be able to trust that08:19that the shoe is a shoe and not really a cake08:23but you know what what i’m sad to see often08:27is that so much of the work on responsible ai08:30you know gets divided into camps of like this you know politically correct08:33culture and non-political or like whatever08:35whatever the opposite of politically right right um but that’s08:39not not what it is it’s not like normative judgment08:43passing uh at least not for me um you know if for me it is you know just08:50making sure that we are aware and have some control or agency and some08:54right to uh understand and have impact on the08:57systems that are you know shaping uh like the actions we’re able09:02to take in our lives yeah and i think you you bring up a09:04really good point because it does seem like09:06that issue of um sort of the critique of responsible ai or the the mechanisms of09:13responsible ai um that talks about political09:17correctness and you know we’re having such a moment09:19where people are uh you know hitting at this bogeyman of09:24cancel culture and and political correctness so so this uh09:28tweet from paul graham uh in the last couple days09:32that he says you know people get mad when ai’s do or say09:35politically incorrect things what if it’s hard to prevent them from09:37drawing such conclusions and the easiest way to fix this is to teach them to hide09:41what they think that seems a scary skill to09:43start teaching ais i imagine you have a response09:47for that i mean just like before i even get into what i think like09:52sort of let’s unpack all of the assumptions behind that statement09:55there’s just a lot of anthropomorphizing happening09:58like what is this like teaching the ai to hide like10:02these are not like these are technical systems right they are making10:05yes it is a predictive model it is quote making decisions but not from the sense10:08that human beings make decisions like teaching you don’t10:12really teach an algorithm to quote lie you can do10:16particular things to it to make it come up with some answers and10:19not come up with certain answers but if you are10:21hiding output or hiding outcomes that’s a human decision10:25from the design perspective so like let’s talk about the people who are10:28creating ultimately that’s the weird thing about10:30that statement this is weird and morphizing happening i just i simply10:34cannot understand like you know and this is not like sort of10:37the responsibility of community saying this10:39we have plenty of people you know who are some of the10:43the trailblazers in the field of artificial intelligence saying like we10:46are nowhere near the singularity we are not near10:48any sort of ai system and you know we will define it as like narrow ai if10:52you’re in the world of narrow ai so let’s10:54let’s let’s let’s box this into what it is today like we are nowhere near10:58creating this system that’s called lying or quote making decisions we’re in a11:02world of narrow ai we apply things to very narrow use cases so that’s11:06that’s one that’s hiding things it’s very odd to me11:09um and like it’s not about having politically correct11:13answers to be like i i work for accenture11:17um accenture is you know obviously that they were poor thinking and hiring11:20somebody to be responsible ai i don’t sit in corporate social11:24responsibility i don’t sit in corporate citizenship those are amazing parts of11:28extension parts of every company but i sit in core business functions if11:31accenture a half million person company thought that11:36responsible ai was creating politically correct answers11:41i don’t know if i you know i mean i’m not a ceo but like11:45that’d be a strange place to put somebody yeah that’s a really11:47interesting point right like i’m part of core business11:50functions my job is to create solutions with value11:53if you’re creating a product that doesn’t serve a portion of your11:56population you have not created a good product11:59so for example if you are making a credit lending model12:02that is discriminatory towards women because of the history of credit12:06discrimination against women this is not about put not about12:09politically correct culture do you just want to not give people money who would12:13pay you back like i don’t understand like do you not want to make revenue off12:17your product because you are you have literally an underserved market12:21so you are telling me that you don’t want to address an underserved market12:24like and and in some sense like in a business12:27sense that is what some of this work is about12:31it is about making good products that serve your12:34that serve your clients that serve your customers yeah yeah 10012:38and i just want to interject there i feel like i’ve seen interviews with you12:42where you talk about the need for there to be discussion12:45that’s bigger than profitability and efficiency12:48when it comes to you know business uses of technology we need to understand you12:51know what is um you know what’s about12:54humanity and her human flourishing so it’s really12:57important that those attributes be part of that discussion13:00too but you’re right at this core level of course business is going to be13:05investing you know primarily into technology that’s going capacity13:09and scale to their opportunities you know i think i13:12think that’s a savvy observation and you’re right like why13:15would there be a function for responsible ai13:18in the core business if it weren’t uh likely to produce13:23you know desirable outcomes for the business right exactly and also i’d say13:27like human flourishing you know creating13:30something with positive impact is not at odds with good business and13:34and frankly you know some this is what some of the13:37biggest ceo the ceos and the biggest companies in the world13:40recognize that you know some of what you build especially if you’re a b2c company13:44is about brand it’s about how people feel13:48when they interact with your technology or your product or13:51you know if you’re making like soda or a fast food chain13:54uh or clothing like you are trying to spark an emotion13:58uh frankly right people by like especially in the us we have no lack of14:02choices a lot of our goods are actually perfectly uh substitutable14:07why why do you buy coke versus pepsi why do you go to mcdonald’s versus burger14:11king i’m just like naming things right right like some of this is an emotional14:15decision um and so it’s again not necessarily14:19some sort of weird like lefty pc culture to say you know we want14:23to make things that make people feel good that are aligned with like14:26society’s values um and you know we’re getting some14:29pretty clear indicators of what a lot a lot of people feel today uh you14:34know branding is is definitely important for14:38companies yeah yeah that’s a really good point too14:40i think that that comes up a lot in my own work that that the uh you know14:44i talk about meaningful experiences and people are14:47always like well how do you measure meaningful experiences it’s like well14:49you know actually if you’re creating meaningful14:52experiences then you should have a whole host14:54of holistic measures that tell you that you’re on the right path14:58and everything you just talked about is all you know part of a model that15:01actually tells you you know you’re moving in the15:03right direction people can remember your brand people have delightful experiences15:07they’ll recommend you they’ll you know your cost15:09of acquisition and retention is going to be lower15:12because people have good experiences with your brand15:15all of those things right it’s also this notion of value15:19right i think sometimes people can get overly narrowly focused on value as15:23revenue generation value comes from many many different15:26things and to be perfectly frank you know people often choose less quote15:31efficient outcomes or you know less economically sound15:34outcomes because of how it makes them feel right uh you know and and i suppose15:39maybe a frivolous example but an extreme example of it would be15:42why people buy luxury brands you know like why would i buy a canvas bag15:46from like louis vuitton versus target canvas is basically canvas right like15:51louis vuitton doesn’t make better canvas but like they recognize15:54that how it makes you feel and the experience or to give a techie example15:58apple spends so much money on design they spend like16:01like there are entire articles on how every apple product16:05opening it is designed to feel like you’re opening a present like you’re16:08getting something special right that was purely intentional and if we’re16:12going to try to make this case that tech is about efficiency and value then you16:16know go talk to apple because they don’t seem16:18to believe that right they fully understand the16:21experience of an individual in interacting with technology like a phone16:24or a computer is also an emotional experience yeah16:29yeah so so in terms of of ai and and what the experiences we’re16:35going to be we are increasingly creating with16:38algorithms algorithmically optimized systems you know how can16:43people think about more meaningful and more human16:47flourishing kind of systems when it comes to those types of16:51interactions what what do you recommend there for people16:54yeah and here’s where i think it’s really interesting like as a political16:57scientist and the social sciences because i draw a lot from my background17:00when i think about these things you mentioned the concept of systems earlier17:03and this is absolutely true like these technologies don’t live in a bubble they17:06exist as part of an existing infrastructure of systems that impact us17:10so if we’re talking about for example a recommendation system17:14to decide if um you know to help judges decide if certain17:18prisoners should you know get bail or not17:21fail um what’s really interesting is not just how this impacts the prisoner17:25but also the role of the judge in sort of the structure of the judicial system17:30and whether or not they feel they can they need to be subject to the output of17:34this model or whether they have the agency to say i disagree with this17:38and i don’t and that impacts you know how this outcome plays17:41out for the individual who’s on trial right17:44so a judge is somebody who is a position of high social standing17:48you know they’re considered to be highly educated if there’s an algorithm and17:51it’s telling them something that they think is wrong17:54they may be in a better position to say i disagree i’m not going to do this17:57versus somebody who is let’s say um you know an employee18:01like a warehouse employee at like at amazon18:04or you know somebody who works in retail at a store where your job is not18:08necessarily considered to be high prestige18:10and you may feel like your job is replaceable or worse18:14you may get in trouble if you’re not agreeing with the output of this model18:17so like thinking about the system that surrounds these models it could18:21actually be kind of an identical structured model but because of the18:25individual’s place in society they can or cannot take action on it so18:28i think these things are really important18:30really important to think of yeah that’s a really important point i find18:34in talking with companies too about employee experience and about thinking18:39about how culture is going to be developed around digital18:42transformation and how they’re going to incorporate more and more automation18:45into their businesses so much i find of that discussion needs18:49to be about you know the increasing importance18:53of good judgment from humans you know like18:56people being able to make good judgment calls and being able to18:59say like this is asking me to do the wrong thing19:02and the machine doesn’t necessarily know that as you already said like there’s19:05not kind of hidden motives within the machine there there are19:09hidden motives within code because coders put them there but you know that19:14it’s not like um like humans shouldn’t be able to19:17question the output of these things so that’s a brilliant point19:21yeah and like two two points to that one is when i19:24talk to companies about governance um and ai governments has actually become19:29like one of the bigger things to think about rather than19:31just purely focusing on like monologue or modern explainability19:35so like a few thoughts on governance so again kind of drawing from my19:38backgrounds of political scientists i find it very interesting that all of19:42us even those in the responsibility community19:44are approaching this notion of governance from a non-democratic19:46perspective like what what every organization is19:49doing uh when we create systems of governance is put19:52the smartest people together and figure out what governance means for everybody19:56and it’s quite interesting because we all claim to adhere to very democratic19:59principles but very few organizations have actually20:02created a truly democratic process for government so that’s one20:05right uh the second very few organizations have created really flat20:08organizations too and even though they claim the two have done20:11so so yeah that’s a very good point yeah um and then the second is like so20:17i i we created um like this this handbook for companies called the20:21government’s guidebook it’s a publicly available document20:24i can share it with you if you have like show notes and yeah20:27put it in there uh one thing one thing that we call for is the notion of20:31constructive descent so how do you actually enable safe20:35channels of descent within your organization20:37how can people feel comfortable saying you know this is not working or this is20:41being done unethically or i disagree with what’s happening here20:44and not just in the way that they’re protected but also20:46in a way that they feel like their voices are being heard20:50i think one of the issues with you know with uh20:53people being at odds with the organizations that they’re with is not20:56just that they disagree with what they’re doing but they everybody has the21:00same story when i tried to go to management i was21:02shut down nobody listened to me it wasn’t meaningfully addressed and i21:06think that that’s a that’s a component of this21:08um that’s really important which kind of also ties into the third point that21:12we haven’t really solved this human in the loop problem everyone21:16loves to use that phrase but i you know it’s21:19really hard to think of difference a good situation21:22in in which we really resolved meaningful interaction between21:27you know a an advanced predictive technology and a human being21:32say more about that because i’m not sure that many of our listeners will be21:35uh as familiar with with that concept yeah so folks always talk about human in21:40the loop within an ai system so you know the the narrative would be okay21:44well we’re worried about runaway ai or ai that makes biased21:48decisions and then the answer seems to be we’ll21:50put a human at the end of it and then the human will kind of21:53judge the output and then the human has agency they can say yes or no and then21:57that’s that right but there’s so many problems with22:00this when you unpack that that story like it seems to22:03work on face but then we’ve already talked about a few issues so number one22:06like who is this person in this structure of you know the22:10hierarchy of humanity within their organization within society22:13and can they actually agree or disagree with the output22:17of the model are they in a position where they would be punished if they did22:20are they incentivized to do so and not do so et cetera and then the second22:24question is this person on the end can they even22:27understand whether or not that decision was a good one or a bad one because that22:31person may not and actually often is not a technical person22:34they’re not a data scientist so how are they to understand whether or22:38not this output makes sense or not um and just a really good example um and22:43a few months ago there was a whole apple card debacle um22:47when apple launched the credit card and we had the husband and wife and the22:51husband got approved and the wife did not even though i think she had a higher22:54credit score and made more money but here’s the part that i think to me22:57was the most meaningful around what we’re talking about so they23:01call you know apple or whoever and they ask23:05you know and again back to this notion of constructive23:07descent and human in the loop they asked like hey23:09you know my wife didn’t get approved for the card and we’re kind of wondering why23:13because you know like that’s weird and the answer was well the algorithm said23:19so and so that’s that’s that right and genuinely that is23:23not a good answer to give but to the person on the end23:26who’s a customer service rep right the question here then becomes23:29how do we enable a customer service representative to understand whether or23:32not this model output was problematic yeah like these are the people who23:37should understand it’s not me as a data scientist or23:39you know you as a technologist it’s actually the people who will be on the23:42receiving end who will be and who end up actually23:45being the front line with the human beings who are being impacted23:48so like that’s that’s the human in the loop that i think needs to be resolved23:51yeah and i think in in in a number of models business models23:56you know the the um the proposed answer tends to be23:59we’ll use a rating system to evaluate how reliable this person’s judgment or24:05outcome or whatever it is and of course then you end up with sort24:07of algorithms all the way down it’s like you know yeah yeah i mean and also in24:13this example like you know this is the this customer24:16service rep didn’t even get any visibility so they24:19they couldn’t they actually didn’t really know how to answer this person’s24:22question um and then even thinking through at a24:25higher level whether or not that model was biased like24:27i will say i haven’t followed the story all the way through but at first glance24:32i think captain also had a good article about this is24:35it’s not whether or not there are these one one-off cases in which things go24:39wrong because fundamentally all of these24:41systems are probabilistic not deterministic meaning like there is an24:44error rate and there things will go wrong24:46but that is just that is just a true truism that’s not even debatable24:50but what the problem would be is if this is systemic24:53it’s not just that this this one woman with24:56good you know who makes a good salary and has a high credit rating got denied25:00it would be if and obviously that should be fixed but25:03the system is a problem if we are seeing this across the board25:06across a number of women you know as compared to like a data25:10scientist have to do an analysis of this system to see if it’s a problem25:14and certainly i mean it’s easy to come up with examples25:17from across different parts of society and parts of technology where25:22you know this algorithmic algorithmic bias reflects25:25systemic bias and that we have those problems and25:29i think the the discourse on that is is raising but it seems like25:33we probably also need you know beyond discourse we need25:37other solutions where are you on regulations from for much of this like25:40where are you feeling like we stand on you know the maturity of that discussion25:44and where we need to be with with that yeah um it’s been really interesting to25:49see what different regulatories are regulatory bodies are coming up with all25:52around the world um so most likely europe will be ahead of25:57the pack on this um the european commission’s uh26:01the hleg has come up with a white paper that came out in april i think there’s a26:05follow-up to it that’s scheduled for december but who knows in26:08endemic times if they’re going to get everything done by then26:11which would be understandable if they didn’t the uk information commissioner’s26:15office also has a really great paper on risk-based approaches26:18to understanding ai systems singapore has launched this project called project26:23veritas which is getting financial services26:25financial service agencies together with their financial regulatory bodies26:29or thinking about it in the u.s we’ve had26:31uh the ftc federal reserve there’s been a lot of noise and there are also bills26:35on the table and what we’ve seen interestingly is26:37there’s been this bottom-up movement in the u.s so for example banning facial26:41recognition is such a great example you see it’s we saw it starting in26:45cities right before we see we saw anything26:47happening at the federal level there are algorithmic accountability26:50bills in like in multiple different cities and states and26:54again before we see it hitting at the federal level26:56so i think the us is going to be really interesting just again as a political27:00scientist yeah focused on american politics this27:03is why american politics is fascinating because of the way we’ve divided federal27:06and state powers and how that push pull like ends up being like sometimes a27:10contentious debate but ultimately like back to like my27:14first point it’s good to have people with different opinions27:16talking right that’s kind of what ends up being27:20and it also seems like it gives you know in theory at least it gives an27:24interesting model for being able to test different27:26approaches in different markets and see you know what are the consequences of27:30doing it this way versus that way uh and and then what’s going to happen27:33with that at scale but of course yeah of course that’s uh27:38it it supposes that that um that we can27:40actually anticipate that scale uh with just what happens at that city27:44level and often often that’s uh that’s going to be very different when27:48it’s applied federally right exactly yeah those are that’s such an27:52interesting area for you given your political science background27:55so do you find that you’re drawn more and more27:57into those not only governance discussions within28:01uh corporations but the governance at an actual28:05sort of political uh government level are you uh participating more and more28:08in those kinds of uh discussions yes absolutely um and i kind28:13of have been from why i wouldn’t say day well no almost from day one um and i28:19i don’t know whether it’s because it’s just my inclination to do so whether28:22it’s kind of a natural part of this job um because it does kind of combine both28:27i can’t just think about the technology i would be remiss not to think about28:31what sort of policy and regulation would be28:33would be coming down the road uh in part because you know28:36i want all these public servants to make informed decisions28:40um and you know it is difficult to wrap your head around the technology28:44when you’ve you know your experience has been something totally different and28:47it’s very difficult to get good information28:50um you know from from these different bodies and like all these28:53groups have you know people may have you know different uh incentives and28:57different reasons for sharing certain kinds of information not sharing29:00others but also i think you know when it comes down to29:03businesses everyone just wants to know what the regulatory29:07landscape will be and it’s useful to have that29:10information i mean not that i have any sort of insider information but just29:13to be aware of what’s happening so that businesses can make29:17good decisions um you know so they’re they’re kind of29:20building products with the future in mind yeah and29:24you know so sort of speaking of building products with the future in mind i29:28i guess i’m i’m curious about your own disposition and views like are there29:32particular applications of ai or just emerging29:35technologies in general that you get really really excited about that sort of29:39maybe even fill you with hope for what they29:42they’re for the good they could potentially do gosh29:45um i feel like lately like everything is very doom and gloomy29:49it is 20 20 so it’s like the world is on fire29:52um what what a great question honestly this is a very good question to ask29:56um i i what i think is amazing about this technology at the30:02meta level and what interested me in it uh in in technology is just how much30:06amazing potential it has for us to question our institutional30:10paradigms and and us to question why things are structured30:13the way they are and i think the the thing if i were to pick one30:17thing that got me the most interested in this technology30:19is actually the potential for it for edtech which is30:23funny because edtech has now become one of the biggest topics of conversation30:27and talking about all the negative of the surveillance30:29state right but you think about it like what’s the what it should be what30:34something like edtech should be is a complete reimagining of education30:37because number one like educational systems do not30:40actually help uh do not actually help people get jobs30:44they don’t help people do well at their jobs like everyone always jokes about30:48the number one skill you need to learn in colleges excel30:50because and that’s the one thing they don’t teach you right so it’s there is30:53this disconnect between the quote the real world the jobs we get30:57and then education educational systems how they should we31:00know there’s inequality we know that people in the us end up31:03with massive student loans you know there’s just so so much that31:06can be resolved with this technology whether it’s remote learning31:09or customized learning or you know like whatever it is and early on in the days31:16of when i started my job at accenture31:18before then people were talking about lifelong learning31:21and how you know the sort of new worlds of technology and ai31:25really means that we have to embrace you know learning and really think about how31:29we’re going to spend the rest of our lives educating us all of this31:32what what amazing aspirations yeah um right and i sincerely hope that what31:38we don’t do is just try to like stick technology31:41into the existing broken infrastructure that is our traditional education system31:46because that that would be a disservice not just to us as31:50humanity but also to the technology and the potential of technology so31:54but is it is it also true or not that once you use technology to sort of32:02accelerate or amplify a given system that where it breaks32:06might be what’s instructive about where those institutions are32:10already failing us like we won’t know those failings until we32:14try to amplify them at some level right i mean i understand32:16that there are real harms that are being caused32:19by doing that and the impacts are real but i’m i’m also wondering if32:23uh if it’s not um if we won’t get to the the level of32:29discussion about the failings of those systems until they’re actually being32:32amplified do you think there’s a way that we can32:35we can do that uh effectively um i mean i think32:39specifically using the education example there are so many people that have32:43already looked at the inefficiency of these systems and what does work and32:45what doesn’t work and you know what and if we really think32:49about this again by going back this notion of human self-determination32:52or you know whether it’s meaning or whatever we’re talking about like what32:56is the purpose of this system and you know32:59frankly can we just objectively take a step back33:01and in a sense almost emotionlessly ask is it serving the purpose it is intended33:07to serve right like you know like what is the meaning of our33:10educational system why is it doing this i think there are plenty of people who33:14have been pointing out the systemic clause33:17and i think usually the pushback is that oh it’s easy to criticize the system33:21like but but who’s going to be the one to solve the problem and really the33:24smart thing to then say is well now we have technologies and33:28systems that theoretically could be designed33:30to solve these problems instead of being designed to simply33:34reinforce the power imbalance and the structural inequalities33:37and we’re gonna ignore what these people say because it’s too messy33:41to deal with that and much easier to just perpetuate amplify and33:45now like cement uh all of these inequalities rather than do like the33:50extra amount of work it would take to like fix things yes no and that33:53that’s a brilliant way to address that that33:56mindset or that problem where do you think the the solutions best originate34:01are you finding in your experience do you find the34:04solutions origin originate with academics or34:09with private corporations or is it kind of a mix in34:13in what you’ve seen in terms of being able to identify the the sort of34:16structural flaws of institutions and what’s going to happen34:19when they’re brought to scale with technology um34:22i think it’s a bit of both um you know i i love34:25all of my academic friends because they you know they do such an34:28an insightful job of understanding systems and you know and again like34:32they’re sometimes able to look at it more objectively34:35because they’re not inside it um but then there is the aspect of34:39there’s the application component of it and that’s you know34:42what industry does so i’ll give you a great example34:45um so about what two years ago at this point a little over two years ago34:49um accenture came up with a fairness tool so we were the first to create34:52a enterprise level bias mitigation tool um34:56and the way we did it was we started off with academic research papers on you35:00know this is things like counter factual35:02fairness bias mitigation blah blah blah you can find all of these papers35:06but what was important to us is whether this works outside of a laboratory35:10setting i think we started off with like 30 some35:12odd papers and we only ended up with actually three35:16three of them that worked if we thought about does this scale35:19you know is this generalizable across multiple different settings35:23and is this possible within the way a data scientist does works that was35:27basically our criteria um so i think everybody has their role35:31to play it’s there’s some there’s definitely value in35:34pursuing research and even research that seems crazy and35:37weird but then there is certainly value to trying to35:40ground that research in something pragmatic and applicable like it’s it is35:45wonderful to live in a world of like all of these possibilities but then at35:49some point if you want to make this reality you have to ask yourself35:52will people use it how can i make it so that somebody will use it35:55and is this actually as beneficial as people are claiming it can be36:00and and does it matter do you think in the36:03the the context in which the technology starts you know we were talking a little36:08bit before we got on about this current story about that broke i36:12think today about facebook using a simulation uh36:16with ai to to simulate bots and other kind of bad user behavior so that they36:22knew better how to moderate against it um which i think you know i think you36:27had said at one level of abstraction seems like a36:30really good idea from like a data science model right36:33but from another level of looking at it you can easily see how36:36this may not be ideal to train to to begin to develop that sort of36:41training so so is there does it matter where the the kind of36:46origins of of a technology are or or do we do we always need to be36:50working toward you know these good outcomes and the36:52best of humanity sort of outcomes yeah um so two parts to it one i think36:57all of my sts and hci friends and i agree with37:00them would say uh the origin of technology absolutely37:04does matter like this is why so many people study the37:06history of technology you know things that are built for uh37:10military use even if it is moved into the commercial37:13space which is by the way a lot of technology37:15it will still hold with it the vestiges of let’s say surveillance or monitoring37:20because it is ultimately built assuming the world is a particular way in other37:24words there are good people and bad people there’s me then there are the37:27others right there’s me then there’s there’s people37:29i’m protecting the people i’m fighting because that’s just how the military is37:32structured right so so then it’s just fundamentally how your view of the world37:38will impact the technology that you build and i think that’s really37:41really important and and maybe even to abstract it even more and going back to37:45like all this conversation about political correctness culture and you37:48know designing an a that quote hides itself37:50i think what paul may be missing and some of you may be missing is that37:54um often you create technology with them often you do you create your ai run37:58optimization function like there’s a goal38:00to this and this has kind of been some of the critiques of the way38:04um like you know some of these research firms have been trying to arrive at uh38:10sentient ai is by having them play these games and they have them play combative38:15games right rather than have them play38:17collaborative right and again your objective function matters if my38:20objective function is to win a game where i have to kill38:24everybody to win or it’s a zero-sum world in which38:27i have to have the most amount of points to win38:29right um then that sets up a very different38:32system than one in which i’m training it to play a game38:36where we have to be collaborative and collectively succeed38:39like two totally different worlds but it’s all a function of your38:42of your objective function so going back to this facebook example38:46i think it is actually really cool to kind of basically like38:49simulating red teaming which is kind of awesome because rather than kind of38:52wait for bad things to happen they’re saying we’re going to have to38:55proactively model the world but the problem with it could be is that38:59you have to it’s not necessarily future adaptable39:02and if a new thing starts to happen that obviously cannot39:06be modeled within the existing system that you built39:10because your existing system is only based on the past and i think a really39:13good pragmatic example might literally be39:15something like gamergate right and a lot of folks and a lot39:18especially the women who are impacted by gamergate will say39:21you know we were yelling and screaming about how gamergate was really like the39:25canary in the coal mine about like this whole in cell culture39:29this whole like underground culture of like just you39:32know like a lot of the a lot of the issues that we talk about today39:36um people getting harassed and doxxed and you know39:39all of this was the player in the coal mine was gamergate39:43and people ignored it but then you think about if you’re trying to build a39:46predictable predictive system gamergate prior to gamergate would not39:50fit into your paradigm of the world because that had never really happened39:53like that before right so it’s a good idea if the world is going39:57to stay static if the world’s going to change39:59you actually need to have some balance to it that40:03understands how the world’s changing yeah and i think by the same token40:07it kind of goes back to what we were saying earlier about you know there’s40:10there’s a body of work already that has identified problems40:13like there’s the um the scholars that have already40:16identified problems with edtech and sort of the40:19systems of institutional education you know40:22that that knowledge already exists the the the scholarship already exists40:26so that it’s parallel here it feels like there’s been plenty of40:29um light being shown on some of the areas that need the most40:35work in terms of content moderation in terms of40:38uh making sure that you know uh bad actors are are banned and that can’t get40:43through on on all the social platforms but it40:46seems that twitter facebook you know and so on40:50don’t necessarily adopt those those recommendations and40:54instead it’s like facebook wants to play a game with itself40:57in order to come up with uh this this war game as you as you41:01so aptly described to be able to identify41:04what it probably could identify just by taking the recommendations of experts41:08who have been saying this kind of thing right41:12um yeah i mean like i said i think there is certainly value41:15in like from a data science perspective and trying to do what they’re doing at41:19scale like one of the issues of like any sort of moderation or41:22tracking is just the sheer volume right there’s just like i can’t even41:26create a number to imagine how many harassing situations or flagged41:31posts there must be on all of the social media so how do they like41:35parse through and it’s it’s it’s actually like again from a like a data41:38science perspective kind of a similar problem to thinking41:41about things like credit fraud which is at a massive massive scale so41:45the cool slash interesting i think the cool41:48part of the problem of addressing things like credit fraud is like yes there are41:51people trying to defraud your system but also there are people41:54who just like happened to go on a vacation in germany and like didn’t call41:58the credit card company and how do you do it in a way that42:00you’re not going to lose a customer because you’re annoying them with phone42:04calls or you’re freezing their credit right so it’s like42:06it’s not just like shut down everything that looks bad and it’s

and then we’ll just go ahead and sort of00:02close out with some broader themes00:04uh put put a little uh bow on the00:07discussion00:08um it’s such a bummer that we we lost00:10signal and lost connectivity while we00:12were talking00:12before because i think we really you00:14really were going through some some00:15interesting00:16uh thought there but i guess look maybe00:19let’s just go back to00:20you know the the discussion about00:22humanity and human flourishing like what00:24do we00:25what do you think uh what do you think00:27we can do00:28in culture and in technology to00:32stand a better chance of of bringing00:33about the best futures with00:35ai and algorithmic systems rather than00:38the worst futures00:39uh and and how do we how do we promote00:42humanity and human flourishing00:43in your view um wow yeah that’s00:47not it not a special question yeah so00:48it’s like it’s a little it’s a little00:50we’ll00:51we’ll figure it out in the last week i00:53mean you and i work around the concept00:55of humanity so we00:56have to think in big terms no absolutely00:59absolutely01:00um i mean there’s a few things just kind01:02of the immediate steps and then there’s01:04kind of the01:04the longer term like how do we view01:06things and and i guess like as a social01:08scientist i can’t help but think of it01:09as like01:10you know like atomized systems that01:12exist so one01:13i would say is first like what can01:16businesses01:16do to enable responsibility and that’s01:19pretty much the crux of01:20my job um so i have a paper out01:23with my research scientist donna rakova01:27as well as jinyoung yang from01:28partnership on ai and henrietta kramer01:30from spotify labs01:31where we um actually interviewed people01:34who work in01:34applied responsible ai or ethical ai um01:37so not in research people who are01:38working on business functions and we01:40actually got their01:41thoughts on what companies can do what01:43actions they can take01:44um you know what’s the president so01:46really it was couched it was01:48guided around like what’s the present01:49state what’s the prevalent state and01:51what’s your ideal future state and from01:53that we sort of drew out multiple levers01:55that companies01:56can use to enable so one it’s there’s01:59this balance of like external pressure02:01and internal pressure and that’s02:02something that’s actually worked in02:04um you know to to really drive change02:07and organizational change02:09i shouldn’t have the literature respond02:12is on the literature on organizational02:14change dynamics so what makes companies02:16culture shift right so there’s this02:19external pressure and external02:20validation and then there’s02:21the internal infrastructure um so one02:24it’s02:24an important to especially responsible02:26use of ai and technology02:28one is um having aligned success metrics02:31and like and that’s multiple things so02:32one02:33having metrics for things or ways of and02:35i say metrics very loosely i don’t just02:37need02:37like measurable quantifiable things and02:40you know qualitative metrics are just as02:42valuable02:43as quantitative metrics right i think02:45that’s a really important clarification02:47because people really do get hung up on02:48but i can’t02:49measure that specific thing and i can’t02:51see it on a dashboard and02:52yeah yeah exactly yeah or worse they02:55find some sort of like02:56you know insufficient metric and then02:59because human nature like we optimize03:01for numbers right03:02uh and that’s actually a pretty bad03:03thing too the analog i always give by03:05the way is like03:06when people are trying to be fit or lose03:07weight or be healthy there’s so many03:09quote metrics there’s like bmi there’s03:11weight there’s number of says all these03:13and03:13it doesn’t actually none of them are03:14actually good ultimately what matters is03:16like holistically how you feel03:18right and like that is a qualitative03:19metric that is very very valid right03:21and frankly more valid than how much you03:23weigh like a number on a scale03:24anyway so uh so when i say aligned03:27metrics for success03:28uh this is not just for products but03:31also for individuals like is it like03:33is it beneficial to my career at this03:36company03:37if i’m doing things like helping the03:38company create responsible03:40ai or is it going to look bad next year03:43in my performance review because i’ve03:45had x number of quote03:46failed projects right yeah and03:48interestingly like03:49there is a tech analog for this so you03:52know the lean startup eric reese’s book03:54is like the like03:55you know one of the core books of03:57anybody who’s starting a company03:59and in it he talks about um sort of like04:01innovation04:02metrics versus your traditional metrics04:04and this is quite similar like what04:05we’re talking about here really is truly04:07innovative04:08and this is part of innovation like we04:10are creating these technological systems04:12that are meant to actually improve04:14humanity in a very fundamental way so04:17like we actually do need to assess04:18people who work in these fields and04:20companies by quote innovation metrics04:22and not just these quarter reporter04:24and quote improvement metrics so there’s04:26that um another04:28is just this concept of tone from the04:29top and having you know your04:31leadership really say like this is04:33important to us as a company to support04:35us as an organization04:36i will honestly say that’s been a really04:38critical part of being for me to be04:40successful at accenture like you know by04:43my boss and our leadership has decided04:45that responsible ai sits in core04:47business functions04:48we have five core capabilities04:49responsible ai is one of them04:51that means something like that that04:53tells the entire organization04:55that you know they haven’t just hired me04:57to like talk on a stage and say nice04:58things04:59hired me to do real work and and that’s05:01very important and it helps me quite a05:03bit05:04um you know and some of it really is05:07just about creating05:08transparency around systems and05:10accountable systems so who’s responsible05:12for what05:13and to be fair if we’re trying to get05:14people to be on board with responsible05:17ai05:17we need to be very clear on what you can05:19and can’t do and what you will and will05:21not be responsible for so if05:23for example a lawyer is being told hey05:25you need to make sure these systems05:27don’t break the law05:28they’re like okay well i know what the05:29law is but i have no idea how these05:31systems work so maybe i don’t want to do05:34that because i don’t want to be left05:35holding the bag05:36if something bad happens right so05:38instead you have to very clearly define05:40as a lawyer it’s your job to enumerate05:42clearly to a data scientist05:44you know what the different aspects of05:45the laws are that make that may05:47come into play with this model and the05:49data scientist is responsible05:50for sharing with you the empirical05:52evidence like clearly defining those05:54responsibilities really help05:56um so that’s kind of one from the05:57organizational perspective and i guess05:59that’s maybe like06:00really specific but at this point i06:02think in responsible ai i love being06:04really specific because we have06:06everybody’s been talking into these high06:08level imperatives it was important to06:09have those imperatives yeah but i think06:11a lot of this pushback is coming from06:13you know certain people feeling like06:15we’re all kind of fluffy concepts and06:17not06:17real actions and we are absolutely real06:19actions um so that’s kind of the06:21corporate perspective i think you know06:22for the average human being06:24there’s a lot about education06:26understanding and sometimes it’s as06:28basic as understanding that there’s no06:29such thing as a free lunch06:30if there’s a technology that you’re06:32using an app you’re using on your06:34phone it is not actually free you’re06:36paying for it in data06:37you know you’re paying for it in some06:38way just like trust me you are06:40right whether it’s because you’re being06:42targeted media whether it’s because your06:43data is being taken and sold06:45like just understand there’s no such06:46thing as a free lunch right um06:48and like thinking about like what this06:50means and being mindful of the tech you06:52choose i think that these are the06:53actions that human beings06:54can take um and also there are06:56increasingly going to be06:58bills you can vote on so like everyone07:01should go vote07:03in general right but also make an07:05informed07:06vote and like look at whether there are07:08laws07:09in your municipality or in your city or07:11in your state around these things and07:13inform yourself on whether or not this07:15is going to give you more rights07:16over your information and whether you07:18want that and then vote accordingly so07:20uh that would be kind of my very high07:22level take that’s perfect07:24the time we had yeah no that’s perfect07:26and it sounds like you know07:27we’ve already talked about you know what07:28you sort of recommend to or what you07:30think07:31that uh government and sort of political07:33systems can do07:34and there’s there’s the the government07:37piece there’s the corporate piece07:39there’s the individual piece and that07:40individual piece i you know i come back07:42to this too it’s07:43it’s just saying again and again we have07:44to be very careful and very mindful07:47about you know how we’re participating07:49in different kinds of technology knowing07:50as you say07:51that we are paying at some level for07:54that participation07:55and and for that technology so super07:58super important07:59concepts uh i just want to make sure to08:02be08:02able to on screen thank you so much for08:05your time08:06for your flexibility and for uh putting08:08up with coming back on after08:10you know who knows why our signal08:12dropped it might have been one of the08:14pets in the background just saying08:15enough08:16of this it could be the cat08:19has had enough she’s trying to nap08:21[Laughter]08:23well i think this was wonderful and i08:25hope everybody got a lot out of it08:27um i want to thank you for being here08:29and hopefully we’ll even maybe come back08:30and do a part two08:32uh part three now at some point in the08:35future08:35so uh rahman thank you very much for08:37being here oh08:39before we go uh sorry where can people08:41find you online08:43oh yeah well you know my twitter it’s08:46ru chow r-u-c-h-o-w-d-h uh and on my08:49website which is just my name ramon08:51choudary.com08:52perfect all right thank you so much all09:01right