Wanda G.TX
Good information about recent case law and statutory law regarding hiring practices. This is a topic not frequently covered.
47min · about 4,800 wordsDownload Transcript
0:00Good afternoon or good morning wherever you are. My name is Diana Marie Laventour. I am a senior associate at Kauffman, Borges and Ryan where I practice in the area of employment defense management side. I also practice in the areas of bed bal and general liability. Today we are going to chat about artificial intelligence, algorithmic bias and employment law and how these issues have come out in the employment context. I'll also talk a little bit about how these issues have popped up in other areas of law and how employers can use these tools to ensure fairness in the workplace. Here is an overview of the objectives. So we'll talk about AI algorithms, legal implications and how bias may result from the use of these AI algorithms, how algorithmic bias impacts underrepresented groups, how current laws and statutes are attempting to mitigate these results and recommendations for employers who like to rely on these AI tools to recruit and retain employers employees.
1:27Excuse me, so what is AI? Artificial intelligence essentially replicates human capabilities. It uses algorithms which is a set of rules that can be followed based on calculations to problem solve and it does this by analyzing data and learning from the data over time so that it can help make informed decisions. You have likely encountered AI before if you use Google for example. Now there is the option if you put in a question like what are the elements of a fraud? And you in addition to getting a bunch of search results with links to maybe case law articles about what the elements are fraud, you'll get a bullet point summary of what the elements of fraud entail, other examples of artificial intelligence and where that technology may be used. If you use Siri or Alexa, if you have a smartwatch like I do, there are automated vehicles and trains such as the one that's on this PowerPoint and then there's AI technology used in health data analysis.
3:03Employers have relied on artificial technology to retain and recruit employees and makes the process much more efficient and it removes bias. For example, there are
3:21AI and employer applications that that actually do first rounds or conduct first rounds of interviews and in doing so relying on that type of technology may mitigate unconscious bias related to physical appearance, accessories, regional accents and tattoos. Assuming that the program is well designed, it can remove bias language from job description bias from this screening process and can also validate qualifications automatically from employees resume. You can also anonymize information about a candidate so reducing the possibility of bias if a human person reviews a resume. So for example, I went to Indiana University in Bloomington for law school and if I am reviewing resumes for a possible associate position and I See somebody that goes to iu, I may unconsciously have favorable ideas about this person simply because we have that in common. And by anonymizing certain information about a candidate, that kind of removes that possibility of from the screening process. AI programs that employers use also can standardize interviews to make sure that the playing field is even.
5:09Let's talk about what algorithmic bias is. It is a set of systematic and repeatable errors in a computer system that creates unfair outcomes. So the algorithm is based on data input by humans or data sets. And the data sets may lack diverse representations or the data set may be skewed. It can also occur from the engineer of the algorithmic input. The engineer holds certain beliefs or preconceptions. Those personal biases can be transmitted in the software. Then there is systematic bias. This results from procedures and practices of particular institutions to operate in ways in which social, certain social groups may be advantaged or disadvantaged.
6:05These don't necessarily result from conscious discriminatory tense, but rather from existing rules or norms. Then there is statistical and computational bias which occurs when the sample size does not represent the population as a whole. And then of course there's human bias which occurs when humans make decisions that may impact outcomes.
6:32So what, what are the current legal implications of algorithmic bias? And you know, AI technology is not necessarily new, but because of how it has been rapidly advancing and how employers and business organizations have been relying on this technology in their business practices. Lawsuits and issues have come up in different areas of law.
7:07One area of law in which AI issues have come up is in the healthcare insurance industry. There have been lawsuits filed against United Humana and Cigna based on the carriers use of AI to make insurance claims decisions. So specifically there were some investigative journalists that had done an investigations regarding these health plans use of algorithms to either deny or curtail post acute care. And essentially these insurance companies were relying on algorithms to deny care and were only relying on the algorithms to deny care. They weren't really having humans review the algorithmic results. And so these this investigations led to congressional committee probes and as a result some class action lawsuits were initiated against these people. Excuse me, these organizations you have lock in vs United States United Health Group. This was a district court case in Minnesota and Barrows versus Humana, that is a federal western district court case in Kentucky. In both cases the insurers pressure case managers to adhere to quote length of stay recommendations from an AI algorithm called NH Predict. And these were despite objections from clinicians and families that patients could not be discharged safely.
9:13Plaintiffs also Alleged that the insurers denied them access to their NIH predict reports saying that the information was proprietary. And the insurer ultimately ended up reversing more than 90% of the denials.
9:31Another case, Christine Lue vs. Cigna Corp. They sued is another class action against Cigna in the Eastern District of California over its use of AI algorithm. Allegedly to batch thousands of claim denials at an average rate of 1.2 seconds each. Which is essentially the allegation is that Cigna was rapidly and automatically denying thousands of claims.
10:09Another area of law in which artificial intelligence suits have popped up is premises liability.
10:19The Federal Trade Commission sued Rite Aid Corporation because of Rite Aid's use of facial recognition techniques technology.
10:30Essentially, the allegation is that Rite Aid had produced a watch list of customers that they thought or that did commit theft. And they took the customers images. And use cameras and facial recognition technology to alert their employees if the customers were in the store. Problem is, as per the allegations in the lawsuit,
11:10the alerts and the facial recognition technology generated false positive that subjected the customers to surveillance and banning them from their stores. So essentially,
11:26the Rite Aids had cameras. The facial technology would recognize, allegedly recognize some of the customers. And then the employees would be alerted to the customer's presence in the stores. And they would allegedly harass the customers, yell at them, accuse them of criminal activity. And this occurred in public, in front of others, the people. And in the presence of the people that these persons were with.
12:03And the allegation is essentially that. The other allegation is that this practice impacted and caused substantial injury, more so to black, Asian and Latino and women consumers. And this is likely because, and I will talk about this later on in the presentation. Facial recognition technology actually does a really poor job of recognizing facial features of black and brown people. And this is because of the data set that most algorithms are based on.
12:53Another area of law that artificial intelligence issues have popped up is civil rights. These two cases that I'm about to speak about are also facial recognition issues, technology issues. This particular case is district court case in the Eastern District of Michigan. Still pending. All of these cases are still pending.
13:25This concerns an erroneous recognition of a pregnant woman. This woman who did have a prior arrest history. She was at home in the winter with her, her family. She was cooking dinner. Police knocked at her door and said that they had an arrest warrant for her for a recent carjacking and robbery. Mind you, she was eight months pregnant. And this picture here on the slide is a picture of her pregnant. The reason why the Police officers picked her up is because she had been identified in a lineup. These pictures were taken from the actual complaint.
14:18If you could see this where my mouse is hovering over the picture, there's this red kind of arrow pointing down. That is her. And again, she had a prior arrest.
14:30The police officers got information from an employee at the gas station that right after the robbery, a woman returned to or woman went to the gas station and said that a phone had been dropped in the gas station. The police officers pulled the surveillance footage from the gas station to get the image of the woman who returned the phone. Then they used that image from the gas station surveillance and input it into their database of pictures of people that they had in their database. And these are people who have been arrested before. That picture produced or matched a picture of Ms. Woodruff, and that is why she was put in the lineup.
15:30So the victim was asked to identify the, the, you know, the perpetrator in the lineup, and she ends up picking Ms. Woodruff. Her attorney raised section 1983 violations, malicious prosecutions and intentional affliction of emotional Distress claims. So this is another facial recognition technology case where it's alleged that the,
16:02the recognition was erroneous. Again, this is the city of, this is the city of Detroit and the specifically the Detroit Police Department. It's another Eastern District Michigan case. Mr. Williams was arrested in January of 2020 for theft of a watch that occurred in October 2018. The Detroit Police Department's facial recognition technology identified him as a possible suspect based on a grainy video of the theft. A security guard who was present at the location of the theft, but who wasn't present when the crime occurred, watched the video and picked Mr. Williams out from the lineup. So I just want to make sure that you heard that he was not present when the crime occurred, but he, he was still asked to identify the. The person who committed this act.
17:04Crazy. So he picked Mr. Williams from Alima and based on that, Mr. Williams was arrested. Mr. Williams alleged alibi is that he was driving home from work at the time of the theft.
17:24And then of course, which is why part of the reason why we're here is to talk about specifically how artificial intelligence impacts or has impacted the employment law arena. EEOC versus iTutor Group is a federal case in the Eastern District of New York.
17:50The EEOC filed this suit because allegedly Itutor Group had a artificial intelligence employment application program that had automatically rejected applicants that were female applicants that were 55 years and older and male applicants that were 60 years and older. This particular Company provided online tutoring from remote locations. Its headquarters, I believe, are in Shanghai. And so overall, the allegation was that Itutor Group rejected more than 200 qualified applicants based on their age. And as I'm sure you know, or maybe you don't, that is a protected characteristic. The company ended up settling the case for $365,000 and actually ceased its operations in the United States.
19:04Mobley versus Workday. This is a Northern California federal district case. It's another example of an employer relying on a kind of automated application process. And it allegedly excluded applicants based on their race and other protected characteristics. And specifically, Mr. Mobley had alleged that he had applied for more than 100 positions on platforms such as ones that we're all familiar with. Monster. Indeed. LinkedIn. And when he applied to these jobs, he was directed to Workday, which is a organization that automates the employment application process for various employers of your employer.
20:09And you kind of want to make your application process more efficient. You might hire Workday to do that for you. So, as per the complaint, Workday would then prompt Mobley to create an account and either upload a resume or enter his information manually. And some physicians that he applied to required him to take a bandit assessment or personality test. After that, he received an email that he was rejected from the position. And then, notably, some of these rejections came within hours of him submitting his application. One came in just 55 minutes.
20:54And the issue before the court is whether Workday would be directly liable under Title VII and other federal civil rights laws. So let's do a little, little hypothetical, and if you want to participate, please, are you. Let's see. Are you able to participate in the chat? I don't know. Let's see.
21:30So I can't see if you are able to participate in the chat, but if not, just think about it to yourself and ask yourself these questions. So this was. This was presented to the court, and it was decided already.
21:48The question is, should Workday, who is not a direct employee, essentially, it is a company that is providing a service to the employer who is seeking to hide employees. So can it be held liable under Title VII for discrimination? So these were the plaintiff's theories of liability. That Workday is an employment agency, that it is an agent, or that it is an indirect employer. What do you think? Which theory do you think prevailed with the court?
22:30I'll give you a few seconds to think about it, and you may know how you may be familiar with this case already.
22:41So the court was persuaded by plaintiff's agency theory that the birthday is essentially an agent and that is why it should be held liable under Title vii. Another question for you. Should the employer or employers who contracted workday or similar vendors also be held responsible? And so what do we think about that? We have this vendor that may create applications based on requirements and information that's inputted or that's provided to them by the employer. And essentially they are screening employees and making judgments depending on employees responses to written in interview questions, submitted information, possible informational interviews.
23:55Based on the way the court rules on this issue, it seems likely that this is the direction that courts are heading, that vendors who are involved in the employment decision process are going to be held liable or of course will at least permit these cases to proceed in litigation. Other question is if we if, if vendors who are contracted by employers to hire employees, should plaintiffs have to prove that the employer knew that the tool was biased and still use it? Or is disparate impact in generating biased applications enough?
24:52Both are permissible with employers as it is. And if courts are allowing lawsuits to proceed against vendors, I can see either theory prevailing. Whether that's to prove that the employer knew that this tool was biased. Although it is likely easier to prove, it may be easier to prove disparate impact in generating the bias applications. And then the last question before I move on. Is there a level of due diligence that employers need to engage in to verify that the hiring tools are are not biased? And I think this is a pretty obvious answer yes, because you risk, you risk being sued if you don't know the extent to which your your vendors are complying with federal and state laws regarding discrimination.
26:04Right.
26:07If you're relying on these vendors to make employment decisions, you have to ensure that these at least at a basic level that their policies are the policies and procedures are complying with discrimination law.
26:41So algorithmic bias and its impact on unrepresented people. This is not something that is new, although it's something that seems to be more prevalent and hot topic in the news lately. But I have an example from of algorithmic bias from 1982 case where St. George's Medical School in Britain. They had an assessment software that excluded people who were who had quote, foreign sounding names.
27:27I read something in the news recently. There is a tenant association from Brooklyn and a building there. They filed a complaint in New York State about their landlord wishing to install facial recognition cameras to enter the building. Some tenants were upset because they felt that the move reflected pressures of gentrification and desire to attract more higher income residents and some residents simply did not want to be surveilled. Other Some Other AI Issues Recent Cases There's a current Southern District of New York lawsuit Borzinkova vs T Mobile USA. According to the complaint, T Mobile use the collection of biometric data from its consumers for its own procurenary benefit and without their permission. And this is in violation of specific law in New York, the New York Biometric act, which states that no private entity in possession of a biometric identifier may sell, lease or trade or profit from a person's biometric identifier information.
28:54Let's go back to how algorithmic bias impacts unrepresented groups, specifically in the employment context. So in the employment context if there's insufficient data that is inputted into these systems, these recruitment systems and may exclude underrepresented groups. So excellent example of this is an Amazon case. Not a case, but it is essentially a case study. So back in 2014, Amazon created an in house tool to retain engineers and other employees. And it was essentially automated recruiting tool. And the the data that they built the tool on was based on existing engineer resumes at the time and in technology at that time. And it's probably still the case today.
30:11It was their employees were engineering employees were mostly male. And so the resumes that the recruiting tool was getting its data from were male resumes. And about a year into using the program, Amazon realized that it was not rating its candidates for the technical positions in a gender neutral way. So the system had taught itself that male candidates were preferable and it penalized resumes that included words for example Women's, women's chess club captain. Basically anything on a resume that referred to women it downgraded as being a profitable candidate. Amazon realized this was a problem and it edited the programs to make them more gender neutral. Instead, the program ended up favoring candidates who use verbs that were more commonly found on male engineers resumes such as executed and captured.
31:28The program also ended up recommending candidates for jobs who were not appropriately qualified for the jobs. So Amazon ultimately ended up shutting down the project.
31:48Other issues in the employment context, but a little background first, because we've been talking about this, especially with those two civil rights cases against the City of Detroit facial recognition system. So there are employment vendors that will use facial recognition systems during the hiring process. For example, with interviews and personality assessment tests, the programs may give a certain score depending on a person's facial reactions. But as I mentioned before, now I'm going to go into detail about it facial recognition systems have higher error rates with non white people. Systems like IBM and Microsoft and Amazon were found to have higher rates error rates in classifying darker skin and female faces resulting from algorithmic design based on primarily lighter skin and male data sets. And so this information is based on a project by two MIT PhD students. It's called the 2018 Gender Shades Project. So one of the researchers is a dark skinned woman and she used, she basically performed tests and she used IBM, Microsoft and Amazon's facial recognition applications.
33:29And with one system,
33:34when she stood in front of the camera, it did not recognize her face as a human. But when she took a white mask, kind of like the Phantom of the Opera mask, but a full mask, when she took the white mask and put it on her face, then the system recognized her as a human. According to the study, the error rate for light skinned men is 0.8% and the error rate for darker skinned women is 34.7%. So she continued to study and she asked some of her lighter skinned friends to participate in the technology had a better performance at recognizing human faces when the person was lighter skinned. And part of the reason for this is because the information that's inputted is based on what's out there in media. There is a much higher instance of white men and lighter skinned people, people that are in the media than darker skinned people. And that's part of the reason why these facial tech, facial recognition systems are problematic.
35:04So how do we address these algorithmic bias and technology issues in the employment context and in general. So there has to be some element of human involvement when employment decisions are being made or even, or healthcare decisions as was discussed in those Cigna Humana and United Healthcare cases. You can't solely rely on the technology to make these important decisions. It's not fair that it's not, it's not proper. And it's not fair to make decisions about denying a person's claim for chemo treatment, for example, just because the AI program recommended it. Same with employment decisions.
36:02Employers may be missing out on highly qualified candidates because the AI vendor misrecognize a facial gesture or movement that a candidate made in a automated interview process. Another example, not in the employment context, but I'm sure you all heard of. There was a case of that attorney who relied on AI recommendation when he wrote his brief and the AI technology created fake cases. The judge's law clerk spotted it and called the attorney out. We you obviously have to cross your T's and dot your I's. I think Artificial intelligence can be extremely helpful and can help you save time and it's good as a reference, but it's still not in its infancy, but it still has flaws and you still have to make sure that the proper decision is being made, even though it could greatly help you in achieving you Making progress. The Best Decision if you're an employer and you want to use artificial intelligence, you want to use a vendor to help you select the best candidate or to help you designate a diverse team, you should be checking what kind of policies and procedures that this vendor has.
37:52What kind of goals can the vendor, the AI technology, help you achieve? Specifically in the realm of diversity, there are AI vendors who claim to have objective tools to help employers reach diversity goals. So for example, Zopa is a AI Power DI objective hiring platform that helps enhance the recruitment process
38:32by, for one, concealing personally identifiable information to eliminate unconscious bias and allow candidates to be reviewed solely on their skills, experience and potential. Tao Vista is another AI driven platform to help employers mitigate bias and it does this through optimized job description. It'll review a job employer's job description and try to highlight words that will detract underrepresented candidates from applying. They have a blind resume process, also reviewing PII and other information that may lead to bias, and they provide structured interviews as well so that objective outcomes can result.
39:35Other tips for employers in terms of relying on AI vendors to make employment decisions, you have to if you decide to use such a program, you have to monitor and audit it to make sure that it's not being misused and to avoid discriminatory outcomes. Remember, you know the the goal is you want to retain and want to attract and retain the best possible candidate, the most talented candidate. But you also don't want to break any laws and you want to avoid lawsuits. So you have to monitor and audit it. Also, there are current laws that require organizations and employers to monitor and audit these AI programs that these employers rely on.
40:36So one here is the one of those laws is the 2019 Algorithmic Accountability act, which allows the Federal Trade Commission to regulate companies enacting obligations to assess algorithmic training and accuracy and data privacy privacy there in New Jersey and New York laws here primarily because I practice in New Jersey New York, so I apologize if you're not from New Jersey New York, but these are still good examples. The state that you're from may also have similar laws. I'll start with the New York Local Law 144 of 2021. This is the New York Bias Audit Law which regulates the the use of automatic employment decision tools, or also known as AEDTs.
41:35Some of the requirements that this law asks employers to do who use ADTs, they have to have an independent auditor check the technology for bias annually. They have to post a result of. They have to post the results of the audit on their website. They have to notify job seekers and employees about the use of the tool and provide information about the technology.
42:06And there are other requirements. I'm just giving you some of them.
42:11If the employers don't comply for a first violation, there is a civil penalty of $500. Each subsequent violation can result in a penalty between 500 and $1500. The law aims to address algorithmic bias and provide oversight and transparency over the use of these AEDTs. New Jersey has a bill. Excuse me, that is pending. It is very similar to the New York law that I just showed spoke about.
42:47And it requires employers to also conduct an independent audit if they use AEDTs. The audit has to include historical data of the employers in terms of the employees that they have. They also have to publicly post about the tool and let candidates and applicants knows that it uses the tool. The New York Biometric act, which I spoke about earlier,
43:21discusses how basically companies are not allowed to use biometric data for profit. And then there's EEOC guidance about software use of software, algorithmic bias and artificial intelligence used in employment selection Procedures under Title vii. It's called Select Assessing Adverse Impact in Software Algorithms and Artificial Intelligence Using Employment Selection Procedures under Title VII of the Civil Rights act of 1964. So these are all things that we should be thinking about and taking into consideration when we're relying on AADTs, automated employment decision tools, any type of artificial intelligence in making employment decisions. And it's especially important that we stay on top of, on top of these laws and to ensure that we're complying with what's necessary to avoid discriminatory outcomes and of course to avoid lawsuits.
44:52You know, if we're. You're representing an employer, you want to ensure that you are doing your best to avoid unfavorable outcomes. Excuse me. Okay, so an AI governance. This is a set of policies, processes and best practices that ensures AI systems are developed, deployed and used in a safe and responsible matter. And this is essentially what all of these laws are trying to do. So if you're an employer, you're a representing employer, you want to make sure that you are complying with your local laws. So if you're not in New York and New Jersey you should definitely look up what SIMBA laws are requiring employers to do work or form AI or AVT audits. And also you need to ensure that you are complying with EEOC requirements as well. The other prospect, the other issue is, in addition to avoiding lawsuits and ensuring that you're complying with laws, is that you want to make sure that your potential candidates are not intimidated or do not feel excluded by by the process. You want to ensure a fair process for everyone. So with that, okay, I would like to thank everybody for joining here today.
46:40I hope that you learned something new about AI algorithmic bias and employment cases. Again, make sure to check EEOC guidance and local laws regarding requirements for automatic employment decision tools and AI vendors
47:05if you have any questions. Otherwise, here is my contact information. Feel free to reach out if you have any questions about this this topic. Thank you everybody. Have a good day.
Wanda G.TX
Good information about recent case law and statutory law regarding hiring practices. This is a topic not frequently covered.
Timothy D.NJ
This presentation was well organized, well presented and provided a good bit of information (statutory, case law, and non-litigation corporate actions) - probably as surprising to me, as my positive review is to you!