Amazon has created a robot for recruiting. He turned out to be sexist

The experimental software that Amazon had developed for recruiting systematically weeded out female candidates, so the company abandoned it. It is reported by Reuters, citing anonymous sources. Amazon tried to retrain its robot, but failed and disbanded the team that worked on it early last year.

Details

• The team started developing software in 2014. The ultimate goal was to mechanize the process of reviewing resumes and finding the best candidates.
• The experimental technology rated candidates and awarded them between one and five stars, much like shoppers rate products on Amazon, Reuters sources say. “Everyone wanted this holy grail,” one of them said. “They literally wanted it to be a machine where you upload 100 resumes and it spits out five and we hire them.”
• But by 2015, the company realized that its new system was not evaluating candidates for software engineering and other technical positions in a gender-neutral way. This is because Amazon's computer models have been trained to screen candidates based on the company's 10 years of experience in screening. The catch was that most of the candidates of previous years were male, which was a perfect reflection of the male dominance of the tech industry. Because of this, the Amazon system has developed an algorithm for itself that prefers male candidates. She also automatically rejected resumes that included the word "female" at all, such as "captain of the women's chess club."
• Amazon tried to correct this feature of the program, but the changes made were not a guarantee that the algorithm would not come up with a new way of discrimination, one of the interlocutors of the agency admits. According to him, Amazon was desperate to finalize the algorithm and eventually disbanded the development team. The company itself declined to comment.
• Amazon's experience could be an important lesson for a growing list of big companies, including Hilton and Goldman Sachs, that are looking to automate the hiring process. According to a 2017 survey by recruiting firm CareerBuilder, about 55% of US hiring managers said artificial intelligence will become part of their work routine over the next five years. However, the experience of Amazon and experts interviewed by Reuters prove that this is still very far away.

Origins of internet machismo

• The Amazon experiment began at a defining moment for the world's largest online retailer. Machine learning has gained momentum thanks to the growth of inexpensive computing power. And Amazon's HR department was about to start playing big: Since June 2015, the company's workforce has more than tripled to 575,700 people.
• Thus, a team appeared at the Edinburgh Amazon Engineering Center, which later grew to a dozen people. Her goal was to develop an AI that could quickly crawl the entire internet and identify suitable candidates for recruiting.
• The group created 500 software prototypes focused on specific job functions and the location of potential candidates. They taught each system to understand about 50,000 terms that appeared on the resumes of past candidates. However, something went wrong and the algorithms stopped paying attention to key skills in the work of IT specialists. Instead, the technology favored candidates who described themselves using verbs more commonly found on male candidates' resumes.
• Gender bias was not the only issue: unqualified candidates were often recommended for all types of jobs. The technology selected candidates almost at random, Reuters sources say.

Context

• Amazon's story is not unique. Other companies are also developing similar technologies. Kevin Parker, chief executive of startup HireVue, said automation is helping firms go beyond traditional recruiting techniques that have long been considered subjective. HireVue analyzes the speech and facial expressions of candidates in video interviews to reduce companies' reliance on well-designed resumes. Goldman Sachs, for example, has created its own resume analysis system that compares a candidate's resume with a desired job to see if it matches the applicant's wishes.
• LinkedIn has gone further. The social network offers employers an algorithmic ranking of candidates based on their suitability for posting vacancies on its website. However, John Jersin, VP of LinkedIn Talent Solutions, said it's not a replacement for traditional recruiters. “Technology is not perfect. I will not trust any AI system today to make a hiring decision,” he said.
• As for Amazon, the company managed to salvage some of what it learned from its failed AI experiment. She currently uses lightweight recruiting software that can perform basic tasks like screening resumes submitted multiple times, the agency source said. According to another source, Amazon has already assembled a new team to finalize the program.

Egor Sonin