Can artificial intelligence and computer games improve hiring diversity? Here's why the jury is out.

It's hard for anyone to get a high-paying or leadership job at a top company, but that's especially true if you are a woman or person of color. Women make up more than half of the labor force, according to the Center for American Progress, but hold less than 11% of the board seats among companies on the Fortune 500. And though top technology companies say they want to recruit more people of color, a 2014 analysis from USA Today showed that black and Hispanic college students are graduating with computer engineering and science degrees at twice the rate that they're getting hired.

Hiring experts blame many factors, including an overemphasis on employee referrals as well as unconscious bias. The rushed nature of the hiring process doesn't help: Recruiters for well-known companies are typically inundated with applicants. One study found they may spend as little as six seconds reviewing resumes before making a judgment about a candidate's fit.

One proposed solution to these problems is, oddly enough, gaming. This idea isn't exactly new, though to date most recruiting games have focused on marketing a certain company or a certain role — rather than replacing the good old-fashioned resume and job interview model. Now, tech companies like Pymetrics and Knack want to change that, by building games informed by behavioral science and neuroscience that uncover traits important to employers — like altruism or the ability to weigh risks and rewards — while avoiding human biases that favor more homogenous hires. Their pitch is enticing at the very least: "Play games, get hired," Pymetrics' website reads.

But does this model actually work? Unilever North America is trying to find out. On Monday the company announced that hundreds of entry-level slots for interns, co-ops and full-time roles, including jobs in marketing and finance, have been filled in recent months — using Pymetrics hiring assessments. The employees were all recruited based on how they performed in a series of online games, as well as in a follow-up interview.

So far, Unilever is billing the experiment as a success, particularly in terms of increasing educational diversity: Hundreds of thousands of people applied from 2,600 different colleges and universities — about eight times the number of schools represented last year, including community colleges. The hiring pool also increased in "ethnic diversity" and stopped skewing male, the company claimed, noting that "gender balance was achieved," though a company spokesperson was not able to provide before-and-after figures by press time.

Experts in education and technology caution that, while heartening, evidence that games improve hiring pools isn't quite definitive yet. The experiment works only if the skills you want in workers end up being accurately predicted by the games, after all: "There is some belief that people have very specific core ability, and you’re trying to assess for that," said Dan Schwartz, dean at the Stanford Graduate School of Education and an expert in educational technology. "Oftentimes that’s a mistake... The question is: Can they correlate them, so that it isn’t actually just testing how good they are at the game?"

And what about filtering out biases? A crucial challenge for anyone building "unbiased" software is that even algorithms can bake in biases based on gender or race. While companies like Pymetrics say their systems avoid these problems, it is still too early in the Unilever experiment to say whether their method was successful or how their hires will fare at the company. Finally, assuming they do work out, another big question is whether these hiring solutions can scale up to help all types of companies — and hires beyond the entry-level. Here's what we know so far.

Can hiring games improve diversity?

Pymetrics claims its method carefully controls for bias: The company uses a database of "tens of thousands" existing profiles to test each assessment for signs of any demographic bias — if a certain group outperforms on a certain task, that task will get less weight in the version of the game that is ultimately served to applicants.

While it seems the new system grew the size of the applicant pool and increased socioeconomic diversity, Unilever's experiment is still in the early days in terms of actual implementation — the first batch of Pymetrics recruits started in January, and the better part of the class is still being trained and on-boarded into new roles.

Unilever officials say they hope recruiting games like Pymetrics' will broaden the hiring pool, while also making the hiring process more engaging and fun: "We wanted to reach more people," said Mike Clementi, a vice president in charge of North American HR at Unilever. "The way that recruiting is done in campus, it’s still the same way it was happening with me 20 years ago."

What do the test tasks generally look like? In one Pymetrics game testing for Unilever hires, screen-grabbed below, users have to indicate the direction of different colored arrows that appear in rapid succession, for example.

Proponents argue that these assessments can capture skills or attributes that traditional test scores and other gauges of achievement, like educational background, cannot. A key differentiator of Pymetrics' tests, said founder and CEO Frida Polli, is that there is not necessarily a right or a wrong answer.

"Schools are great, they do a fantastic job of educating people," Polli, neuroscientist and the founder and CEO of Pymetrics, said in a phone interview. "But there's probably lots of diamonds in the rough there. One of the most interesting things that came out of the Unilever story, they went from a very small sample to a huge one. They ended up hiring more people from community college, extremely diverse socioeconomic backgrounds."

If Unilever's hires turn out to be successful, that would be a breakthrough for the recruiting industry, which has long sought to become more predictive and fair, said Ethan Mollick, a professor of management at the Wharton School who studies the intersection of gaming and business. "Games in recruiting have had ups and downs over the years," Mollick said, at least in part because of the fact that, "every kind of game discourages some people and encourages others."

In its early days, for example, Google erected a series of billboards with a cryptic math problem. "{first 10-digit prime found in consecutive digits e}.com," the billboard read. The solution to the billboard's problem was a URL that led solvers to a webpage with a second math problem — and upon solving that one you were then finally taken to a Google recruiting page. The game was exclusionary by design: "For people who were really interested in math, they couldn’t resist the opportunity to do that problem," Mollick said.

But, of course, there's no way to know how many mathematically-talented potential hires were turned off by that gimmick. (Google says it no longer uses brainteasers in hiring.) Similarly, implicit biases might underlie one "canonical" recruiting game, an experiment from the U.S. military called America's Army, a first-person shooter game that mirrored military life. While America's Army was successful at increasing recruitment, at least one trailer for a version of the game features only male characters and voices.

The pros and cons of recruiting with games

The Pymetrics assessments are designed to detect skills that make you good for a role that you might not be keen to admit about yourself, Polli said. For example, high-achieving sales people tend to be a little impulsive, she said, and blurt things out. These aren't characteristics you'd put on a resume, but may reveal a hidden aptitude that employers actually want.

Polli says the company believes Pymetrics' form of testing could be implemented up and down the career ladder: "Any type of hiring can benefit from it," she said. "What research has shown is that cognitive profiles is more indicative of how they’re going to work in the field ... It’s how do your traits fit to a certain role."

Though early signs might be encouraging, some experts in organizational psychology caution that using games to successfully recruit is still experimental. "There is some real innovation there. But where the lagging is, what does it in fact relate to in the employment context?" said Frederick Morgeson, a management professor at Michigan State University.

He added that experimental testing, if it goes awry, may actually create a liability for hirers: "The recruiting process is one of the most litigious and regulated part of what [an employer] might do with their employees." The Equal Opportunity Employment Commission counts upward of 90,000 discrimination lawsuits each year. Indeed, the litigiousness of hiring might be one reason why the process is so hard to reform.

A 2016 Department of Labor audit to detect hiring bias at federal contractors found violations at many of Silicon Valley's biggest names, including Palantir Technologies, Oracle, Qualcomm and Google.

But Unilever's current class of new hires is a sign that the system can change, the company claims. "I don’t know that there’s any form of human interaction where some kind of bias doesn’t come out," Unilever's Clementi said. "So what you want to do is limit as many variables as possible... Is [the new application process] more male-centric? We haven’t seen that to be the case, based on our acceptance rate, based on our offer rates, and the application rates. It’s something to watch, something we’re curious about, we just haven't seen it play that way."

How do you get a better shot at getting hired?

It may be awhile before anyone can confirm whether recruiting games set people up for long-term success. Until Pymetrics recruits, for example, are further-established in their fields, it will be hard to say for sure whether their games predict a successful career arc. Just because you show some of the emotional and cognitive traits of a salesperson, for instance, doesn't mean you're ready to work in sales for a 30-year or 40-year career. That's a reason — before you accept a job offer — to make sure it's not just that you are right for the job: The job needs to be right for you, too.

As for discrimination? In general, the best way to protect yourself against biases during the hiring process is to know your rights, and keep a rigorous paper trail so you have the receipts if you believe you have been treated unfairly. If you belong to a protected class (like women or veterans) as acknowledged by the Equal Opportunity Employment Commission, you can file a claim directly through the agency, or have a surrogate file on your behalf.

And regardless of whether you are in a protected class or not, anyone can make an effort to work at companies and in states that place more emphasis on increasing the diversity of their workforces.

Finally, remember that it's still illegal for employers to inquire about your protected status, even if they do it in a roundabout way. A hiring manager is probably smart enough to know they can't ask about your religion, for example, or inquire if you plan to get pregnant in the near future, but they may inquire using a euphemism — known as stealth — for instance by asking whether you anticipate a "prolonged medical absence."

If that has happened to you, you might want to consider lawyering up.

Sign up for the Payoff — your weekly crash course on how to live your best financial life. Additionally, for all your burning money questions, check out Mic’s credit, savings, career, investing and health care hubs for more information — that pays off.