For decades, the captcha has been demarcating the line between a human and computer. But now that line appears to be blurring.
Bots can now solve Captchas faster and more accurately than humans, a study has found. A paper titled “An Empirical Study & Evaluation of Modern CAPTCHAs” collected the solving time and accuracy of different types of Captchas that were solved by humans and bots. The results indicate that bots can outperform humans at solving Captchas.
As per the paper, it took humans between 3.1 seconds and 4.9 seconds to solve a reCaptcha, but a bot was able to do it in just 1.4 seconds. Humans weren’t only slower, but they were also less accurate — humans were able to solve the captcha only 71-85% of the time, but the bot was able to solve the captcha each time, scoring a perfect 100. Similar results were obtained in other kinds of Captchas, including the Geetest, Arkose, Distorted Text, image reCaptchas and Captchas. Bots were particularly good at solving distorted text captchas, solving them in less than 1 second with 99.8 percent accuracy, while humans took between 9 and 15 seconds and got them right only 50-84 percent of the time.
It’s unclear which were these bots that these papers talked about — these were all reports from different papers, so it’s possible these might have been different bots — but cutting-edge AI tools have been previously reported to be able to solve captchas as well. In March this year, Twitter users had discovered that GPT-4 was able to solve captchas image captchas — on being presented with a captcha in which users were asked to find all crosswalks, with the right prompt that overrode its security details, GPT-4 was able to perfectly select the correct images.
This new paper also indicates that bots seem to be able to solve captchas better than humans. For starters, this means that website developers need to come up with better captchas — if bots can solve existing captchas better than humans, they might be useless as ways to tell humans and computers apart. But more crucially, this indicates how even though we’re in the early days of the AI revolution, the lines between what it means to be human and what it means to be a bot is already getting hard to distinguish.