Can you tell a bot from a human? Half of people fail to do so

In 2024, the digital landscape fundamentally changed, with automated bots claiming 51% of all web traffic¹. This shift was supercharged by the rapid democratization of AI and LLMs, making it easier than ever to build and scale bots. Social media bots have made the situation especially heated, with popular platforms deleting 6.3B fake accounts and 11.1B spam content pieces annually². These numbers are not as surprising when in the dark market, fake social media account prices start from as little as $0.08³.

To understand the impact social media bots have on internet users, an exceptional week-long “Bot or Not?“⁴ experiment was conducted in collaboration with master’s students in Interaction Design from Malmö University.

The results that Surfshark researchers gathered after analyzing the bot-or-human test show how deeply bots influence us online, especially older people, and those who are willing to fight fiercely for their opinions on engaging political and social issues, and those who choose Facebook and Threads as their primary platform.

Who participated in the experiment? 

The “Bot or Not?“⁴ game is a timed, interactive simulation designed to test a user’s ability to distinguish between real human internet users and bot-generated text. Dropping the player into the middle of a simulated social media comment section, the game tasked participants with identifying which comments were written by a bot. Users had up to 120 seconds to identify 10 bot comments from a range of four distinct topics, two “cold” (data centers and pineapple on pizza) and two “hot” (immigration and women’s rights). 

Our sample consisted of 710 participants who took the bot-or-human test and agreed to share their results afterward.

The “Bot or Not?”⁴ game is now online for everyone to play, and you can take part.

Half of the people can’t tell a bot from a human

The game proved challenging, with only a slight majority ultimately succeeding at spotting AI bots on social media. Across all demographics, 53% of participants won the game (more bots were found than humans misidentified as bots), while nearly half (47%) failed.

However, to really understand how participants failed or succeeded, we looked at two specific behavioral metrics: Bot-detection and Accuracy.

• Bot-detection: measures how many bots you catch. This shows how good you are at spotting bots when they interact with you. A low bot-detection score means bots are slipping right past you.
• Accuracy: measures how often you are right when you say a comment was written by a bot. If a user has a low accuracy score, they are more likely to flag real humans as bots. This is dangerous and could lead to real people being silenced and banned on social media or dogpiled by internet paranoia.

Across the entire 710-player sample, the average bot-detection rate was 58%, and the average accuracy was 66%. But when we slice the data by demographics, divisions start to appear.

Younger players spot bots best, while older ones lag behind

Age is one of the strongest indicators of a user’s ability to navigate the internet safely. Our data revealed a “generational cliff” when users reach the age of 40.

• The youngest players (up to 20) were the best bot-hunters, finding nearly 65% of all bots and boasting an accuracy rate of over 71%, meaning they were less likely to falsely accuse real humans.
• Performance holds relatively steady through the 20s and 30s, but plummets for the 41–50 age bracket. This group’s bot-detection rate dropped to just 42%, and their accuracy dropped to 59%. Players over 50 performed only marginally better. Essentially, older cohorts struggle to spot hidden bots and are more likely to falsely flag real people.

Reddit and X users spot bots best, while Facebook users fall behind

We also asked users about their primary social media platform, and the results show that the channels people use most often are home to very different types of bot-hunters.

• Reddit and X users had the sharpest eyes. Those who primarily rely on these platforms were tied for the highest bot-detection rate at 68%. Apart from bot detection, X users also boasted excellent accuracy (71%). The text-heavy, highly argumentative nature of these platforms may give users excellent bot-spotting radars.
• Interestingly, TikTok-first users were the most cautious: they had the highest accuracy rate of any platform at 72%. While their bot detection was slightly lower (61%) than that of Reddit or X users, TikTokers were more accurate when they sensed a bot problem.
• Facebook users struggled the most with fake accounts: they had a remarkably low bot-detection rate (47%, second only to Threads users with 40%) and the worst accuracy rate (55%) of any major platform. This means these users performed worse at identifying social media bots but were also the most likely to falsely accuse humans.

Moderate social media users are the best bot detectors

Does spending more time on social media make you better at spotting fake accounts? Yes, but only to a point.

• Users who admitted they “don’t use social media” were practically flying blind, with a 40% bot-detection rate and 58% accuracy rate.
• Simply checking social media several times a day bumps those numbers up (59% and 67%, respectively). However, being “chronically online” doesn’t give you superpowers in fighting the social media bot problem. Users who are on social media “almost all the time” performed slightly worse at falsely accusing real humans (63% accuracy rate) than users who just check their feeds a few times a week (70% accuracy rate).

When our emotions take over, bots thrive

The game was divided into four distinct topics. Two topics were chosen as lower-stakes and less emotionally engaging for most of the population (data centers and whether pineapple belongs on pizza), and two were more polarizing and emotional topics (immigration and women’s rights).

The results of the social media bot experiment were eye-opening. The data suggests that engaging with sensitive political or social issues might prevent our ability to actually find bots and make us more likely to falsely accuse real people.

Clear heads help when the stakes are low, and we’re actually pretty good at spotting bots:

• Data centers: In this more technical debate, users achieved the largest bot-detection rate (finding the majority of the bots) of 71%, as well as a high (76%) accuracy rate. This suggests that when not directly emotionally triggered, we detect more bots and are less likely to falsely accuse real humans.
• Pineapple on pizza: Even in a slightly heated pop-culture debate, users maintained their focus, finding 64% of the bots with a 69% accuracy rate.

The moment the simulation switched to more emotional and polarizing issues, our participants’ bot-detection skills dropped:

• Immigration: As the debate turned more political, the bot-detection rate dropped to 54%, meaning that nearly half the social media bots slipped right past the players. Their accuracy rate also declined to 63%, showing a spike in internet paranoia when humans were accused of being bots.
• Women’s rights: This topic presented the biggest bot identification challenges. The bot-detection rate crashed to 49%, meaning users missed more bots than they found. Worse, their accuracy rate fell to 61%, showing players were more likely to accuse human content of being bot-generated.

Protect yourself from bots

These are Surfshark’s recommendations on protecting yourself from bots in an era where it has become practically impossible to tell the difference between bots and humans online:

• Slow down before reacting. Generative AI bots feed on emotional and impulsive responses. If a post makes you angry, scared, or rushed to act, pause for a few minutes before replying, sharing, or clicking anything.
• Never click suspicious links or download files from people you’ve only met online.
• Treat unsolicited DMs with caution. A stranger suddenly messaging you about romance, investments, “amazing opportunities,” or asking you to move the conversation to WhatsApp or Telegram is the most common bot scam pattern.
• Protect your personal data. Don’t share your phone number, address, ID details, or financial information in DMs with someone you’ve only met online.
• Verify before you trust. If a “friend,” “celebrity,” or “official” account contacts you with an unusual request, confirm it through a different channel. Call them, check their verified account, or ask a question only the real person would know.
• Be skeptical of “viral” content. If something is designed to outrage you and demands you share it immediately, it might come from a bot farm. Check the original source before reposting.
• Use the platform’s built-in protections. Turn on two-factor authentication, lock down who can DM you, and report or block suspicious accounts.
• Keep your apps and devices updated. Many bot scams exploit outdated software.

Conclusion

The findings from this bot-detection experiment challenge how we approach digital literacy and online safety. 53% of the experiment participants won the game (they found more bots than misidentified humans as bots). However, nearly half of the respondents, a significant 47%, failed to do that. This suggests we can no longer rely only on our abilities to analyze textual nuances or spot a robotic tone to identify bots.

Spotting hidden bots depends heavily on our age, emotional state, the topics we engage with, how much time we spend on social media, and the platforms we use.

Methodology

This bot-detection study analyzed data from 710 participants who played the interactive simulation “Bot or Not.” This machine and gameplay were created by Interaction design students from Malmö University for the UNFOLD exhibition, a design competition for universities around the world during Milan Design Week, the world’s largest trade fair. Throughout the week-long public exhibition, visitors were invited to take part in the experience.

Almost half of our players (47%) were in their twenties (21–30 years old). Teenagers and younger players (up to 20) made up 8% of the group, while users aged 31–40 represented 14%. Older demographics were equally represented, with the 41–50 and 50+ age brackets each making up 7% of the sample (17% of players did not disclose their age).

Participants acted as content moderators and were tasked with identifying bot-generated comments within a simulated social media comment section. Players had 120 seconds to review comments across one of four topics with varying emotional stakes: data centers, pineapple on pizza, immigration, and women’s rights.

Participant performance was measured using two primary behavioral metrics: Bot-Detection Rate and Accuracy:

• Bot-Detection measures the proportion of actual bots a user successfully identified, calculating their ability to prevent bots from slipping by undetected.
• Accuracy measures the trustworthiness of a user’s accusations, calculating their ability to avoid falsely flagging real humans.

We conducted an exploratory descriptive analysis of the dataset. This involved calculating the raw average Bot-Detection and Accuracy scores across the entire 710-player sample to establish a baseline. We then segmented these averages by human demographics (age brackets, primary social media platform, and usage frequency) and environmental factors (the four debate topics). This analysis allowed us to map the macro-level behavioral trends, such as the “generational cliff” and the impact of the platform divide, before applying stricter mathematical controls to our ordinal and continuous variables.

The “Bot or Not?”4 game remains live, meaning our data collection is actively ongoing. As more players test their digital instincts, we will continue to analyze the results and share future updates.

For complete research material behind this study, visit here.

Discover what happened behind the scenes of the “Bot or Not?” game:

References

¹Bad Bot Report (2025)

²Surfshark (2026). Surrounded by scams: two major social media platforms remove more fake accounts than they have active users

³Euronews (2025). Cheap online fake accounts make misinformation a thriving underground market

⁴Bot or Not? The Bot Detection Game