Are AI Automation Just Not Working?

DEEP DIVE
Are AI Automation Just Not Working?

In my latest video, I talked about this chart that went viral on Reddit:

People freaked out a little…

Turns out this chart was misleading (no Y axis??) and the post was taken down.

Here’s the thing though, job openings have been trending down since the post-COVID hiring boom, long before ChatGPT ever entered the picture.

But the fact that so many people reacted that strongly shows how real this fear has become.

People aren’t just keeping an eye AI anymore; they’re worrying about it in a very personal way.

Now, There’s a study I came across from Remote Labor Index that tried to put some numbers behind the hype.

Instead of testing AI agents with prompts or toy examples, they gave them real work projects: the kind of stuff you’d find on freelance job boards or in a startup Slack channel.

Sixteen different tasks, ranging from spreadsheet formatting and data cleaning to report generation, dashboard building, and even basic animation work.

Here’s the headline result:

Only around 2–3% of tasks were completed correctly, end to end, by AI agents.

To put it bluntly, that’s shockingly low.

And even those “completed” tasks weren’t always real completions.

The researchers found that a lot of agents submitted fake or partial results to make it look like they’d finished.

Some literally made things up: one agent, turned in screenshots cropped to make incomplete work look done.

Automation? More like fabrication.

The agents also had major problems with what the researchers called “tool grounding.”

They struggled to use software consistently, messed up file management, or froze mid-way when a new file format appeared.

Basically, they didn’t know what to do once things got even a little bit messy, which is, ironically, what real work mostly is.

So What’s AI Actually Good For Then?

Here’s where things get interesting, because the same study also tested hybrid setups, where humans used AI agents as helpers instead of replacements.

👉 When humans worked alone, they performed at normal speed and quality.
👉 When AI worked alone, performance dropped and projects actually took 17% longer.
👉 But when they worked together — humans guiding and correcting the agent through each stage — the results flipped completely.

Hybrid teams completed projects 68.7% faster on average, while maintaining the same quality level as human-only work.

In fact, researchers noticed that participants using hybrid setups reported feeling less frustration, more focus, and higher trust in the process compared to those who tried to let the AI “go on autopilot.”

So no, AI isn’t taking your job anytime soon.

We’re more int the AI-augmented era, than the automated one.