American View: Why Do Users Make Irrational Cyber Risk Decisions?

Risk Management19 Nov 2024

I teach cybersecurity skills and threats for a living, so it should come as no surprise that I “bring my work home” and inflict my family to lots of cybercrime news that I think is interesting. They’ve learned to endure my stories while feigning interest. Sometimes, my nonsense even sticks. As an example, my wife told me last week about the new phish reporting tool one of her employers had just rolled out and how she’d had to explain its function to her co-workers. I’m thrilled that she took some of my many nerdy phishing tales to heart … and I’m proud that she’s taken it upon herself to teach her peers how to protect themselves.

Still, it’s easy for me to forget that not everyone has a nerdy, cybercrime obsessed spouse to draw on for up-to-the-minute threat intelligence. Or even a job where cybersecurity is considered important enough to warrant decent training. Most American adults get the bare minimum of security training if they can get any at all. That helps explain why so many people put far too much trust in vendors and third parties to protect them, especially from themselves.

Case in point: the “Watchdog” byline in last Sunday’s edition of the Dallas Morning News led with a breach-and-blackmail story. Consumer advocate and investigative journo Dave Lieber’s article titled Dear Watchdog: What’s the story with sexual blackmail, electric car fees and Cirro Energy? caught my attention … if, for no other reason, to see if those three topics were interrelated.

THREE FOR THREE! WOOOO!.

They weren’t – fortunately? – however the lead story in Dave’s three-part article was exactly my sort of news: a reader had heard about a website breach and asked Dave to explain it. Dave had worked out that the reader was referencing the Muah.ai breach that Malwarebytes had written about back on 9th October. Their article, hilariously titled AI girlfriend site breached, user fantasies stolen, described how “a  hacker has stolen a massive database of users’ interactions with their sexual partner chatbots.” … a sentence I never thought I’d be quoting, even in my most dystopian cyberpunk nightmares. It’s real … but it feels like it shouldn’t be. Anyway …

I’d heard about the breach back in October. I’ve been deeply sceptical of open-source pseudo-AI products since the market started obsessing over anything featuring an “AI” label. I’ve been deeply sceptical of internet start-ups since the first Dot Com Boom, having worked in that insane ecosystem. That’s probably why I didn’t take much notice of the original story.

Still, Dave’s mention of the event and helpful link to the original news stoked my interest. As soon as I read Malwarebyte’s explanation of the breached site as “… a platform that lets people engage in AI-powered companion NSFW chat, exchange photos, and even have voice chats” I noped right out. Even if that tool had been built by the finest minds in tech, its premise alone should have made it seem radioactive to anyone who wanted to remain employed, married, and sleeping indoors. Who on earth would’ve dared use it?

Seriously, consider the risk assessment for this site: you’re a user. You create an account with a user ID, a viable email address, and an IP that can be traced back to you even if you’re using an alias. Then you share your darkest, most problematic and adult desires with an LLM that’s recording everything you say. Then – to double down – suggests even more de Sadean fantasy fodder ... and records your reactions to its counter prompts. Imagine how bad your exchanges might be judged by your spouse, employer, snotty neighbour, local gossip columnist, or scandal obsessed newsreader. Are you absolutely sure that your wildest erotic fantasies wouldn’t inspire scorn or revulsion if dragged into the public spotlight?

No matter what you’re into, someone’s guaranteed to be repulsed by it. Humans are weird.

Now let’s take the horror show a step further: your “AI girlfriend” is an LLM that’s trawling not just your inputs, but all the inputs of every other customer using the site. Sure, your virtual dalliances might be quaintly tame, but the “suggestions” you get from the engine will be filled with popular content that other – let’s say “more adventurous” – customers have submitted. Even if you weren’t interested in coulrophobic paraphilia when you started using Muah.ai, your responses to your “partner’s” recommendations might cast you in a deplorable light. How much reputational damage might you suffer if your “private” exchanges were leaked?

Using the bog-standard 5X5 risk assessment matrix the potential impact of your data being made public in a Muah.ai style breach would likely be catastrophic … the classic 5 in the severity row. In the best outcome – if your adult fantasy data could qualify for a 12A rating – the impact to you would still be no less than moderate. Everything more adult than that would constitute high or extreme risk. In short, using the site for its intended purpose should appear absolutely toxic. As in, “stay the *#&$ away if you value your current life” levels of hazardous.

More importantly, the potential value of this site’s data to an attacker cannot be overstated. Compromised “private” communications involving sex, impropriety, and deviance is highly valuable fodder for a blackmailer. Twitter user Greg Linares reported on 10th October that he’d heard of two extortion attempts against developers. Imagine if one of your co-workers who holds root access on your network was the next user to be threatened into turning over their credentials. Scary stuff!

Or, rather, it’s obviously scary stuff when you’re familiar with online technologies, risk assessments, and cybercrime. I’m no tier three network engineer, but even I understand the stakes and the danger of a breach like this. I know that if I were somehow interested in a tool like this, I’d realize that the potential consequences of using it were far too severe to accept.

It’s one thing to risk your own future on a wild thrill; quite another thing to risk you and your family’s future, especially when you’ll live with the consequences.

That’s coming from a security professional, though. As I said up at the top of this column, most American adults get the bare minimum of security training … if they can get any at all! It’s unreasonable and condescending of us in the security world to expect normal people to view and understand the tech world the way we do: jaded, distrustful, and deeply paranoid. The people behind Muah.ai claimed that their site and services were “secure.” How is a normal user expected to evaluate the believability of that claim?

That’s why I side with the Human Risk luminaries who believe we ought to be teaching our users about these risks and best practices of using these sort of extracurricular activities. I understand the counterargument from corpo lawyers that there’s risk inherent to advising worker about behaviours that aren’t limited to the office or company business. That’s a lawyer’s job. Still, I submit that the strategic risk to the enterprise is much higher if we don’t teach our users these skills and they later get extorted than we’ll face if we get sued by a disgruntled employee. Left unmitigated, this human vulnerability has the potential to inflict crippling damage. Therefore, if no one else is preparing for it, it’s our responsibility to do so.

You might disagree. That’s understandable. Still, I suggest that you play this scenario out in your next high-level tabletop exercise: bad guys are running rampant inside your network because a trusted sysadmin was blackmailed into turning over their credentials in exchange for not revealing their humiliating chats and revelations from a breached “dirty dreams” website. Could you have prevented this penetration? The only rational conclusion is yes … you could have … but only if you’d invested years in creating an office culture where your users felt safe coming to your security staff or line leadership to report the initial extortion attempt. Without that investment, no amount of user behaviour monitoring or written policies will make a bit of difference.