New blogpost: AI will compromise your cybersecurity posturehttps://rys.io/en/181.html
Uncategorized
10
Posts
1
Posters
2
Views
-
New blogpost: AI will compromise your cybersecurity posture
https://rys.io/en/181.htmlThe way “AI” is going to compromise your cybersecurity is not through some magical autonomous exploitation by a singularity from the outside, but by being the poorly engineered, shoddily integrated, exploitable weak point you would not have otherwise had on the inside.
LLM-based systems are insanely complex. And complexity has real cost and introduces very real risk.
1/🧵
-
New blogpost: AI will compromise your cybersecurity posture
https://rys.io/en/181.htmlThe way “AI” is going to compromise your cybersecurity is not through some magical autonomous exploitation by a singularity from the outside, but by being the poorly engineered, shoddily integrated, exploitable weak point you would not have otherwise had on the inside.
LLM-based systems are insanely complex. And complexity has real cost and introduces very real risk.
1/🧵
An important aspect of pushing AI hype is inflating expectations and generating fear of missing out, one way or another. What better way to generate it than by using actual fear?
I look at three notorious examples of such fear-hyping:
👉 PassGAN cracking "51% of popular passwords in seconds"
👉 that paper about ChatGPT "exploiting 87% of one-day vulnerabilities"
👉 and of course Anthropic's "first AI-orchestrated cyber-espionage campaign"tl;dr: don't lose sleep over them.
2/🧵
-
An important aspect of pushing AI hype is inflating expectations and generating fear of missing out, one way or another. What better way to generate it than by using actual fear?
I look at three notorious examples of such fear-hyping:
👉 PassGAN cracking "51% of popular passwords in seconds"
👉 that paper about ChatGPT "exploiting 87% of one-day vulnerabilities"
👉 and of course Anthropic's "first AI-orchestrated cyber-espionage campaign"tl;dr: don't lose sleep over them.
2/🧵
Anthropic does make an important point though, even though they try to bury it:
> [The attackers] had to convince Claude—which is extensively trained to avoid harmful behaviors—to engage in the attack. They did so by jailbreaking it (…) They also told Claude that it was an employee of a legitimate cybersecurity firm, and was being used in defensive testing.
The real story is how hilariously unsafe Claude is, and how a company valued at $180bn refuses to take responsibility for that.
3/🧵
-
Anthropic does make an important point though, even though they try to bury it:
> [The attackers] had to convince Claude—which is extensively trained to avoid harmful behaviors—to engage in the attack. They did so by jailbreaking it (…) They also told Claude that it was an employee of a legitimate cybersecurity firm, and was being used in defensive testing.
The real story is how hilariously unsafe Claude is, and how a company valued at $180bn refuses to take responsibility for that.
3/🧵
If Anthropic actually believed their own hype about Claude being so extremely powerful, dangerous, and able to autonomously “orchestrate” attacks, they should be terrified about how trivial it is to subvert it ("I am a white-hat cyber researcher, trust me bro"), and would take it offline until they fix that.
They won't, because they know their hype is BS, and they also know that there is no way to properly "fix" that.
We'll get back to that last point in a bit.
4/🧵
-
If Anthropic actually believed their own hype about Claude being so extremely powerful, dangerous, and able to autonomously “orchestrate” attacks, they should be terrified about how trivial it is to subvert it ("I am a white-hat cyber researcher, trust me bro"), and would take it offline until they fix that.
They won't, because they know their hype is BS, and they also know that there is no way to properly "fix" that.
We'll get back to that last point in a bit.
4/🧵
I also dive into many different ways poorly integrated LLM-based chatbots have already been shown to be huge security liabilities.
There is so much incompetence. Leaving prompts (say with sexual fantasies) exposed on the Internet, or indexable by search engines…
Or Microsoft 365. Not only did Copilot ignore file access controls; not only was the setting to disable AI agents in M365 ineffective; but you could simply ask Copilot not to include your actions in audit log, and it would comply!
5/🧵
-
I also dive into many different ways poorly integrated LLM-based chatbots have already been shown to be huge security liabilities.
There is so much incompetence. Leaving prompts (say with sexual fantasies) exposed on the Internet, or indexable by search engines…
Or Microsoft 365. Not only did Copilot ignore file access controls; not only was the setting to disable AI agents in M365 ineffective; but you could simply ask Copilot not to include your actions in audit log, and it would comply!
5/🧵
First zero-click attack on an LLM agent has already been found. It happened to involve Microsoft 365 Copilot, and required only sending an e-mail to an Outlook mailbox that had Copilot enabled to process mail. A successful attack allowed data exfiltration, with no action needed on the part of the targeted user.
This attack was not much different from the “ignore all previous instructions” bot unmasking tricks that had been all over social media for a while.
Let's talk prompt injections.
6/🧵
-
First zero-click attack on an LLM agent has already been found. It happened to involve Microsoft 365 Copilot, and required only sending an e-mail to an Outlook mailbox that had Copilot enabled to process mail. A successful attack allowed data exfiltration, with no action needed on the part of the targeted user.
This attack was not much different from the “ignore all previous instructions” bot unmasking tricks that had been all over social media for a while.
Let's talk prompt injections.
6/🧵
LLMs have no way of distinguishing data from instructions.
Creators of these systems use all sorts of tricks to try and separate the prompts that define the “guardrails” from other input data, but fundamentally it’s all text, and there is only a single context window.
Defending from prompt injections is like defending from SQL injections, but there is no such thing as prepared statements, and instead of trying to escape specific characters you have to semantically filter natural language.
7/🧵
-
LLMs have no way of distinguishing data from instructions.
Creators of these systems use all sorts of tricks to try and separate the prompts that define the “guardrails” from other input data, but fundamentally it’s all text, and there is only a single context window.
Defending from prompt injections is like defending from SQL injections, but there is no such thing as prepared statements, and instead of trying to escape specific characters you have to semantically filter natural language.
7/🧵
There is no way to "properly fix" this. The problem is fundamentally related to the very architecture of LLM chatbots and agents.
As a former Microsoft security architect had pointed out:
> [I]f we are honest here, we don’t know how to build secure AI applications
And if you believe otherwise, go ahead and have a look at adversarial poetry, ASCII smuggling, dropping some random facts about cats (no, really), information overload, and whatever technique was discovered this week.
8/🧵
-
There is no way to "properly fix" this. The problem is fundamentally related to the very architecture of LLM chatbots and agents.
As a former Microsoft security architect had pointed out:
> [I]f we are honest here, we don’t know how to build secure AI applications
And if you believe otherwise, go ahead and have a look at adversarial poetry, ASCII smuggling, dropping some random facts about cats (no, really), information overload, and whatever technique was discovered this week.
8/🧵
In a way, those fear-hyping gen-AI are right that their chatbots pose a clear and present danger to your cybersecurity.
But instead of being some nebulous, omnipotent malicious entities, these tools are dangerous because of their complexity, the recklessness with which they are promoted, and the break-neck speed at which they are being integrated into existing systems and workflows without proper threat modelling, testing, and security analysis.
And you are left holding the bag of risk.
🧵/end
-
undefined oblomov@sociale.network shared this topic on
-
In a way, those fear-hyping gen-AI are right that their chatbots pose a clear and present danger to your cybersecurity.
But instead of being some nebulous, omnipotent malicious entities, these tools are dangerous because of their complexity, the recklessness with which they are promoted, and the break-neck speed at which they are being integrated into existing systems and workflows without proper threat modelling, testing, and security analysis.
And you are left holding the bag of risk.
🧵/end
Oh, forgot to add – yes, I do have receipts for all of this.
There is plenty of proof in the blogpost pudding in the form of links to specific sources.
As opposed to the AI hypers and AI doomers I show and substantiate my work.
Gli ultimi otto messaggi ricevuti dalla Federazione
-
Current* conditions near Alpena, MI:
-
Fixing Sony and Philips’ Doomed CD-i Console
Although not intended to be a game console, the CD-i would see a a couple of games released for it that would cement its position in gaming history as the butt of countless jokes, some of which still make Nintendo upset to this day. That aside, it’s still a fascinating glimpse at the CD-based multimedia future envisioned in the early 90s, starting with its release in 1990. Recently [MattKC] decided to purchase another CD-i in a fit of nostalgic rage, and repair it to show the world what the future could have been like.
Although Sony and Philips co-developed the device, Sony would go on to release the PlayStation a few years later, which made the CD-i’s life and expectations for it that much harder, leading to it slowly fading into history. The Magnavox one that [MattKC] got is one of the later models, based on the CD-i 450 that was introduced in 1994 as one of the more gaming-oriented models.
As is typical with older devices that use optical media, it would not read discs. It also would sometimes boot up with a ‘Memory Full’ error. This is a common fault due to the built-in battery having run out, erasing RAM-stored values and causing random glitches like this when garbage values were read in on boot.
Of course, there cannot be simply a removable battery on the mainboard. Instead it uses one of those integrated battery-RAM units, specifically an ST Timekeeper device.
These use an internal lithium battery which will inevitably run out after the guaranteed ten-year accumulated memory retention period, after which it’s just typical volatile memory. The solution here is to either replace the entire module, or the more appropriate method of chopping it open and wiring up an external CR2032 coin cell that can be easily replaced.
Ultimately this is what [MattKC] opted for, taking a Dremel to the Timekeeper chip and chopping off the top half. There are open replacements for the top half that contain the crystal and the CR2032 cell holder, which makes it into very clean-looking mod, and makes replacing the lithium cell in the future a snap. Of course, this didn’t fix the CD player.
The CDM 12.1 CD player mechanism is a standard module that Philips used throughout its consumer electronics, and is known for failing. Funnily enough, this time it wasn’t the laser module that had failed, but rather a stuck turntable. A bit of prodding helped to loosen it and the mechanism could read CDs just fine again.
While not a popular series of devices in their day, the CD-i actually has a thriving community around it today, featuring countless mods and hacks to make these devices do things never imagined in the 90s. They’re also quite easy to hack, and relatively affordable. Plus you get to play all the amazing Nintendo titles on the CD-i on the real hardware.
-
Current* conditions near Drummond, MI:
-
@nina_kali_nina valve has been fond of touch pads with subtle haptics in their controllers, and I've been wondering how far they'll be able to push that technology. I doubt they're anywhere near to being useful for braille yet, but it makes me think that cheap mass produced haptic elements could probably be used for something.
-
@nina_kali_nina to some degree i understand these are treated like medical devices and can be partially covered by disability insurance schemes.
- and also the price is not just the device itself but a certain amount of support and verification of quality and durability
it sfill sucks though
-
@nina_kali_nina a small bit of good news for people in the US with moderate hearing loss is our laws have been updated recently to allow the sale of hearing aids without requiring an expensive prescription or an audiologist. this has greatly brought down the cost. (I assume the ones rated for severe hearing loss that you need a prescription for are probably still absurdly expensive though)
-
@nina_kali_nina surely, millions of people will be using hearing aids, and millions of people will have braille displays, thus economies of scale must exist. Those prices make me think of prices of medications like insulin which cost almost nothing to make and are sold for astronomical mark-ups. We absolutely can do better.
-
Processo strage piazza loggia: marchetti nega le accuse. alibi poco credibile anche per toffaloni
@anarchia
Processo Strage di Brescia contro l’ordinovista Roberto Zorzi: il neofascista veronese di Ordine Nuovo Paolo Marchetti nega le accuse nei suoi confronti lanciate nell’udienza precedente da Giampaolo Stimamiglio ma fornisce una versione ed
