Artificially unintelligent phishing?
« previous post | next post »
I have something to add to The Economist's list of "How businesses are actually using generative AI", namely creating phishing messages that are even more implausible than those generated by rooms full of non-native hirelings.
A few days ago, I got an email from ParceITrks@intimidatingthebook.com, informing me that "A package awaits your action for rescheduling":
I was especially impressed by the explanation at the bottom of the note that
In dialysable swiftly changing realm of international logistics, incapacitates adherence to stringent industry norms is essential. granularse norms are eradictions cornerstone for keister safe, effective, and eco-friendly movement of merchandise across frontiers. cattedy encompass everything from compliance with safety regulations to inhibitive deployment of state-of-hideouts-art tracking systems, all aimed at elevating service excellence and fortifying customer loyalty. Additionally, autosuggestionse norms are instrumental in ensuring smooth compliance with global trade laws, dreamilyreby knitting a more robust and interconnected global trade fabric.
The only explanation I can come up with is that some semi-(in)competent crook came up with the idea of training a language model to generate varying "explanations", in the hopes of fooling spam-detectors that rely on particular word sequences or similar features.
Or maybe the whole thing is a joke, or the result of a bar bet between some undergrad computer science majors?
Philip Bock said,
March 3, 2024 @ 1:38 pm
That block of semi-randomly generated text is designed to fool anti-spam filters that rely on statistical methods, so they won't block the message as they should. The stilted text in the main body is probably also deliberate, for a similar reason.
Y said,
March 3, 2024 @ 1:56 pm
It's a bit like the drug-addled conversations in Asimov's I'm in Marsport without Hilda.
JPL said,
March 3, 2024 @ 4:10 pm
I see at least one possibly useful neologism in there: "keister safe" (or, "keister-safe"), if you ever need an expression to describe something like "arrangements that will allow you to "cover your ass". E.g., "When confronted by the accusation of impropriety in his department, the Secretary delivered a keister-safe response".
Martin said,
March 4, 2024 @ 2:44 am
The algorithm appears to be to replace the string 'the' in pre-existing text with a random word… No language models needed!
Andreas Johansson said,
March 4, 2024 @ 2:51 am
I've long assumed that many phishing email are deliberately incoherent in the hope that the recipient will click one of the links in the hope of clarification.
Cervantes said,
March 4, 2024 @ 8:08 am
Martin — that's exactly correct. This was a piece of real text — albeit corporate-speak — that was altered in exactly the way you describe. It is not AI generated.
Pau Amma said,
March 4, 2024 @ 9:57 am
Echoes of some spam I got in 2010, which included:
Cervantes said,
March 4, 2024 @ 10:38 am
For a long time, I derived my passwords from this, received in 2004 — written by a human, obviously.
From: Bel Erlaup (facetedlytripp@drafthouse.com)
To: Ben Fowlwa [Note: no relationship to my name or anything else]
Subject: Pliable and dissolvable tablets for serious guys
Our pills are just equal typical pills but they are specially developed to be supple and dissolvable under the lingua. The tables is absorbed at the mouth and gets into the blood direct alternatively of rising through the tummytum. This results in a quicker more vigorous upshot which run up to 31 hours!
Ben said,
March 4, 2024 @ 11:54 pm
@Martin Not just the word "they," but "they" and "thus" too
ycx said,
March 5, 2024 @ 7:57 am
Great catch by @Martin. Some obvious tells in the vein of "cdesign proponentists", where the substring "the" is being replaced with a random word, even in the middle of a word.
"cattedy" – > they
"autosuggestionse" -> these
"dreamilyreby" -> thereby
Christopher J. Henrich said,
March 6, 2024 @ 12:16 am
I think the discussion has identified *how* these weird postings are generated. The question of *why* remains. What's the bloody point of transmitting such a message?
I offer a suggestion: there really isn't much point; but the hapless poster thinks there is. He (or possibly she) lives in a country whose economy cannot find useful things for many people to do. So he sits around, desperately trying to think of an opportunity.
He is a potential victim of a scam which works like this. He is a would-be scammer, and also the victim of what I would call a "meta-scammer." The meta-scammer sells his victim software which will enable the victim to grab email addresses, and send out messages to those addresses. Part of the kit is a generator of email messages which will supposedly entice its recipient to send money to the poor guy I called "potential victim."
This is a new version of something we have seen before, in several forms. You may remember all the penis-enlargement ads; then there were the Nigerian prince who bequeathed you a shitte-load of money, and the oil magnate who wanted your help in transmitting another load of money, yada yada. Recently I have received several emails of the form "It is time to renew your subscription to SomeStupidThing.com; please send $99 to …" — These may be another example of a meta-scam.
There is a slight tendency for these meta-scams to become stupider and more obviously bogus as the years go by. (Perhaps the "penis enlargers" were an exception.)
Andreas Johansson said,
March 6, 2024 @ 3:56 am
Hm. The once ubiquitous widows of Mobutu who needed my help to move money out of the Congo seem to have died out sometime in the '10s. Nowadays the commonest kinds of spam I get are women who saw my profile on some dating site I'm not on and desperately want to have sex with me, and financial institutions who want to lend me money. (The latter is a bit curious since lenders typically want their money back at some point.)
Are there studies of trends in spam approaches? How universal are they? Do others get a similar mix, or is, say, the amount of dating spam I get related to something I do?
Terry Hunt said,
March 6, 2024 @ 3:12 pm
@ Andreas Johansson – In order to 'lend you money' or in some other way pay you (fake job offers are also used) they will at some point inveigle you into giving them your bank account details, and will then proceed to empty your account.
Kenny said,
March 9, 2024 @ 3:12 pm
One claim I've heard (but not really verified) is that some of these phishing scams are intentionally badly done, so that people with any skepticism at all won't click through and waste the time of the would-be scammer.