I see sentiment like this (which is valid; it’s a different perspective), and then I look at my company’s current caseload of breaches and how most of the (insane) increase in business we’ve received is caused by poorly coded apps with obvious security issues, along with the inability of orgs to remediate those issues or adequately follow incidents because no one actually knows the applications anymore.
And I’m wondering if this isn’t the enormous amount of organizational debt from having security second to everything finally coming calling.
I’ll give you an example of one that was written recently actually.
The initial compromise happened because the app explicitly did not verify auth claims when a specific string was in the ISS field. Well, fuzzers exist and are common.
The next issue was that once you’re in, there was no delineation between admin and regular users. Everyone had all privileges if they just made the calls.
Anyway, we did the usual post-remediation investigation and write up. The devs were of course using the latest models, as they were instructed, and the issue stemmed from a problem they’d been having integrating a specific company into their auth scheme.
Eventually, after many enumerations, the model opted to just skip auth altogether if that companies ISS was present. The devs, being in the habit of just accepting the changes did so and because of the nature of the code implemented nothing caught it in the pipeline.
This is sadly an incredibly common story and it won’t be fixed by models improving I don’t believe.
Not in security, but "vibe coding remediation"--that's the best way I can think to describe it--has been a booming business in my line of work (SIP/VoIP infrastructure engineering) as well.
How long it'll last, I can't say, only that before LLMs, the industry entered a phase of consolidation and quiescence that had me seriously worried about a fade into irrelevance. The delusions of businesspeople and the managerial class about what LLMs can do has fueled a very nice business renaissance, as they run into the limits but have committed to contracts, business models, etc.
And I’m wondering if this isn’t the enormous amount of organizational debt from having security second to everything finally coming calling.
I’ll give you an example of one that was written recently actually.
The initial compromise happened because the app explicitly did not verify auth claims when a specific string was in the ISS field. Well, fuzzers exist and are common.
The next issue was that once you’re in, there was no delineation between admin and regular users. Everyone had all privileges if they just made the calls.
Anyway, we did the usual post-remediation investigation and write up. The devs were of course using the latest models, as they were instructed, and the issue stemmed from a problem they’d been having integrating a specific company into their auth scheme.
Eventually, after many enumerations, the model opted to just skip auth altogether if that companies ISS was present. The devs, being in the habit of just accepting the changes did so and because of the nature of the code implemented nothing caught it in the pipeline.
This is sadly an incredibly common story and it won’t be fixed by models improving I don’t believe.
How long it'll last, I can't say, only that before LLMs, the industry entered a phase of consolidation and quiescence that had me seriously worried about a fade into irrelevance. The delusions of businesspeople and the managerial class about what LLMs can do has fueled a very nice business renaissance, as they run into the limits but have committed to contracts, business models, etc.