I find it a breath of fresh air to be able to tackle backlogged tech debt items simultaneously with business priorities in parallel and stuff like that.
That being said, I can't see this entire field existing in five years anymore. I'm hoping for at least two more years, but who knows?
This stuff is coming for all white collar, the barrier to entry is completely gone now. Maybe not the barrier to mastery (yet), but the bottom has fallen out.
... could never have had a proof attempt presented without the mathematicians setting the stage and working to verify the proof.
I think you might be viewing AI and humanity as a zero sum experiment. It's really not. Go read some David Brin (Existence is a good start). We don't know all the positive and negative aspects to come, but we aren't in a dark forest situation. Existentially, AI is here, what are we going to do with it?
I see sentiment like this (which is valid; it’s a different perspective), and then I look at my company’s current caseload of breaches and how most of the (insane) increase in business we’ve received is caused by poorly coded apps with obvious security issues, along with the inability of orgs to remediate those issues or adequately follow incidents because no one actually knows the applications anymore.
And I’m wondering if this isn’t the enormous amount of organizational debt from having security second to everything finally coming calling.
I’ll give you an example of one that was written recently actually.
The initial compromise happened because the app explicitly did not verify auth claims when a specific string was in the ISS field. Well, fuzzers exist and are common.
The next issue was that once you’re in, there was no delineation between admin and regular users. Everyone had all privileges if they just made the calls.
Anyway, we did the usual post-remediation investigation and write up. The devs were of course using the latest models, as they were instructed, and the issue stemmed from a problem they’d been having integrating a specific company into their auth scheme.
Eventually, after many enumerations, the model opted to just skip auth altogether if that companies ISS was present. The devs, being in the habit of just accepting the changes did so and because of the nature of the code implemented nothing caught it in the pipeline.
This is sadly an incredibly common story and it won’t be fixed by models improving I don’t believe.
Not in security, but "vibe coding remediation"--that's the best way I can think to describe it--has been a booming business in my line of work (SIP/VoIP infrastructure engineering) as well.
How long it'll last, I can't say, only that before LLMs, the industry entered a phase of consolidation and quiescence that had me seriously worried about a fade into irrelevance. The delusions of businesspeople and the managerial class about what LLMs can do has fueled a very nice business renaissance, as they run into the limits but have committed to contracts, business models, etc.
I agree, and the field possibly not existing in 5 years is a big issue. Like, how are young people supposed to take a mortgage, start a family, or any other big commitments like that, if their whole education and work experience may be useless in a couple of years?
I don't care whether I'm writing code or reviewing LLM generated stuff, but the prospect of losing my job and having to survive on welfare for the rest of my life isn't nice.
From my perspective this is the thing that the people who have been talking about how good these damn things have gotten now were trying to get you to do all along. Understand that they're getting better, it doesn't seem to be stopping, and we really, really need to figure this part out ASAP.
What seems to happen though is it always gets pulled into a discussion of "But they can't X" or "but ma taste!" or the old generic canard "can't replicate what's not in the training data"
All I want is for people to see that yes, this is happening, accept it, then figure out what a good response would be to it. Instead we get everything from stochastic parrot parrots to "Dario is just marketing when he tries to warn us" to the old an thoughtless "But if you think it's bad, why are you doing it?"
Taxing profits of companies that use AI probably. I really don't see other way, if AI starts replacing human work in large enough scale. In that situation, without welfare there will be soon nobody buying products and services, leading to an economic collapse.
That being said, I can't see this entire field existing in five years anymore. I'm hoping for at least two more years, but who knows?
This stuff is coming for all white collar, the barrier to entry is completely gone now. Maybe not the barrier to mastery (yet), but the bottom has fallen out.
I think you might be viewing AI and humanity as a zero sum experiment. It's really not. Go read some David Brin (Existence is a good start). We don't know all the positive and negative aspects to come, but we aren't in a dark forest situation. Existentially, AI is here, what are we going to do with it?
And I’m wondering if this isn’t the enormous amount of organizational debt from having security second to everything finally coming calling.
I’ll give you an example of one that was written recently actually.
The initial compromise happened because the app explicitly did not verify auth claims when a specific string was in the ISS field. Well, fuzzers exist and are common.
The next issue was that once you’re in, there was no delineation between admin and regular users. Everyone had all privileges if they just made the calls.
Anyway, we did the usual post-remediation investigation and write up. The devs were of course using the latest models, as they were instructed, and the issue stemmed from a problem they’d been having integrating a specific company into their auth scheme.
Eventually, after many enumerations, the model opted to just skip auth altogether if that companies ISS was present. The devs, being in the habit of just accepting the changes did so and because of the nature of the code implemented nothing caught it in the pipeline.
This is sadly an incredibly common story and it won’t be fixed by models improving I don’t believe.
How long it'll last, I can't say, only that before LLMs, the industry entered a phase of consolidation and quiescence that had me seriously worried about a fade into irrelevance. The delusions of businesspeople and the managerial class about what LLMs can do has fueled a very nice business renaissance, as they run into the limits but have committed to contracts, business models, etc.
I don't care whether I'm writing code or reviewing LLM generated stuff, but the prospect of losing my job and having to survive on welfare for the rest of my life isn't nice.
What seems to happen though is it always gets pulled into a discussion of "But they can't X" or "but ma taste!" or the old generic canard "can't replicate what's not in the training data"
All I want is for people to see that yes, this is happening, accept it, then figure out what a good response would be to it. Instead we get everything from stochastic parrot parrots to "Dario is just marketing when he tries to warn us" to the old an thoughtless "But if you think it's bad, why are you doing it?"
Please.
Change doesn't happen until we are at the absolute brink of IMMEDIATE catastrophe.
And yeah, I agree, there is no bottom anymore.