This is one (of several) reason why many larger open source project ban or severely restrict the usage of LLMs. They want contributors to fully understand the changes they are making.
Recently we have received several bug reports with AI suggested fixes. They looked reasonable enough and seemed to resolve the issue at hand. I then took a deeper look and discovered that every single suggestion only hid the symptom but did not address the underlying issue. Applying the suggested fixes would have hurt our codebase, so I eventually had to reject all of them and write my own solutions. This was only possible because I'm deeply familiar with the codebase and the broader context. Someone with less knowledge would have probably just applied the LLM suggestions without much questioning. I find this rather worrying.
I guess in popular open source projects the stakes are just higher than in some proprietory software that will be replaced in a few years anyway. The irony, of course, is that LLMs have mostly been trained on open source code.
Also, some of the projects I'm involved in are over 20 years old. One even celebrated it's 30th anniversary. This means there is also the question of historical responsibility.
Recently we have received several bug reports with AI suggested fixes. They looked reasonable enough and seemed to resolve the issue at hand. I then took a deeper look and discovered that every single suggestion only hid the symptom but did not address the underlying issue. Applying the suggested fixes would have hurt our codebase, so I eventually had to reject all of them and write my own solutions. This was only possible because I'm deeply familiar with the codebase and the broader context. Someone with less knowledge would have probably just applied the LLM suggestions without much questioning. I find this rather worrying.
I guess in popular open source projects the stakes are just higher than in some proprietory software that will be replaced in a few years anyway. The irony, of course, is that LLMs have mostly been trained on open source code.
Also, some of the projects I'm involved in are over 20 years old. One even celebrated it's 30th anniversary. This means there is also the question of historical responsibility.