> Now AI coding agents write tests for everything they touch, so tests come for free. And now that I finally have all of them, I can see that most are pointless. They need constant updates, and they slow everything down.
That does not mean tests are pointless. It means your prompts result on pointless tests. And that you are not removing pointless tests before commits nor forcing AI to make meaningful tests.
This should be titled "I don't know how to write good tests, so I consider them pointless".
> They need constant updates,
Good tests don't. They test one specific thing and only change when you change that specific behavior.
> they slow everything down.
They speed everything up, because otherwise you need to test your entire app every time you make a change. A good suite of tests gives you high confidence a change doesn't regress anything and means you can review, submit, and deploy it much faster.
I work on a large product, and I specifically own the framework that generates and powers our UI, API, and backend all from one set of config files. I almost never need to modify tests when adding features - the tests are resilient and I simply write new tests for the new feature.
> I have worked as a freelancer on client projects. Almost none of my clients had a good test suite
The kind of projects that have to hire freelancers might not have the best codebase, news at 11.
> This should be titled "I don't know how to write good tests, so I consider them pointless"
Yes. An LLM certainly doesn't know this. Most of the input data to the LLM will not be "good tests" as these are quite rare in practice. The median test in the input data is not good, so the output of a LLM generating more tests won't be either. Most human oversight won't even recognise this either.
> only change when you change that specific behaviour.
Correct, as Kent Beck observed: "Tests should be coupled to the behaviour of code and decoupled from the structure of code"
bad tests "need constant updates" because they reverse this.
I came here to say something very similar - glad to know that there are other level-headed engineers out there.
As you've pointed out, the problem is literally a skill issue - the OP never really "got into" testing, they don't understand the usefulness, or what makes a test good or bad. This is like hiring a contractor, telling them vaguely "build me a house" and then complaining afterwards that the house they built isn't what you wanted. If you give ai no feedback, you'll get code (and tests) which are the median of what's publically visible in your field. If you want good results, you have to be prepared to teach the agent what you want, which includes feedback about how things should be done, and, ideally, _good examples in your code-base_. So I'd say it's easier for someone without testing fundamentals to get into the "tests are bad" state - they don't know what they did wrong, they don't know how to fix it, and they (boldly, and incorrectly) assume that the ai _does_ know. It knows _nothing_. It's a tool, and you'll spend at least some of your time correcting it.
I couldn't imagine feeling any sense of security without a healthy unit test suite. In particular, when adding a new feature, or updating something somewhere else, having some kind of automation that can tell me I didn't break stuff that wasn't broken before is invaluable.
Where the OP does have a point (albeit by implication) is that:
WHEN TESTS RUN SLOWLY, EVENTUALLY, NO-ONE RUNS THEM.
When I joined the company I'm at now, they had about 2k tests on their main product - they couldn't be run reliably as a suite, so no-one did. They were indeed pointless. Now we have 15k tests on that project, and they run on every push, and before every deployment. The reason? My primary task was getting testing working reliably (which also included trying to get other devs to opt into proper testing - which had variable results - some people will push against what they see as "more work" for "no purpose" simply because they don't understand the purpose. The point is: now that they all run within about 5-10 minutes (depending on the host machine), they're run all the time, and they provide useful feedback when dev in one area has unintended side-effects in another area.
also, what really changes the game is requiring that one write the tests _first_. Generally, I haven't had to do this with an ai - but it's an interesting idea (get it to write the test(s) first, and vet them - like I would do in real life - except only one test at a time, then the red-green-refactor dance). Writing the test first forces you to narrow down what you want to accomplish and makes you think about how, such that the prod code runs well and tests don't run like treacle.
Anyhoo, 'nuff said: my opinion is that the OP should "git gud" or accept that they will never reap the benefits of a good test suite.
That does not mean tests are pointless. It means your prompts result on pointless tests. And that you are not removing pointless tests before commits nor forcing AI to make meaningful tests.
> They need constant updates,
Good tests don't. They test one specific thing and only change when you change that specific behavior.
> they slow everything down.
They speed everything up, because otherwise you need to test your entire app every time you make a change. A good suite of tests gives you high confidence a change doesn't regress anything and means you can review, submit, and deploy it much faster.
I work on a large product, and I specifically own the framework that generates and powers our UI, API, and backend all from one set of config files. I almost never need to modify tests when adding features - the tests are resilient and I simply write new tests for the new feature.
> I have worked as a freelancer on client projects. Almost none of my clients had a good test suite
The kind of projects that have to hire freelancers might not have the best codebase, news at 11.
Yes. An LLM certainly doesn't know this. Most of the input data to the LLM will not be "good tests" as these are quite rare in practice. The median test in the input data is not good, so the output of a LLM generating more tests won't be either. Most human oversight won't even recognise this either.
> only change when you change that specific behaviour.
Correct, as Kent Beck observed: "Tests should be coupled to the behaviour of code and decoupled from the structure of code"
bad tests "need constant updates" because they reverse this.
As you've pointed out, the problem is literally a skill issue - the OP never really "got into" testing, they don't understand the usefulness, or what makes a test good or bad. This is like hiring a contractor, telling them vaguely "build me a house" and then complaining afterwards that the house they built isn't what you wanted. If you give ai no feedback, you'll get code (and tests) which are the median of what's publically visible in your field. If you want good results, you have to be prepared to teach the agent what you want, which includes feedback about how things should be done, and, ideally, _good examples in your code-base_. So I'd say it's easier for someone without testing fundamentals to get into the "tests are bad" state - they don't know what they did wrong, they don't know how to fix it, and they (boldly, and incorrectly) assume that the ai _does_ know. It knows _nothing_. It's a tool, and you'll spend at least some of your time correcting it.
I couldn't imagine feeling any sense of security without a healthy unit test suite. In particular, when adding a new feature, or updating something somewhere else, having some kind of automation that can tell me I didn't break stuff that wasn't broken before is invaluable.
Where the OP does have a point (albeit by implication) is that: WHEN TESTS RUN SLOWLY, EVENTUALLY, NO-ONE RUNS THEM.
When I joined the company I'm at now, they had about 2k tests on their main product - they couldn't be run reliably as a suite, so no-one did. They were indeed pointless. Now we have 15k tests on that project, and they run on every push, and before every deployment. The reason? My primary task was getting testing working reliably (which also included trying to get other devs to opt into proper testing - which had variable results - some people will push against what they see as "more work" for "no purpose" simply because they don't understand the purpose. The point is: now that they all run within about 5-10 minutes (depending on the host machine), they're run all the time, and they provide useful feedback when dev in one area has unintended side-effects in another area.
Anyhoo, 'nuff said: my opinion is that the OP should "git gud" or accept that they will never reap the benefits of a good test suite.