Funny how GIMP spawned the Gimp Toolkit, aka GTK, which is widely used in Linux, and yet GIMP's own UI seems to have long languished as a rather outdated design. Glad that they have tests, though, and hope they continue to improve it.
> We have 13 tests at this time. They check that: GIMP starts and shows its main window. The Welcome dialog opens on the correct tab. It also stays hidden when you turn off “Show on Start”.
That's great you guys. Have you tested whether anyone can actually figure out the UI once they get into the app though?
There is a reason ArtCraft took off like a rocketship and GIMP has struggled to even retain users who try to stick with it.
Low effort snide but sometimes there's really no other way to convey the feedback...
They made a blog post to announce that they’d written thirteen unit tests? Sure, there’s value in handmade software and all that but GIMP is not really a particularly excellent piece of software - if anything was to benefit from unleashing the agents it would be GIMP.
Does GIMP have a very restrictive AI policy or something? I get not wanting to ship a gigantic pile of UI tests without thorough review, but basically any model released in the last six months would be able to write UI tests covering every single control in an afternoon.
It’s frustrating to create tests with models, not sure if it’s everyone’s experience, but every time I asked it to create tests, it makes the tests conform to the sometimes broken code. Rather than calling out what the logic/code is wrong, it will make the tests conform to whatever is already broken.
It can’t magically know expectations. If the code is well documented or there’s decent associated documentation, I’ve found it has no issue writing tests.
Not using AI to at least writes tests in this day and age is crazy work. I started a desktop application 2 months ago and it has almost 3,000 tests now, MANY of them UI tests.
AI is a bit too excellent at writing tests. The number of tests doesn't really convey any information on test coverage. At least I've found that agents that naively improve codebases will write tests for pretty much anything, including irrelevant behavior that has little value in ensuring stability, security or correct behavior. It would probably help to run a test review stage with a strong model, where agents attempt to correct and consolidate tests to guard behavior rather than ensuring that theoretical code paths are followed.
That's great you guys. Have you tested whether anyone can actually figure out the UI once they get into the app though?
There is a reason ArtCraft took off like a rocketship and GIMP has struggled to even retain users who try to stick with it.
Low effort snide but sometimes there's really no other way to convey the feedback...
Does GIMP have a very restrictive AI policy or something? I get not wanting to ship a gigantic pile of UI tests without thorough review, but basically any model released in the last six months would be able to write UI tests covering every single control in an afternoon.
Prod your agent first until it can explain back to you what's wrong with the tests.
Then ask your agent to write a better prompt for writing tests (and save it to an .md file)
Then reference that .md file and iterate on the tests again. It may take a couple of tries, but I've had good results with this strategy.
Ended up with a whole prompting framework.