This seems neat in a couple of ways. It’s a lot like fuzzing or property-based testing, but those approaches usually target crashes or try to catch functional bugs. This feels like a way to do fuzzing for performance or other kinds of quality. The idea is straightforward, but it’s impressive to me that this can be practically implemented and used at the scale of real languages and compilers.
As a vaguely related similar idea, once upon a time at a hardware company we had a regression suite that failed if our chip was less performant than a competitor’s. But that test suite was largely hand crafted. The ability to generate these cases randomly is super cool.
This seems neat in a couple of ways. It’s a lot like fuzzing or property-based testing, but those approaches usually target crashes or try to catch functional bugs. This feels like a way to do fuzzing for performance or other kinds of quality. The idea is straightforward, but it’s impressive to me that this can be practically implemented and used at the scale of real languages and compilers.
As a vaguely related similar idea, once upon a time at a hardware company we had a regression suite that failed if our chip was less performant than a competitor’s. But that test suite was largely hand crafted. The ability to generate these cases randomly is super cool.