One other thing which I find very useful is to write tests which will start failing once the <trigger condition> changes. For example if today the database version I'm testing against disallows some query shape I can:
- write a test that verifies the failure
- add a todo comment in code linking to the test to fix the code when test fails
Then eventually the test will start failing, you can fix the code and the test and profit.
Good point. But that works only if the version number is the thing you want to test. And also it's possible that newer versions include the bug or some even newer version introduces a regression re-introducing the bug.
I'm a believer of user-facing testing - i.e. test things like how a user would operate them. So in the context of building a federated query engine our users won't care what version of Postgres they are connecting to - they will care however that the query they wrote doesn't work - doesn't matter what Postgres version.
- write a test that verifies the failure
- add a todo comment in code linking to the test to fix the code when test fails
Then eventually the test will start failing, you can fix the code and the test and profit.