[Question] - Is there a space for quasi-acceptance test in CD?

[santunioni]

I want to understand you signatories thoughts on something I am calling "quasi-acceptance tests".

Let me describe a scenario: I work in a company where a few bugs in production from time to time is not a big problem because:

• We revert fast.
• State corruption is not a problem because almost all data are recoverable by the systems themselves (self-healing), except a few dozens. Those few dozens without self healing rely on backups we perform periodically on our databases.

Tests are critical to us to provide fast feedback on our changes. However, we don't write Acceptance Tests in the original definition that they run on production-like environments. I think we can agree these tests are just costlier and slower than others that evaluate the system running locally on the developer machine or the deployment pipeline runner.

I still find acceptance tests useful for

• self documenting the system
• using ATDD to make sure we spot our wrong assumptions about what the code should do early in the development process
• testing functionality as a whole instead of pieces of code

I am calling the tests I am writing "quasi-acceptance" tests because they are written almost as business criteria acceptance tests, in the sense that they describe the business functionalities the code is implementing. They are not acceptance tests because they don't run in production-like environments. The advantage is that they run very fast, just like small tests, allowing us to run them on commit stage.

I want to known and understand the signatories thoughts and opinions about these "quasi-acceptance" tests. Can you help me scrutinize this, please?

[bdfinst]

The acceptance tests that run after CI are everything else needed to certify the artifact meets the definition of “releasable.”
That could be performance tests, load tests, compliance validation, additional security tests, etc. being able to rollback quickly is good, but adding a test to the pipeline to prevent the issue that occurred in the future is better.

[bdfinst]

I'm re-reading this question. What you're describing sounds like acceptance test-driven development. I highly recommend it. Since they run pre-commit and in the CI portion of the pipeline, I'd still consider them part of CI. Names can be tricky. :)

I typically test this way.