Tests: Improve isolation through resource teardown by closing/disposing connections etc.

[amotl]

About

This patch has been broken out of GH-284 in the same spirit like GH-283, supporting to run combinations of individual test cases without using xdist.

Our evaluations are that it feels like software tests have not been explicitly isolated yet, so xdist came to the rescue for you, because it implicitly isolates test cases by distributing them amongst worker processes?

Background

Most prominently, adjustments in the area of DuckDB connection handling get rid of this error.

Connection Error: Can't open a connection to same database file with a different configuration than existing connections

dlt.destinations.exceptions.DestinationConnectionError: Connection with DuckDbSqlClient to dataset name 
foo failed. Please check if you configured the credentials at all and provided the right credentials values.
You can be also denied access or your internet connection may be down. The actual reason given is:
Connection Error: Can't open a connection to same database file with a different configuration than existing connections

-- https://github.com/bruin-data/ingestr/actions/runs/16102670949/job/45433825506#step:11:607

Notes

[...] without using xdist.

On the workbench, we are running individual software tests like this to speed up cycle time by avoiding xdist startup overhead, that's why GH-283 came up.

export TESTCONTAINERS_RYUK_DISABLED=true

time pytest -vvv -k "(replace or append) and (duck and postgres)"
time pytest -vvv -k "(duck and postgres)"

time pytest -vvv -k "(dynamo and postgres) or (gcs and clickhouse)"
time pytest -vvv -k "cratedb and not sqlserver"