Job resource and data source data structure parity with dbt cloud API

[adrianburusdbt]

Currently, there are a number of mis-matches between the provider's data structures and the dbt cloud API's in the job resource and data source.
Some of them are:

• execution block is present in job data source and API, but not present in resource, instead a timeout_seconds field exists (which is also in execution)
• retrieving all jobs is not consistent with other data sources (e.g. having a XXX_All data source), instead there is a SingleJob model
• there are additional blocks not consistent between the data source, resource and API, like settings, environment, etc.

A thorough investigation of the API needs to be done and the data structure must be changed to match what we have there.
Also, these will be breaking changes, so ideally we would address all of them in one release.

Draft PR with a start of implementation work: #500

Here are some linked issues to the above:

• dbtcloud_job.timeout_seconds returns misleading warnings #475

[b-per]

@adrianburusdbt I'd expect the dbtlcoud_job resource to be the most used out there (at least I think that the community member who created the provider initially mostly wanted to manage jobs)

Do we know what would be the impact for existing TF projects/modules?
Is there a way to implement this without introducing any breaking change to the config as it is today?

[adrianburusdbt]

I think the only way to be backwards compatible is to just keep the old fields, as well as add the new ones. It's basically what we have today on the data source for job. This is not great and we need to ultimately move forward at one point.
For now, the changes are shelved, I think this needs wider investment to wrap it up in one go, and minimize the hard updates users will need to do.

[github-actions]

This issue has been marked as Stale because it has been open for 90 days with no activity. If you would like the issue to remain open, please comment on the issue or else it will be closed in 7 days.