feat(glue-alpha): wrap DataQualityRuleset DQDL in a typed value object - #38587
Conversation
The ruleset's primary payload was a bare `rulesetDqdl: string` passed straight to
the L1 with no type identity. Introduce a `Dqdl` value object built via
`Dqdl.fromString(...)` and take it as the `dqdl` prop, so the DQDL document is a
named, discoverable type with a clear escape hatch rather than an anonymous
string. Glue still parses and validates the DQDL at deploy time; the rendered
string is unchanged, so synthesized output is identical.
This is the minimal (escape-hatch-only) shape; modeling individual rules as typed
objects was considered and deliberately deferred — the service is the source of
truth for DQDL validation and the language is large and evolving, so `fromString`
gives full coverage today.
Addresses the DataQualityRuleset.rulesetDqdl finding from the aws-glue-alpha
pre-GA API review.
BREAKING CHANGE: `DataQualityRulesetProps.rulesetDqdl: string` is replaced by
`dqdl: Dqdl`. Build it with `Dqdl.fromString('Rules = [ ... ]')`.
otaviomacedo
added
the
pr-linter/exempt-integ-test
|
PRs without a linked issue will receive lower priority for review and merging. Please update the description to follow the PR template and include a line like |
The ruleset's primary payload was a bare `rulesetDqdl: string` passed straight to
the L1 with no type identity. Introduce a `Dqdl` value object built via
`Dqdl.fromString(...)` and take it as the `dqdl` prop, so the DQDL document is a
named, discoverable type with a clear escape hatch rather than an anonymous
string. Glue still parses and validates the DQDL at deploy time; the rendered
string is unchanged, so synthesized output is identical.
This is the minimal (escape-hatch-only) shape; modeling individual rules as typed
objects was considered and deliberately deferred — the service is the source of
truth for DQDL validation and the language is large and evolving, so `fromString`
gives full coverage today.
Addresses the DataQualityRuleset.rulesetDqdl finding from the aws-glue-alpha
pre-GA API review.
BREAKING CHANGE: `DataQualityRulesetProps.rulesetDqdl: string` is replaced by
`dqdl: Dqdl`. Build it with `Dqdl.fromString('Rules = [ ... ]')`.
✅ Updated pull request passes all PRLinter validations. Dismissing previous PRLinter review.
aws-cdk-automation
added
the
pr/needs-maintainer-review
GitHub Actions
Failure
| * | ||
| * @param dqdl the DQDL document, e.g. `Rules = [ RowCount > 100 ]`. | ||
| */ | ||
| public static fromString(dqdl: string): Dqdl { |
There was a problem hiding this comment.
In the future, we can add more factories, for more structured inputs, like a list of rules (of a possible Rule type).
There was a problem hiding this comment.
The future extensibility is a good option but did we have any specific request or reason to do this type wrapping ? Are the ddql rule based wrappers coming up in next set of PRs ?(Or even ddql query validation checks)
There was a problem hiding this comment.
If we don't make it type safe now, we won't be able to do it after graduation (other than by deprecating it and adding another type safe prop). This is in line with the rest of the library. For example, Schedule, that you can construct with cron elements, or a rate, but there is always the string escape hatch. A closer analogue is probably StepFunction's ASL, for which we have also created a type safe API around.
I'm not planning on creating the Rule type for now, because it takes some care to get right. Here's the syntax so you have an idea. But I think it's worth doing it, eventually.
|
Thank you for contributing! Your pull request will be updated from main and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork). |
Merge Queue Status
This pull request spent 1 hour 50 minutes 33 seconds in the queue, including 56 minutes 59 seconds running CI. Required conditions to merge
|
aws-cdk-automation
removed
the
pr/needs-maintainer-review
|
Thank you for contributing! Your pull request will be updated from main and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork). |
|
Thank you for contributing! Your pull request will be updated from main and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork). |
|
Comments on closed issues and PRs are hard for our team to see. |
3 participants
The ruleset's primary payload was a bare
rulesetDqdl: stringpassed straight to the L1 with no type identity. Introduce aDqdlvalue object built viaDqdl.fromString(...)and take it as thedqdlprop, so the DQDL document is a named, discoverable type with a clear escape hatch rather than an anonymous string. Glue still parses and validates the DQDL at deploy time; the rendered string is unchanged, so synthesized output is identical.This is the minimal (escape-hatch-only) shape; modeling individual rules as typed objects was considered and deliberately deferred — the service is the source of truth for DQDL validation and the language is large and evolving, so
fromStringgives full coverage today.Addresses the DataQualityRuleset.rulesetDqdl finding from the aws-glue-alpha pre-GA API review.
BREAKING CHANGE:
DataQualityRulesetProps.rulesetDqdl: stringis replaced bydqdl: Dqdl. Build it withDqdl.fromString('Rules = [ ... ]').By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license