Add v2.2 test suites#53
Draft
karlnaden wants to merge 44 commits into
Draft
Conversation
* WIP - reorganize * WIP reorg - loading * WIP - clean rspec and rubocop * missed spec test * update scripts
* WIP: v220 test suites * fix specs * thread ig version through tests that involve profile conformance checks, add initial 220 profiles * rubocop fixes * clean v2.0.1 requirements from 2.2.0 tests until they can be re-mapped * split out discovery responses for 2.0.1 and 2.2.0 (no changes for 2.2.0 yet) * add spec test * separate discovery details and update preset to as close to passing as possible
* WIP 220 prefetch checks for clients * appointment book example * use published 2.2.0 ig * update prefetch tests for 2.2 * update 2.2 complete prefetch examples for future use in execution scripts * fail on extra prefetch key * update to core 1.1.1 and add logicla model validation of requests * update complete prefetch examples to conform to models * organize hook group * add required selections * rubocop fix * point to 2.2.1 IG * cors fix * cors again * update gemfile.lock * handle inability to fetch * update groupings to match 2.2.0 client * fix requirements and spec tests
* WIP bundle-based FHIR server * server test hook invocations during a wait to support fhir requests * include and revinclude support, defaulting, updates fix, spec tests * requests made by inferno's simulated client use the simulated EHR endpoint * bundle endpoints together add metadata endpoint allow source metadata files to be determined at include time * rubocop * add crd-specific metadata, explicit resource type when searching * fix spec tests
* import us core tests with 3 version options, default inputs from cds hook request fields * typo * update preset
…extension (#58) * advertise and support multiple versions on the 2.2 client suite simulated service * add requested version check and remove unused tests * update presets * add spec tests
* Implement coverage-info configuration option to suppress specific cards and system actions in hook responses * Centralize coverage info configuration logic and update test runner to support disabled configuration requests * Prevent redundant coverage-info follow-up requests in InvokeHook job and add verification test
* skip if no responses rather than requests for response attestation * fix order dispatch mocking for v2.2.1 * move request logical model validation into shared location, add spec test for client request verification test * checks for ids in context resources on requests * fixes * rename prefetch checker * factor out pretch profile checking to cross_suite, add resource id check * update server 2.2.1 preset request bodies for order-sign and order-dispatch * update validator version * preset typo
* requirements planning tweak * fetch coverage payor during hook processing * capture Inferno organization id during registration * check that the coverage payor id matches what indicated in registration and matches the CRD Organization profile * check that additional data could be fetched during hook invocations * add new tests to hook group * update presets
* client test to check granted scopes * add requested scopes to registration attestation * add new test to hook groups * rubocop fixes * update server preset to use us core 3.1.1 scopes * typos * typo
* message and load tags helpers * tls test and add to hook groups * refactor to use helper modules and update spec tests
* delete unused api tests * tagged request loader handles multiple hooks * fix file name * cov-info update test
* add check for davinci-crd.version discovery extension validation * add spec for davinci-crd.version extension validation * check type of crd version extension array entries * update version regex * fix regex escaping Co-authored-by: Tom Strassner <[email protected]> --------- Co-authored-by: Tom Strassner <[email protected]>
* clarify variable name * rename group * fetch locations during hook calls, update to use tags * check location address propagation * minor fixes and spec coverage
* catch know fhirpath engine error case before execution * prefetch checker finds cases * prefetch completeness test outputs a flag when behavior observed * cross suite test checks for the completeness test flag * add fhirpath service error * fix rspec and rubocop * error clarification * error clarification
…equests (#75) * refactor hook endpoint and receive tests to use just the jwt iss as the wait id for the session * revamp error handling and hook detection * make sure output indexes line up, clean up auth verification code, and remove dependency on configuration for auth tests * move registration tests into their own folder * clean hook group renamed inputs * long-running request test group * documentation fixes * keep the existing order
* refactored data fetch tags to support separation from hook invocations * add uniqueness check test * rubocop
* cds hooks requirement mapping for client v 2.2.1 * crd client v221 requirements mapping * add us core sets, generate coverage * fix added requirements
… are intended to be CRD-conformant (#67) * Add CRD discovery service ignore list * Skip if all cds services ignored. * Ensure stale services cleared if discovery empty/ignore all services.
* remove group and tests for required response types across hooks * add test to verify that responses always include coverage information for primary hooks * add spec for all responses include coverage info test * add special handling for order dispatch coverage info check * ignore requests which have disabled coverage info responses * handle order resources in the prefetch for order dispatch * clean up code * restore test for coverage information across all hooks * check for coverage info extension on all order resources
…#81) * initial updates to support a second cds service endpoint * must_support -> cross_hook * add test for subset prefetch endpoints * additional response template filter for endpoint paths * add requirements mapping * rename subset service endpoint * handle alternate prefetch keys * add second organization id for subset services * handle absolute urls and update fhirpath logic to support bundle resolution rules * updated check for distinct complete vs subset prefetch provision * code review fixes * id typo * typo * updated presets and corrected subset service declarations * fix prefetch data set comparison * update error messages to use the full url * typo * update requirements coverage
* add discovery test for valid configuration options * add spec for discovery valid configuration options test * ensure coverage info config test looks at a request which would produce a coverage info response * omit instead of skip for secondary hooks with no coverage information responses * update discovery test description * check for duplicate values in configuration options * verify config options for all hook services * incorporate service ignore list into discovery configuration test * fix error message Co-authored-by: Tom Strassner <[email protected]> * remove unused method --------- Co-authored-by: Tom Strassner <[email protected]>
* map requirements * update server coverage file * update requirements based on PR feedback * add comment about typo in hook-39 * regenerate coverage csv
* sync complete vs subset prefetch differences * refactor discovery URLs * documentation updates * updated suite documentation * normalize around "client" instead of "Client" unless capitalized for another reason * normalize to CRD server, unless capitalized for other reasons * services -> server * test name and description updates * spec fixes * udpate requirements coverage * additional test andgroup documentation tweaks * input documentation tweaks * wait dialog updates * wiki tweaks for consistency * error message review and other typos * rubocop fix
* Add must support test for coverage info extension * Move coverage-info metadata to shared location, improve spec
* add checks for davinci-crd.if-none-exist extension on questionnaire creation actions * add specs for verifying the presence of davinci-crd.if-none-exist extension * fix linting errors * change if-none-exist from array to string
* clean up request validation using logical models, including adding CRD profile checks * Update client vs server presets to have passing requests * update requirements mapping * fixes found during testing
* recreate coverage information cross suite test for primary hooks * moved to the response handling subgroup where it belongs * update requirements mapping
* map requirements * regenerate coverage csv * fix all responses contain coverage info requirements * check type of coverage info config option * regenerate coverage csv * fix requirement * regenerate coverage csv
…he coverage-information extension (#83) * Verify add or update coverage-info only * Extract hook request resource resolution module * Regenerate requirements coverage
* add test to verify server does not require billing options extension * add spec for billing options test * update group title/description * add requirement mapping * regenerate coverage csv * fix purple error in cross-hook test
* update invoke hook job to send a request with unknown configuration * update invoke hook spec to check for unknown configuration request * wip * add unknown configuration test to hook groups * add spec for unknown configuration test * remove unused method * add spec for unknown configuration test * add request with unknown context to invoke hook job * update invoke hook spec to check for unknown context request * WIP * add test for unknown context * add spec for unknown context test * update invoke hook to send a request with an unknown cds hooks element * update invoke hook specs to account for unknown cds hooks element call * add test for unknown cds hooks elements * add spec for known cds hooks elements test * update language in unknown element tests * add requirements * update requirements coverage * document expected client test failures * update expected failures docs
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR combines a large set of updates that together add test suites covering the v2.2.1 version of the CRD IG, including a client and server suite. In order to support these updates, a significant re-organization of the code base to cleanly separate code for client suites, server suites, and shared code (both between actors and within actor suites) - see the Code Organization section of the wiki for details. Otherwise, the updates for each suite are described within the contributing pull requests listed below.
PRs for which key design decisions were made are also flagged in the lists below for potential additional review.
Client v2.2.1 suite PRs
Server v2.2.1 suite PRs
Testing Guidance
Key areas of focus: