Skip to content

Conversation

SFJohnson24
Copy link
Collaborator

@SFJohnson24 SFJohnson24 commented Aug 15, 2025

This PR adds data structure scoping for ADaM as well as normalized the input for ADaM products input by users in both rule editor and engine.

Product Version
adamig 1-0
adamig 1-1
adamig 1-2
adamig 1-3
adam-adae 1-0
adam-md 1-0
adam-nca 1-0
occds 1-0
adam-tte 1-0
adam-poppk 1-0

are now the entered standards in the library tab. also for a CLI validation -s adamig -v 1-3 works now. The parsing under the hood allows for an easier entry by users and parses it to what is expected by library (standard is adam, version is adamig-1-3 for the above case). This makes it easier for the user to enter the IG for adam, or other adam products.

this also has the parsing logic for data structures and uses the library metadata from adam to select datasets to run.

This pull request introduces support for ADaM standards normalization throughout the codebase, adds data structure-based rule applicability, and improves cache key handling. The main changes are the addition of ADaM product normalization logic, integration of data structure checks for rule applicability, and updates to cache key generation to ensure consistent internal format for ADaM-related inputs.

ADaM Standard Normalization and Support:

  • Added the normalize_adam_input utility function to standardize ADaM user inputs, and a list of supported ADaM products in ADAM_PRODUCTS. This function is now used throughout the codebase to ensure consistent handling of ADaM standards and versions. (Fd48126eL169R169, cdisc_rules_engine/constants/adam_products.pyR1-R9)
  • Updated cache key generation functions (get_standard_codelist_cache_key, get_library_variables_metadata_cache_key, get_standard_details_cache_key, get_variable_codelist_map_cache_key) to use normalized ADaM inputs, preventing cache misses due to inconsistent formats. (Fd48126eL69R69, Fd48126eL158R158, Fd48126eL169R169, Fd48126eL347R347)

Rule Applicability by Data Structure:

  • Added logic to check if a rule applies to a dataset's data structure, including new methods in rule_processor.py and updates to is_suitable_for_validation. This ensures rules are skipped if they do not apply to the dataset's data structure. (cdisc_rules_engine/utilities/rule_processor.pyR191-R223, F52bf509L643R643)
  • Extended the rule metadata model to include a data_structures field, allowing rules to specify applicable data structures.
  • Implemented get_data_structure in the data service interface and base class, enabling retrieval of a dataset's data structure for rule applicability checks. [1] [2]

General Improvements and Minor Fixes:

  • Updated .flake8 configuration to ignore E713 for more flexible linting.
  • Improved logging in validation and reporting scripts for better traceability. (scripts/run_validation.pyR185, F8363ee2L193R193)
  • Minor refactoring and consistency fixes in error handling and operation execution methods. (cdisc_rules_engine/rules_engine.pyL179-R179, F5f2fcd6L459R459, F52bf509L39R39, F52bf509L442R442)

These changes collectively improve the robustness and extensibility of rule validation, especially for ADaM datasets and standards.

To test: see original issue ticket/rule 47 for adam.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant