Feature/rsu info migration#68
Closed
drewjj wants to merge 60 commits into
Closed
Conversation
* ui: adjust user admin tab to show only users of signed in organization * ui: adjust intersection admin tab to show only intersections of signed in organization * ui: adjust rsu admin tab to show only rsus of signed in organization * chore: fix spacing * chore: remove unneeded method calls * fix: adjust preloaded data for admin page to update when organization is changed * chore: remove unused import * chore: remove unused import * test: adjust unit test for removal of unnecessary dispatch * refactor: move organization filtering to back-end * chore: remove unused variables * test: update front-end unit tests * chore: adjust delete messages for RSU and User tabs * chore: adjust delete messages for RSU tab * test: adjust unit tests for clarity
* UI: update help page descriptions and images * UI: update help page images * test: adjust unit test for help page UI * Adding markdown help page with new formatting and additional rsu configuration descriptions * Securely removing IP Addresses from images * Update webapp/src/components/help.md Co-authored-by: Drew Johnston <31270488+drewjj@users.noreply.github.com> * Adding help section on permissions * Updating V2x msg viewer case to V2X * ui: adjust formatting and additional descriptions * chore: refactor help page images and add unit test * Re-formatting help page * Adding in-page permissions table * Update Help.test.tsx.snap * ui: remove environment variables from help page and make descriptions clearer for non devs * ui: remove redundant word from help page --------- Co-authored-by: jacob6838 <jfrye@neaeraconsulting.com> Co-authored-by: Jacob Frye <51723791+jacob6838@users.noreply.github.com> Co-authored-by: Drew Johnston <31270488+drewjj@users.noreply.github.com>
Adding default environment variables for all non-required elements, and adding errors for required variables. un-specified non-required variables will produce warnings when relevant. Changes include re-working python api env variable access, rewriting sample.env with only 3 required variables, overhauling run documentation, and updating debug configurations * Moving Dockerfile to match other services * Adding in required jar dependencies * Re-Adding runtime to Dockerfile * Updating Reference BSM * Switching from Message Frame types to Specific types * Removing Data Loaders from read accessors * Adding in Accessors for Processed BSM messages * Removing unneeded print statements * Adding in Processed BSM MongoDB accessor and Endpoints * Adding PSM support to decoder * Removing ASN.1 Codec Submodule * Fixing bug when decoding capitalized messages * Switching to hosted jar files * Removing SNMP controller - Controller to be added in rsu-status manager instead. * Switching Ode Data objects to OdeMessageFrame * Removing un-needed print statements * Removing print statements * Updating version number * Switching BSM queries to use common bounding box calculation * Removing DataLoader from Events and Assessments * Updating TIM message size allowance * Adding static modifiers to decoder mappers * Moving large strings into files to improve readability * Updating ci/cd scripts to use repo host instead of USDOT * Updating CI to Java version 22 * Updating processed BSM, SPAT, and MAP webapp types * Cleaning up removed OdeBsmData type * Updating webapp intersection map to use processed bsm * Updating webapp asn1 processing for ProcessedBsm * Cleaning up tests * Adding organization-permissions chart to README * Switching to proper bounding box * Adding valid date check for spat visualization * Removing Geo Coordinate reference requirement * Combining orgs_to_add to bulk update queries * Implementing write_db_batched for batched add queries * Updating unit tests for batched queries * Resolving processed-msg unit test merge conflicts * Resolving unused variable warnings * Update StompControllerTest.java * Adding "Update time range" intersection map button * Update control-panel.tsx * Updating env access in python addons * fixing python unit tests * Fixing issues with Serializing page data from intersection-api * Update sample.env * Fixing python api bugs * Adding default values to docker-compose yml files * Updating GITHUB token instructions * Cleaning up unnecessary imports * Updating intersection-api README * Converting map timestamps in intersection map * Update map-slice.tsx * Re-naming sliderValue to sliderValueDeciseconds * Add pull request file change limit check to CI workflow * Enhance CI workflow to log max allowed changed files and validate pull request size * Improve CI workflow logging for changed files count in pull requests * Add cancellation of parallel jobs on failure in CI workflow * chore: revert cancel * Enhance CI workflow to enforce changed files limit before subsequent jobs * Refactor CI workflow to directly use MAX_CHANGED_FILES variable in error message for clarity * Update control-panel.tsx * Fixing python api env var references * Update api_environment.py * Updating python unit tests * Fixing rsu query bugs * Update main.py * Fixing env defaults for addons and webapp * Switching visualizer to pull processed BSMs * Re-generating sample mongo data with processed-msg updates * Resolving unit test errors * Revert "Re-generating sample mongo data with processed-msg updates" This reverts commit d7bce4d. * Reapply "Re-generating sample mongo data with processed-msg updates" This reverts commit 1a0cdba. * Update release notes 2025-Q3 for version 2.0.0 * Update paramaterization * Rename the param variable to be consistent with changes from develop * Fixing timeBeforeSeconds merge conflict * Update userSlice.ts * Switching time API server url * Updating time-server api unit test * Updating map/spat times to be in milliseconds * Updating map/spat times to be in milliseconds * Update map-slice.tsx * Update timeSyncSlice.ts * Removing print statments * Update map-slice.tsx * Update control-panel.tsx * Finalizing sample.env and KEYCLOAK_ENDPOINT default * Create sample.env * Documenting required environment variables * Pulling live spat offset calculation into control-panel * Updating live data streaming cleanup * Fixing rsu organization query bugs * Default spat timestamp in decoder mode when leaving/returning * Clearing sourceData when resetting intersection map view * Update map-slice.tsx * Cleaning up live intersection data start vs stop logic * Adding cancellation of pullInitialData * Temporarily removing live data updates * Adding quick-start section to README * Switching Decoder unit tests to use json comparison in unit tests * Switching live intersection map to use deduplicated map/spat * Re-enabling live data updates * Live intersection map pulling initial spat data * Cleaning up latent request on live data restart * adding abort controllers to live data map/spat * Removing live data offset display * Delaying live data initial SPaT message retrieval * Adding spat message age to live viewer * Tweaking time offset sync to reduce error in response time * Updating timeRenderWindow logic * Update map-slice.tsx * Updating ffm version to usdot 2.0.2 * Updating keycloak sample theme * Adding logging level empty check * Update Dockerfile.intersection-api * Updating slider value on live message received * Adding asn1application binaries and configuring codec confid for linux vs windows loading * Defaulting spat signal group to latest even if outside of render interval * Improving live ws disconnection * Improving live ws disconnection * Working codecconfig asnapplication dll/so loading * DEVELOPMENT ONLY - adding usdot to settings.xml * Fixing spat signal group latest processing * Update sample-full.env * Updating wzdx enabled default * Tweaking live imap slider update call * Adding Mapbox token creating instructions * Upgrading python-keycloak to 5.8.1 to resolve introspect errors * Tweaking connect/reconnect for live imap * Update map-slice.tsx * Rewriting live imap ws client * Fixing new imap live ws connection * Removing random } from aggregation endpoint timestamps * Tweaking live imap reconnect logic * Tweaking cleanupLiveStreaming method * Update map-slice.tsx * Adding reconnect on webDocketFactory error * Updating jpo-utils to USDOT develop * Update map-slice.tsx * Increasing wait on WS reconnection attempt * Adding optional variables to sample.env * Fixing default keycloak URL * Adding session ID to live intersection map * Updating README getting started docs * Resolving missing params in test_ping_data_query * Update test_rsu_online_status.py * Tweaking webapp decoder parsing * Re-setting live data active after failed reconnect * Re-pulling map/spat each reconnect * Hard resetting map view on live data active toggle * Removing hard map reset on live data active toggle * Updating ODE bsm json tests for schema * Update test_rsu_online_status.py * Converting ANY auth queries to IN * Update configSlice.ts * Reducing WZDx icon height * Reducing Intersection icon width and label font size * Tweaking rsu_geo_query query formatting * Replaceing get 0 with getFirst * Fixing auth_tools org permissions check parsers * Removing print statement * Fixing auth rsu IP address comparison * Fixing formatting bug in RSU geo query * Final re-formatting of rsu geo fwd query * Removing unnecessary print statements * Removing cvmger_api query logger * Updating DefaultConfig tests * Adding intersection config boolean test * Updating jpo-utils submodule to usdot develop * Removing overrides from iapi settings.xml * Updating jpo-utils to USDOT master * Updating conflictmonitor iapi dep version * Update settings.xml * Update docs/Release_notes.md Co-authored-by: jacob6838 <51723791+jacob6838@users.noreply.github.com> * Update docs/Release_notes.md Co-authored-by: jacob6838 <51723791+jacob6838@users.noreply.github.com> * Update docs/Release_notes.md Co-authored-by: jacob6838 <51723791+jacob6838@users.noreply.github.com> * Update docs/Release_notes.md * Updating default webapp port to 3000 Adding WEBAPP_PORT env var and changing default port to 3000 * Create developer_best_practices.md * Adding submodule init to quick-start section * Adding getting-started link to the quick-start section * Adding fork sync instructions * Updating fork sync PR creation description * Improving mocking in timeSyncSlice test * Update timeSyncSlice.test.ts * Updating KC_LOG_LEVEL env var name * Updating PR length recommendation * Cleaning up unused code and print statements * Fixing docker host ip in sample.env.local * Removing deprecated VSCode virtual environment instructions * Adding additional docker build debugging steps * Updating squash merge resolution to prefer merge * Adding python API .vscode default env vars * Removing unused intersection_no_api docker profile * Cleaning up docker-compose commands and language * Adding localhost:3001/3002 to realm.json * Cleaning up realm.json * Removing unnecessary configure_logging calls * Updating vscode compoint task references * Updating readme to use webapp port 3000 * Updating jpo-utils to cdot-cv for mongo execute permissions * Improving webapp npm setup documentation * Setting python debug config to use 8081 * Set Flask run port to 8081 and update launch configuration * Adding FLASK_RUN_PORT env var --------- Co-authored-by: john-wiens <jwiens3141@gmail.com> Co-authored-by: Brandon Payne <bpayne@trihydro.com> Co-authored-by: Drew Johnston <31270488+drewjj@users.noreply.github.com> Co-authored-by: Drew Johnston <drewjj901@gmail.com>
Updating Heatmap Visuals and Data Controls API rsucounts endpoints returns counts for all message types at once with aggregation query Webapp uses RTK query hooks to retrieve rsu counts data New heatmap cluster layer shows clustered counts for selected message type Counts message type fields migrated to MenuSlice * Adding dropdown on heatmap * Adding prototype aggregation endpoint to rsucounts * Adding fork sync instructions * Updating fork sync PR creation description * rsu counts returning all message types at once * Adding rtk-query rsuCountsApi * Updating DisplayCounts to use local state and rtk query counts * Improving mocking in timeSyncSlice test * Update timeSyncSlice.test.ts * Updating KC_LOG_LEVEL env var name * Adding additional heatmap display layers * Removing explicit mapbox gl import * Fixing heatMapData post processing * Updating msg count types to uppercase * Update Map.tsx * Filtering out heatmap data with less than 0 count * Updating RSU cluster colors * Adding extrusion geometry * Reducing extrusion height * Updating cluster sizing * Tweaking count labels color * Tweaking heatmap formatting * Removing heatmap extrusion layer * Adding message counts to RSU side menu * Opening message counts on heatmap display * tweaking heatmap layer opening count menu * Updating heatmap unit tests * Update test_rsu_querycounts.py * Update test_rsu_querycounts.py * Updating PR length recommendation * Cleaning up unused code and print statements * Fixing docker host ip in sample.env.local * Removing deprecated VSCode virtual environment instructions * Adding additional docker build debugging steps * Updating rsu querycounts tests for aggregation * Updating squash merge resolution to prefer merge * Updating heatmap stops array * Keying map layers object * Update Map.tsx * Tweaking heatmap intensity interpolation * Tweaking heatmap intensity interpolation * Distinguishing heatmap cluster borders * Final tweak of heatmap intensity * Adding python API .vscode default env vars * Removing unused intersection_no_api docker profile * Cleaning up docker-compose commands and language * Adding localhost:3001/3002 to realm.json * Cleaning up realm.json * Fixing timeSyncSlice test * Update Map.test.tsx.snap * Removing unnecessary COUNTS_MSG_TYPE variable * Updating unit tests * Updating heatmap intensity stops * Tweaking heatmap intensity scaling --------- Co-authored-by: john-wiens <jwiens3141@gmail.com> Co-authored-by: Brandon Payne <bpayne@trihydro.com> Co-authored-by: Drew Johnston <31270488+drewjj@users.noreply.github.com> Co-authored-by: Drew Johnston <drewjj901@gmail.com>
* Removing RSU configuration admin permission requirement * Requiring operator role to view RSU configuration menu
Fixing RSU configuration menu permissions to allow all org-roles above Operator to access the menu (operator and admin). The menu should only be hidden from users
…#261) * feat: add RsuSnmpFwdFetch resource for fetching SNMP configurations * feat: implement RSU message forwarding configuration fetch functionality * feat: display RSU message source type and enhance fetch functionality in SnmpwalkMenu * refactor: centralize SNMP message forwarding configuration formatting logic * refactor: introduce constants for message forwarding types and table names in SNMP helpers * feat: add memoized check for empty message forward configurations in SnmpwalkMenu * chore: move message source type display to the bottom of SnmpwalkMenu * feat: add error handling for missing RSU fields in SNMP config fetch * feat: refactor message forwarding types and table names to use Enums * feat: enhance type safety and handling for RSU message forward config fetch * refactor: rename RSU config fetch methods and actions for clarity and consistency * feat: update error message for RSU message forwarding configuration fetch * feat: add line break for improved layout in SnmpwalkMenu component * fix: update message forwarding type checks to use Enum values for consistency * chore: simplify error toast message for RSU message forwarding configuration fetch * refactor: change logging level to warning and remove unused error response in rsu_snmp_fwd_fetch * fix: avoid exposing raw exception details to client * fix: use logging.exception for better traceback visibility during SNMP config fetch errors * fix: add rejectWithValue for improved error handling in geoRsuQuery * fix: handle non-Error and unknown exceptions during RSU config fetch * fix: use a copy of rsu_info to preserve original data during SNMP config fetch * test: add unit tests for `snmp.rsu_message_forward_helpers` module * test: update config slice tests to include changeSuccess, destIp, and snmpMsgType * fix: ensure consistent use of Enum `.value` for TableNames in RSU message forwarding * fix: rename `m_type` to `msgfwd_type_value` for clarity in RSU message forwarding logic * chore: move closing brace * test: clean up unused imports in `test_rsu_message_forward_helpers` * test: update assertions to use `.value` for TableNames Enum in RSU message forwarding tests * docs: add detailed docstring for `format_snmp_msgfwd_configs` function * fix: return 500 with error details if RSU info is missing required SNMP fields * docs: add docstrings for `RsuSnmpFwdFetch` class and `get` method to improve clarity * test: add unit tests for `RsuSnmpFwdFetch` class to cover various scenarios in SNMP config retrieval * test: add unit test for handling API exceptions in `configSlice` * test: add unit test for handling non-ok response in `ApiHelper._getData` * chore: remove debug `console.log` statements from `configSlice` async actions * fix: handle undefined payload in `getRsuMsgConfigsFromRsu.rejected` case in `configSlice` * fix: handle optional chaining for undefined tables in `SnmpwalkMenu` and correct conditional logic * fix: improve error message for empty RSU message forwarding response in `configSlice` * fix: update TS type for `getRsuMsgConfigsFromRsu` response in `SnmpwalkMenu` * test: add unit tests for handling string and unknown exceptions in `getRsuMsgConfigsFromRsu` * fix: make `snmp_encrypt_pw` optional in `RsuSnmpFwdFetch` and update corresponding tests * test: update snapshot for `SnmpwalkMenu` to reflect new UI changes * refactor: improve error handling in RsuSnmpFwdFetch by raising appropriate exceptions * fix: ensure case insensitivity for message forwarding configuration types in rsu_message_forward_helpers * chore: update empty message forwarding configurations text in SnmpwalkMenu for clarity * fix: correct logic for checking message forwarding configuration tables in SnmpwalkMenu * test: parameterize message type and IP tests for improved readability and maintainability * chore: update snapshot for SnmpwalkMenu * test: update RsuSnmpFwdFetch tests to use ANY for flexibility in mock assertions * test: remove unnecessary caplog assertion in test_format_snmp_msgfwd_configs_unknown_type
* Updating map layer ID references * Removing python API moove ai features * Removing moove ai webapp features * Removing moove ai env vars * Removing moove ai unit tests * Removing additional moove ai env vars * Moove ai removal unit test tweaks
* chore: update dependencies via `npm audit fix` * chore: upgrade `jspdf` to 4.0.0 and update related dependencies * chore: add `TextEncoder` and `TextDecoder` to global setup in tests * chore: add dependency overrides for `nth-check`, `postcss`, and `webpack-dev-server` in `package.json` These overrides are used to resolve vulnerabilities in react-scripts' sub-dependencies that are no longer maintained. * chore: run up to date, audited 1954 packages in 2s 462 packages are looking for funding run `npm fund` for details found 0 vulnerabilities for newly detected vulnerabilities * chore: upgrade `eslint` to 9.39.2 and add override in `package.json` This resolves deprecation warnings for @humanwhocodes/config-array@0.13.0, @humanwhocodes/object-schema@2.0.3 and eslint@8.57.1 * Revert "chore: upgrade `eslint` to 9.39.2 and add override in `package.json`" This reverts commit e8fc5ad. * chore: remove `webpack-dev-server` override from `package.json`
* chore: run `npx viject` to migrate to Vite * chore: update dependencies for @material-table/core and @types/react-dom * chore: update testing setup to use Vitest * chore: update Jest snapshots to Vitest format * chore: migrate Jest mocks to Vitest * chore: update more snapshots to reflect migration from Jest to Vitest * chore: remove `overrides` section from package.json * chore: update Node.js version in Dockerfile to 22-alpine * chore: improve environment variable handling and update Mapbox style loading logic * chore: update environment variables to VITE_ prefix and migrate configuration files accordingly * chore: update ESLint and TypeScript-related dependencies * chore: update jsdom to v27.4.0 * chore: update component snapshots to reflect style changes in test files * chore: update Node.js version in CI workflow to 22 and remove redundant npm commands * chore: set CI timezone to UTC, switch to `npm ci`, and enhance `DateTime.local` override in tests * chore: update TypeScript and development dependencies in package.json * chore: update test files to use ISO 8601 date strings for consistency * chore: replace `jest` with `vitest` mocks test files failing in CI * chore: update CI timezone to America/Denver * chore: remove `.env.development` and update references to `sample.env.development.local`
* Creating Admin RSU postgres objects and repositories * Adding all queries for getModifyRsuData * Creating initial admin getRsuData endpoints * Successfully compiling RSU repository queries * Converting admin RSU queries to native * Working admin rsu iapi GET endpoint * Writing iapi rsu management unit tests * refactor: regenerate PostgreSQL entity classes for RSU management using JPA Buddy * Fixing repositories after table renaming * Fixing warnings * Removing RSU admin endpoint files * Re-generating all jpa models * Moving postgres models * Regenerating all jpa postgres objects * Tweaking jpa table data models * Converting PostgresService to Repository queries * Removing unused repos * Adding missing @id annotations * Update PermissionServiceTest.java * Updating sql to use booleans and adding missing pkeys * Fixing JPA Object data types * Updating iss_keys and obu_ota_requests JPA defs with pkeys * Minor code cleanup * Revert "Updating sql to use booleans and adding missing pkeys" This reverts commit 55b791c. * Updating iapi jpa boolean types * Moving create schema keycloak to top of sql script --------- Co-authored-by: Matt Cook <matt.cook@trihydro.com>
* Adding RSU admin endpoints to IAPI * Update OrganizationControllerTest.java * Cleaning up comments * Cleaning up comments and code * Removing all relations when deleting RSU * Update RsuManagementServiceTest.java * Updating delete rsu 404 and 403 processing * Cleaning up code and comments * Updating tests * Updating comments and exception types * Improving multi-rsu permission checks * Enforcing organization permissions on edit rsu * Fixing modifyRsu unit tests
* Adding RSU admin endpoints to IAPI * Migrating webapp to iapi admin rsu endpoints * Migrating webapp to iapi admin rsu endpoints * Update OrganizationControllerTest.java * Fixing minor admin edit rsu bugs * Fixing AdminTable typing * Fixing user org permissions check * Update auth_tools.py
* fix: prevent non-super users from granting super user privileges in backend modify_user function * fix: enhance user privilege checks to prevent non-super users from modifying super_user status * fix: update modify_user tests to include user validation and mock query for super_user status * fix: add tests to validate user privilege checks for modifying super user status * fix: conditionally render super user toggle based on user privileges * refactor: raise 'Forbidden' instead of 'BadRequest' for privilege escalation attempt * refactor: move `query_db` call into try block * chore: add TODO to `admin_new_user.py` * fix: conditionally render 'Super User' checkbox when creating new users * fix: add super_user check to `admin_new_user.py` to prevent privilege escalation via user creation * chore: update comments in a couple tests in `test_admin_user.py` * test: add unit test to `test_admin_new_user.py` to verify privilege escalation attempt fails * test: update unit tests in `test_admin_new_user.py` to provide requesting user when calling `add_user` * chore: uncomment call to `enforce_organization_restrictions` * chore: explicitly catch and re-throw Forbidden exception
* Temporarily remove tags from admin rsu rtk * Re-adding rsuApiSlice tags * Removing armin rsu rtk onQueryStarted * Testing adminTable paging fetchData integration * Working ss paging and sorting within admin rsu table * Cleaning up comments * Update webapp/src/components/AdminTable.tsx Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Fixing tests * Updating iapi CI pipeline * Updating iapi github CI * Setting iapi CI docker engine to v28 * Update ci.yml * Update ci.yml * Updating testcontainer deps * Updating testcontainers version * Update AdminRsuTab.tsx * Update webapp/src/components/AdminTable.tsx Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update webapp/src/features/adminRsuTab/AdminRsuTab.tsx Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Fixing sorting keys in admin rsu page * Cleaning up admin rsu ordering code * Adding rsu manufacturer to admin rsu query * Tweaking searching of rsu manufacturer --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Migrating user-auth endpoint to webapp only * Updating unit tests * Update README.md * Tweaking login failure logic * Fixing unit tests
* feat: added "TIM Deposit" checkbox functionality to AdminAddRsu and AdminEditRsu forms * feat: add "Enable/Disable TIM Deposit for all RSUs" functionality in AdminOrganizationTabRsu component * chore: add TODO for implementing organization-level TIM deposit toggle in admin_org * feat: add "TIM Deposit" column to RSU tables and display status in AdminOrganizationTab components * refactor: rename `timDeposit` to `timDepositStatus` and update related handlers in AdminOrganizationTabRsu * chore: update AdminFormManager snapshot to include "Enable/Disable TIM Deposit for all RSUs" buttons and "TIM Deposit" column adjustments * ui: add SNMP Monitoring support to Admin RSU management forms and tables * chore: update snapshots to include SNMP Monitoring toggle and adjust layout spacing in Admin components * test: update tests to include SNMP Monitoring and adjust TIM Deposit handling for RSUs and organizations * refactor: move TIM Deposit and SNMP Monitoring logic from state to selectors and update related tests * ui: add dynamic endpoints for TIM Deposit and SNMP Monitoring operations in admin organization tab * ui: update success/error color shades for TIM Deposit and SNMP Monitoring status in Admin tabs * refactor: simplify organization data fetching logic in AdminOrganizationTab and remove redundant effect * Introduce RSU TIM deposit flag in database and backend (#272) * db: add `tim_deposit` column to `rsus` table with default and organization-specific values * api: add `tim_deposit` field support for RSU operations * db: replace `tim_deposit` column with new `rsu_options` table * api: integrate `rsu_options` table and add `snmp_monitoring` field support for RSU operations * api/db: switch `tim_deposit` and `snmp_monitoring` fields to boolean type * db: set default `snmp_monitoring` to `FALSE` in `rsu_options` table * db: update comment in rsu_options.sql * tests: update RSU data queries to include `snmp_monitoring` field from `rsu_options` table * api: remove boolean conversion for `tim_deposit` and `snmp_monitoring` fields * api: standardize boolean handling for `tim_deposit` and `snmp_monitoring` fields across RSU operations * tests: update RSU data handling to include `tim_deposit` and `snmp_monitoring` fields * refactor(api): simplify boolean handling for `tim_deposit` and `snmp_monitoring` fields in RSU operations * api: add endpoints for managing TIM Deposit and SNMP Monitoring settings per organization * api: handle RuntimeError for `request.endpoint` lookup when determining bulk updates * tests: add permission result handling to admin org and new RSU tests * tests: update admin RSU data to use Python boolean values for `tim_deposit` and `snmp_monitoring` * api: make `tim_deposit` and `snmp_monitoring` fields optional with default values * api/tests: use `COALESCE` for `tim_deposit` and `snmp_monitoring` to ensure default boolean values * tests: update RSU deletion calls to include `rsu_options` cleanup * sql: add clarification comment for `Region 1` in `rsu_options.sql` update * api/tests: use `COALESCE` for `tim_deposit` and `snmp_monitoring` to ensure default boolean values in queries * api: use `INSERT ... ON CONFLICT` for upserting RSU options * tests: add assertion for message comparison in admin new RSU test * tests: remove unused `PermissionResult` and `ORG_ROLE_LITERAL` imports in admin org and new RSU tests * api: remove unused `endpoint` field from admin org schema * api: change `Boolean` to `Bool` for `tim_deposit` and `snmp_monitoring` fields * api: consolidate bulk update logic for `tim_deposit` and `snmp_monitoring` in RSU options * Revert "api: remove unused `endpoint` field from admin org schema" This reverts commit 4a7b9a2 * fix: make bulk RSU updates more selective by avoiding unnecessary column overwrites. * fix(sql): update `tim_deposit` default value to `FALSE` in `rsu_options` table creation scripts * fix(sql): update `tim_deposit` default population to `FALSE` for all RSUs in `rsu_options` * test: refactor admin organization tests to use flask test request context instead of mocked request objects * test: add bulk RSU update test cases for `tim_deposit` and `snmp_monitoring` in admin organization module * test: improve admin RSU tests by verifying `pgquery` calls with specific parameters * fix: improve JSON body validation for admin RSU endpoints by switching from `validate` to `load` * chore: update snapshot * Restore RSU Admin Support for `tim_deposit` and `snmp_monitoring` in Intersection API (#286) * feat: add tim_deposit and snmp_monitoring fields to RsuInfoDto * feat: add RsuOption entity with tim_deposit and snmp_monitoring fields; update Rsu class relationships * chore: add TODOs for repository and relationship updates * fix: change parameter type from List to Set in mapOrganizationNames method * fix: update RsuManagementServiceTest to use HashSet for rsuOrganizations and add boolean parameters in constructor calls * test: update RsuControllerTest to include boolean parameters in RsuInfoDto constructor calls * fix: change import from ArrayList to HashSet in RsuManagementServiceTest * feat: add mappings for timDeposit and snmpMonitoring in RsuInfoMapper * feat: add RsuOptionRepository for managing RsuOption entities * feat: add timDeposit and snmpMonitoring properties to RsuPatch * feat: implement updateRsuOptions method to manage RSU option fields * feat: add snmp_monitoring property to Rsu type definition * feat: integrate tim_deposit and snmp_monitoring into RSU form handling * feat: reorder RSU options handling to occur after saving RSU entity * chore: update RSU entity relationships to use mappedBy for consistency * feat: establish bidirectional association for RsuOption and update save behavior * test: add unit tests for modifying RSU options in RsuManagementService * fix: handle undefined values for tim_deposit and snmp_monitoring in AdminEditRsu * Revert "feat: reorder RSU options handling to occur after saving RSU entity" This reverts commit fcb734d * chore: revert changes to `RsuManagementService` * feat: add modifyRsuOption method to handle RSU option updates * feat: enhance RsuManagementService to handle updates for both snmp_monitoring and tim_deposit fields * feat: implement RsuOptionManagementService for managing RSU options and update RsuController to use it * chore: remove logging statements from RsuManagementService * chore: update logging levels in RsuOptionManagementService * docs: simplify comments and documentation in RsuOptionManagementService * test: add RsuOptionManagementService mock to RsuControllerTest for option modification verification * feat: add methods to remove RSU options by IPv4 address in RsuOptionRepository and update RsuManagementService * feat: add sample data for rsu_options in CVManager_SampleData.sql * fix: correct order of comparison for tim_deposit and snmp_monitoring values in RsuOptionManagementService * refactor(api): remove AdminRsu endpoint and related functionality (#291) * refactor(api): remove AdminRsu endpoint and related functionality * refactor(api): remove admin_rsu_data mock data * chore: update snapshots to include TIM Deposit column changes and adjust table layout * refactor: remove endpoint-based bulk update logic and simplify `modify_org_authorized` with explicit `is_bulk_update` parameter * refactor: remove schema specification from `@Table` annotations in `Rsu` and `RsuOption` classes * chore: add TIM Deposit and SNMP Monitoring endpoints to middleware configuration * chore: update middleware configuration to not require organization header for TIM Deposit and SNMP Monitoring endpoints * chore: add TIM Deposit field mapping and test sorting by TIM Deposit in RSU controller * refactor: reorder `findRsuByIp` call in `modifyRsuOption` method * chore: add custom mapping methods for TIM Deposit and SNMP Monitoring in RsuInfoMapper * feat: add SNMP Monitoring field to Admin RSU tab and map SNMP Monitoring in RsuController * refactor: update `modify_org_authorized` return value and incorporate `get_modify_org_data_authorized` utility, adjust related tests and snapshot layouts * test: remove unnecessary stubbing * refactor: improve `getSelectedTab` logic to use filtered tab paths when determining active tab * test: add unit tests for `VerticalTabs` component to verify tab selection logic and fallback behavior * fix: add change handlers for TIM Deposit and SNMP Monitoring fields, and delay after RSU update for backend consistency
* feat: added a new Spring Boot service `rsu-info-bridge` * config: set custom server port for `rsu-info-bridge` service * docs: add initial README for `rsu-info-bridge` service * chore: make `mvnw` executable * feat: add spring-boot-devtools dependency to `rsu-info-bridge` * docs: update README to simplify structure and remove placeholders * chore: make `mvnw.cmd` executable * chore: update dependencies in `rsu-info-bridge` pom.xml * chore: update Java version to 22 in `rsu-info-bridge` pom.xml
* sql: update table creation script to add owner_organization_id column to rsu_credentials and snmp_credentials tables * sql: update sample data script to insert data for owner_organization_id column into rsu_credentials and snmp_credentials tables * sql: implement migration script to add owner_organization_id column to rsu_credentials and snmp_credentials tables * chore: add new columns to RsuCredential and SnmpCredential java models * feat: add new methods to RsuCredentialRepository and SnmpCredentialRepository for retrieving by owner organization id * sql: refactor owner_organization_id updates in migration script for credential tables * sql: clean up NULL owner_organization_id rows before adding NOT NULL constraint * refactor: remove 'public' schema declaration in RsuCredential and SnmpCredential classes * refactor: remove unused methods for fetching credentials by owner organization id * refactor: replace ownerOrganizationId with lazy-loaded Organization relationship in RsuCredential and SnmpCredential classes * sql: handle orphaned credentials before enforcing NOT NULL on owner_organization_id * sql: rename foreign key constraints for clarity in credential tables
* feat: added a new Spring Boot service `rsu-info-bridge` * feat: add RsuDto model to rsu-info-bridge service * feat: add RsuController, RsuService, and ExampleDataRepository to rsu-info-bridge service with basic functionality and tests * refactor: replace ExampleDataRepository with mock data in RsuService and remove unused repository * test: add mock data and update RsuControllerTest with enhanced assertions * test: make getMockData and initializeMockMvc non-private in RsuControllerTest * config: make rsu-info-bridge server port configurable via environment variable * fix: correct default port variable syntax in application.yaml * chore: update Java version to 25 in pom.xml and README.md * feat: add Docker Compose setup and sample environment file for rsu-info-bridge service * docs: update README with instructions for running via Docker Compose * refactor: consolidate RsuController endpoints into a single method with timDepositEnabled filter * chore: remove redundant RSU_INFO_BRIDGE_PORT environment variable in Docker Compose configuration * refactor: update latitude and longitude in RsuDto to use primitive types * feat: add POST endpoint to query RSUs by ID in RsuController * refactor: update `getAll` method in RsuController to return ResponseEntity * refactor: remove ResponseEntity wrapper from RsuController methods * refactor: remove POST endpoint and related test in RsuController * docs: remove reference to RSU_INFO_BRIDGE_PORT in README.md * refactor: remove unused `getByIds` method in RsuService * docs: update README with `.env` setup instructions for Docker Compose
* feat: added a new Spring Boot service `rsu-info-bridge` * feat: add RsuDto model to rsu-info-bridge service * feat: add RsuController, RsuService, and ExampleDataRepository to rsu-info-bridge service with basic functionality and tests * refactor: replace ExampleDataRepository with mock data in RsuService and remove unused repository * test: add mock data and update RsuControllerTest with enhanced assertions * test: make getMockData and initializeMockMvc non-private in RsuControllerTest * config: make rsu-info-bridge server port configurable via environment variable * fix: correct default port variable syntax in application.yaml * feat: integrate OpenAPI documentation with springdoc and update README with usage details * feat: enhance RsuController and RsuDto with OpenAPI annotations for improved API documentation * refactor: remove optional comments from authenticationProtocol and privacyProtocol fields in RsuDto * chore: update Java version to 25 in pom.xml and README.md * feat: add Docker Compose setup and sample environment file for rsu-info-bridge service * docs: update README with instructions for running via Docker Compose * refactor: consolidate RsuController endpoints into a single method with timDepositEnabled filter * chore: remove redundant RSU_INFO_BRIDGE_PORT environment variable in Docker Compose configuration * refactor: update latitude and longitude in RsuDto to use primitive types * feat: add POST endpoint to query RSUs by ID in RsuController * refactor: update `getAll` method in RsuController to return ResponseEntity * refactor: remove ResponseEntity wrapper from RsuController methods * refactor: remove POST endpoint and related test in RsuController * docs: remove reference to RSU_INFO_BRIDGE_PORT in README.md * refactor: remove unused `getByIds` method in RsuService * docs: add description for timDepositEnabled parameter in getAll endpoint * refactor: rename `timDepositEnabled` to `timDepositEnabledOnly` in `getAll` method parameter * docs: update example value for RSU ID in RsuDto schema * refactor: update example RSU IDs in RsuService mock data * test: update mock RSU IDs in RsuControllerTest for consistency * test: update query parameter in RsuControllerTest to `timDepositEnabledOnly`
* Adding RSU admin endpoints to IAPI * Migrating webapp to iapi admin rsu endpoints * Migrating webapp to iapi admin rsu endpoints * Update OrganizationControllerTest.java * Adding createRsu endpoint * Working createRsu endpoint * Resolving merge conflicts * Temporarily remove tags from admin rsu rtk * Re-adding rsuApiSlice tags * Removing armin rsu rtk onQueryStarted * Testing adminTable paging fetchData integration * Working ss paging and sorting within admin rsu table * Cleaning up comments * Update webapp/src/components/AdminTable.tsx Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Fixing tests * Updating iapi CI pipeline * Updating iapi github CI * Setting iapi CI docker engine to v28 * Update ci.yml * Update ci.yml * Updating testcontainer deps * Updating testcontainers version * Working rsu creation through admin page * Update AdminRsuTab.tsx * Update webapp/src/components/AdminTable.tsx Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update webapp/src/features/adminRsuTab/AdminRsuTab.tsx Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Reverting CI container changes * Update services/intersection-api/api/src/main/java/us/dot/its/jpo/ode/api/mappers/RsuInfoMapper.java Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Fixing sorting keys in admin rsu page * Fixing primaryRoute other * Cleaning up admin rsu ordering code * Adding rsu manufacturer to admin rsu query * Tweaking searching of rsu manufacturer * Cleaning up admin rsu webapp code * Update services/intersection-api/api/src/main/java/us/dot/its/jpo/ode/api/services/RsuManagementService.java Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Cleaning up tests and RsuInfoDto * Adding MapperUtils for MapStruct DTOs * Tweaking createRsu endpoint * Update RsuControllerTest.java * Improving rsu creation exception handling * Removing python admin new rsu code * Fixing and reformatting RSU tests * Updating mapstruct mappers * Adding basic GlobalExceptionHandler * Tweaking global exception handler and adding tests * Supporting ErrorResponse esceptions in webapp * Removing handleQueryExceptions * Update services/intersection-api/api/src/main/java/us/dot/its/jpo/ode/api/services/RsuManagementService.java Co-authored-by: Matt Cook <mattheworion.cook@gmail.com> --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Co-authored-by: Matt Cook <mattheworion.cook@gmail.com>
* sql: update table creation script to add owner_organization_id column to rsu_credentials and snmp_credentials tables * sql: update sample data script to insert data for owner_organization_id column into rsu_credentials and snmp_credentials tables * sql: implement migration script to add owner_organization_id column to rsu_credentials and snmp_credentials tables * chore: add new columns to RsuCredential and SnmpCredential java models * feat: add new methods to RsuCredentialRepository and SnmpCredentialRepository for retrieving by owner organization id * chore: set up controller and service skeletons with corresponding test classes * feat: implement create endpoint for RsuCredential and add corresponding DTO and mapper * feat: add getByNickname endpoint to RsuCredentialController and corresponding service method * feat: implement update endpoint for RsuCredential with corresponding response and patch classes * feat: implement deleteByNickname endpoint for RsuCredential with corresponding request and response classes * feat: refactor RsuCredential endpoints to use request objects for create and get operations * feat: enhance RsuCredential endpoints to return structured responses with success status and error messages * test: add TODOs for implementing unit tests for unhappy paths in RsuCredentialControllerTest * feat: enhance RsuCredential endpoints with exception handling for create, get, update, and delete operations * chore: add TODO to update RsuCredential endpoints with @PreAuthorize annotation * chore: rename createRsuCredential method to create in RsuCredentialManagementService and update references * test: add verification for service interactions in RsuCredentialControllerTest * feat: implement create method in RsuCredentialManagementService * feat: implement getByNickname method in RsuCredentialManagementService * feat: implement update method in RsuCredentialManagementService * test: add unit test to RsuCredentialManagementServiceTest for organization update * feat: implement deleteByNickname method in RsuCredentialManagementService * chore: add TODOs for unhappy path tests in RsuCredentialManagementServiceTest * chore: set up skeleton contents for SnmpCredentialController and SnmpCredentialControllerTest * feat: implement create endpoint in SnmpCredentialController * feat: implement getByNickname endpoint in SnmpCredentialController * feat: implement update endpoint in SnmpCredentialController * feat: implement deleteByNickname endpoint in SnmpCredentialController * chore: add skeleton tests to SnmpCredentialManagementServiceTest * feat: implement create method in SnmpCredentialManagementService * feat: implement getByNickname method in SnmpCredentialManagementService * feat: implement update method in SnmpCredentialManagementService * test: implement test for updating organization in SnmpCredentialManagementServiceTest * feat: implement deleteByNickname method in SnmpCredentialManagementService * test: add unit tests for failure cases in RsuCredentialManagementService * refactor: remove unused variables in RsuCredentialManagementServiceTest * chore: remove catch block in deleteByNickname endpoint * chore: add TODO to SnmpCredentialController * test: add unit tests for failure cases in RsuCredentialController * test: add unit tests for failure cases in SnmpCredentialManagementService * chore: remove unnecessary newline * feat: add partialUpdate method to RsuCredentialMapper with null value property mapping strategy * chore: add TODOs for refactoring exception handling in credential services and controllers * refactor: simplify exception handling in credential controllers and tests, add GlobalExceptionHandler for centralized handling * refactor: replace @Getter, @Setter, and @requiredargsconstructor with @DaTa in credential request and response classes * refactor: return DTOs directly from credential endpoints and remove unused response classes * chore: remove outdated TODO in RsuCredentialController * refactor: standardize exception handling by replacing custom exceptions with EntityNotFoundException in credential services, controllers, and tests * refactor: improve GlobalExceptionHandler to return ProblemDetail with HTTP status and message * test: add unit tests for GlobalExceptionHandler covering exception scenarios * refactor: simplify delete credential methods by removing response objects and enforcing exceptions * test: update deleteByNickname test to assert EntityNotFoundException using updated request object * test: add unit tests for unhappy paths in SnmpCredentialController * refactor: add @PreAuthorize annotations to protect credential endpoints * chore: add TODOs for organization-level permission checks and ownership validations in credential services and controllers * feat: enforce organization-level permission checks and ownership validations in credential services and controllers * sql: refactor owner_organization_id updates in migration script for credential tables * sql: clean up NULL owner_organization_id rows before adding NOT NULL constraint * refactor: remove 'public' schema declaration in RsuCredential and SnmpCredential classes * refactor: remove unused methods for fetching credentials by owner organization id * refactor: replace ownerOrganizationId with lazy-loaded Organization relationship in RsuCredential and SnmpCredential classes * refactor: replace ownerOrganizationId references with ownerOrganization entity in mappers and services for RsuCredential and SnmpCredential * refactor: remove redundant @JsonProperty annotations from DTO classes * refactor: simplify RsuCredential and SnmpCredential service methods by removing organization parameter and enhancing PermissionService integration * refactor: remove redundant existsByNickname query from SnmpCredentialRepository
* Setting base path to absolute for refresh support * Combining GlobalExceptionHandlers * Fixing merge conflicts * Update GlobalExceptionHandler.java
* chore: add new dependencies to `rsu-info-bridge` for JPA, JTS, and validation functionality * feat: add JPA entity models and repository for `rsu-info-bridge` * feat: add initial RsuService with placeholder method and constants * chore: add RsuOption entity and establish relationship with Rsu * chore: add custom query method to RsuRepository for filtering by TimDeposit * feat: implement filtering in RsuService for TimDeposit enabled options * testing: update RsuControllerTest to use Rsu and RsuOption models with mock data * test: configure embedded database for integration tests and update application-test.yaml * test: refactor RsuControllerTest to use mock RsuRepository for data retrieval * feat: implement RsuDtoMapper for converting Rsu entities to RsuDto and update RsuController to use it * test: enhance RsuControllerTest with detailed assertions for RSU properties * test: add comprehensive tests for RsuController including success and error scenarios * test: add embedded PostgreSQL dependency for integration tests and clean up imports in RsuInfoBridgeApplicationTests * feat: add Testcontainers configuration for PostGIS-enabled PostgreSQL and update application-test.yaml * test: add RsuServiceTest with comprehensive tests for RSU retrieval and database operations * refactor: simplify RSU creation in RsuServiceTest and update TestDatabaseHelper methods * fix: remove duplicate test organization entries from sample data * chore: enhance Testcontainers configuration and update TestDatabaseHelper for sample data integration * refactor: simplify RSU creation in RsuServiceTest and update test assertions * chore: import TestcontainersConfiguration in RsuInfoBridgeApplicationTests.java * chore: add PostgreSQL datasource configuration and environment variables * refactor: remove unnecessary query annotation from RsuRepository method * refactor: correct latitude and longitude values in RsuControllerTest * chore: update Hibernate dialect to PostgreSQLDialect in application-test.yaml * chore: remove specified version for jakarta.validation-api dependency in pom.xml * chore: update Hibernate dialect to PostgreSQLDialect in application.yaml * chore: add JsonInclude annotation to RsuDto for non-null serialization * feat: add validation annotations to RsuDto and update related classes * test: update query parameter in RsuControllerTest for consistency * test: update RsuControllerTest with to align with RsuDto validation requirements * chore: add Manufacturer, Organization, and RsuModel entities to ensure proper mapping of NOT NULL foreign key columns * chore: update database configuration to include POSTGRES_DB environment variable * feat: implement global exception handler in rsu-info-bridge for improved error management * chore: remove unused import statement from RsuRepository to clean up code * chore: remove redundant validation dependencies from pom.xml to simplify project configuration * chore: update PostgreSQL server URL in compose.yml and sample.env for improved Docker compatibility * docs: update README to include instructions for starting PostgreSQL service before running the application * chore: update testcontainers version to 1.20.0 and add explicit dependency for commons-compress to address security vulnerabilities * ci: add CI workflow for rsu-info-bridge with Maven build and test steps * ci: remove conditional from test results archiving step in CI workflow * refactor: simplify RsuDtoMapper mappings and remove manually defined mapping methods * refactor: update RsuDto to use primitive types and remove null validation tests * refactor: remove validation annotations from RsuDto and RsuController * test: update RsuDtoMapperTest to use GeometryFactory for Point creation * refactor: update TestDatabaseHelper to use repositories for database operations * refactor: update RsuServiceTest to use repository methods directly instead of relying on TestDatabaseHelper * refactor: update method signature in RsuRepository to avoid using unnecessary parameter * docs: add comments documenting manual MockMvc setup in tests * docs: add comments documenting manual MockMvc setup in tests * refactor: simplify MockMvc setup in tests by using @SpringBootTest and @AutoConfigureMockMvc * refactor: remove unnecessary @Injectmocks for RsuService and RsuDtoMapper in RsuControllerTest * refactor: remove unused imports in RsuControllerTest * refactor: replace mocked Point instances with GeometryFactory for coordinate creation in RsuControllerTest * refactor: move repository and model classes to testutil package for cleaner organization * refactor: remove redundant environment variables from compose.yml * fix: add ArrayList import to RsuManagementService.java * chore: add Docker Compose configuration for PostgreSQL integration in application.yaml and update README * chore: update `compose.yml` to include PostgreSQL service and modify README for usage instructions * refactor: move model classes back to `tables` package * chore: remove JsonInclude annotation from RsuDto * chore: update testcontainers dependencies for PostgreSQL integration * test: add `@Testcontainers` annotation to RsuServiceTest * chore: remove duplicate import of ActiveProfiles in RsuServiceTest * chore: update application configuration for local development * ci: use wrapper script for Maven build in rsu_info_bridge CI configuration
* Using decoded JWT data for simple user auth * Updating unit tests * Update services/intersection-api/api/src/main/java/us/dot/its/jpo/ode/api/services/PermissionService.java Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Cleaning up token decoding code, adding DecodedTokenTest * Debugging permission service test failures * Fixing PermissionService tests * Adding RequestScopedDecodedToken to cache decoded JWT tokens * Update services/intersection-api/api/src/main/java/us/dot/its/jpo/ode/api/models/keycloak/DecodedToken.java Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update services/intersection-api/api/src/main/java/us/dot/its/jpo/ode/api/models/keycloak/DecodedToken.java Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update services/intersection-api/api/src/main/java/us/dot/its/jpo/ode/api/models/keycloak/DecodedToken.java Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Adding AuthUtils * Update .vscode/launch.json Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update services/intersection-api/api/src/main/java/us/dot/its/jpo/ode/api/models/keycloak/DecodedToken.java Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update services/intersection-api/api/src/main/java/us/dot/its/jpo/ode/api/models/keycloak/RequestScopedDecodedToken.java Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update DecodedTokenTest.java * Documenting fields in DecodedToken * Adding IAPI token expiration validation * Adding hasRoleInOrgs * Update RsuControllerTest.java * Update RsuControllerTest.java * Switching from DecodedToken to CvManagerAuthToken * Combining GlobalExceptionHandlers and re-adding isSuperUser * Handling null values in CvManagerAuthToken * Update AuthUtils.java * Allowing super users to view admin pages * Update package-lock.json --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Adding notif email generation and sending to the iapi * Adding notification email examples * Adding iapi email controller * Fixing EmailServiceTest * Update EmailController.java * Update EmailServiceTest.java * Fixing broken config property refs * Updating email generator comments, colors, and content * Updating default iapi env vars to avoid localhost * Update EmailRecipient.java * Adding email frequency to tables and queries * Update launch.json * Improving getCombinedResponseEntity to handle partial failures * Removing secure env vars from application.yaml * Renaming email template to email_template.html * Removing duplicate dependency * Update services/intersection-api/api/src/main/java/us/dot/its/jpo/ode/api/emails/generators/IntersectionNotificationSummaryEmailGenerator.java Co-authored-by: Matt Cook <mattheworion.cook@gmail.com> * Handling unsubscribe URL generation errors * Removing unused methods * Cleaning up email generation code * Writing email provider tests * Adding sql update script * Update email_subscription_type_update.sql * SUpporting authenticated SMTP server * Regenerating email postgres JPA objects * Working email sending with JPA repositories * Temporarily force enabling emailer * Renaming INTERSECTION_SMTP_SERVER_HOST * Adding spring mail ssl trust policy * Hardcoding gmail for testing * Adding logging for email recipients * Adding additional iapi email logging * Adding transport listener for email send debugging * Adding sentDate header * Reverting add transport listener * Reverting email logging changes * Combining GlobalExceptionHandlers * Update EmailService.java * Update launch.json --------- Co-authored-by: Matt Cook <mattheworion.cook@gmail.com>
* Using decoded JWT data for simple user auth * Updating unit tests * Update services/intersection-api/api/src/main/java/us/dot/its/jpo/ode/api/services/PermissionService.java Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Cleaning up token decoding code, adding DecodedTokenTest * Adding admin user IAPI endpoints * Debugging permission service test failures * Fixing PermissionService tests * Adding RequestScopedDecodedToken to cache decoded JWT tokens * Update services/intersection-api/api/src/main/java/us/dot/its/jpo/ode/api/models/keycloak/DecodedToken.java Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update services/intersection-api/api/src/main/java/us/dot/its/jpo/ode/api/models/keycloak/DecodedToken.java Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update services/intersection-api/api/src/main/java/us/dot/its/jpo/ode/api/models/keycloak/DecodedToken.java Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Adding AuthUtils * Update .vscode/launch.json Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update services/intersection-api/api/src/main/java/us/dot/its/jpo/ode/api/models/keycloak/DecodedToken.java Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update services/intersection-api/api/src/main/java/us/dot/its/jpo/ode/api/models/keycloak/RequestScopedDecodedToken.java Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update DecodedTokenTest.java * Adding userApiSlice * Working admin user viewing/editing * Documenting fields in DecodedToken * Adding IAPI token expiration validation * Casing super_user to bit in iapi * Adding admin user tests * Removing python admin-user endpoints * Adding org user endpoints * Update OrganizationControllerTest.java * Update OrganizationController.java * Finalizing user organization page iapi conversion * Adding hasRoleInOrgs * Update RsuControllerTest.java * Update RsuControllerTest.java * Switching from DecodedToken to CvManagerAuthToken * Combining GlobalExceptionHandlers and re-adding isSuperUser * Update adminOrganizationTabUserSlice.test.ts * Migrating to CvManagerAuthToken * Adding constants for rtk query tag names * Using path params in user rest queries * Enabling refresh of admin user data on tag invalidation * Creating UserOrganizationDtoMapper * Updating rtk tag names * Handling null values in CvManagerAuthToken * Update AuthUtils.java * Allowing super users to view admin pages * Re-organizing DTO directories * Updating dto locations * Fixing admin user import in python api --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Adding typescript validation to build scripts and configuring tsconfig * Removing unused imports * Cleaning up tsconfig * Reverting react-hook-form upgrade test * Removing and ignoring tsbuildinfo * Adding test run typechecking * Update notifications-table.tsx * Update AdminOrganizationTab.tsx * Documenting ts commands * Removing typescript from npm commands * Enabling CI checks to run when change limit fails * Update AdminAddRsu.tsx * Update notifications-table.tsx * Removing additional unused imports
* Setting base path to absolute for refresh support * Combining GlobalExceptionHandlers * Fixing merge conflicts * Switching from Zonky to TestContainers * Update services/intersection-api/api/src/test/java/us/dot/its/jpo/ode/api/config/AbstractIntegrationTest.java Co-authored-by: Matt Cook <mattheworion.cook@gmail.com> * Update services/intersection-api/api/src/test/java/us/dot/its/jpo/ode/api/config/AbstractIntegrationTest.java Co-authored-by: Matt Cook <mattheworion.cook@gmail.com> * fix(tests): downgrade Testcontainers version and simplify AbstractIntegrationTest config - Downgraded `testcontainers` dependency in `pom.xml` from `2.0.3` to `1.21.4` for compatibility. - Corrected mismatched class and file name - Removed `@ServiceConnection` annotations and reorganized bean definitions. * feat(tests): add integration test profile for Intersection API - Introduced `application-integration-test.yaml` for service-layer integration tests. - Configured test containers for PostgreSQL using `tc:` JDBC URL and MongoDB via `@DynamicPropertySource`. - Defined Kafka topics and schemas, security, email, and other environment-specific settings for testing. * refactor(tests): remove AbstractIntegrationTest and migrate tests to use Testcontainers MongoDB configuration - Deleted `AbstractIntegrationTest` class and associated PostgreSQL container setup. - Migrated all test classes to use standalone MongoDB containers with `@Testcontainers` and `@DynamicPropertySource`. - Updated application test profiles to reflect the removal of PostgreSQL references. - Simplified test setup and adjusted annotations for improved maintainability. * fix(tests): update Hibernate `ddl-auto` setting to `none` in integration test config - Changed `ddl-auto` property in `application-integration-test.yaml` to prevent schema recreation during tests. * feat: add conditional bean and property loading for ASN.1 decoders - Applied `@ConditionalOnBean` to all ASN.1 decoders and `DecoderManager` to ensure presence of `MessageFrameCodec` before initialization. - Added `@ConditionalOnProperty` in `CodecConfig` for enabling ASN.1 decoder configuration (`enable.asn1-decoder` flag). - Updated `Asn1Controller` with `@ConditionalOnBean(DecoderManager.class)` to ensure dependency injection. - Modified `application-integration-test.yaml` to include `asn1-decoder` configuration. * feat: centralize Testcontainers version with BOM and add Spring Boot Testcontainers dependency - Added `spring-boot-testcontainers` dependency for enhanced test configuration. - Introduced `dependencyManagement` with `testcontainers-bom` to manage Testcontainers version consistency. - Removed redundant explicit version declarations for Testcontainers dependencies. * fix(ci): still run tests if change limit exceeded This allows us to still see test results when we exceed file change limits in valid cases. * test(containers): use ServiceConnection to speed up integration tests * refactor(tests): migrate decoder tests to Mockito and simplify setup - Replaced Spring Boot test annotations with Mockito for improved test performance. - Adjusted test setup to use `@BeforeEach` for initializing decoders with mocked dependencies. - Updated package paths for decoder test classes. - Removed unused imports and redundant annotations. * Revert "refactor(tests): migrate decoder tests to Mockito and simplify setup" This reverts commit 17932ec. * refactor(tests): update package paths for decoder test classes - Moved decoder test classes from `decoderTests` to `asn1` package for correctness. - Removed unused imports across decoder test files. * refactor(tests): enable ASN.1 decoder property and simplify test setup - Added `@MockitoBean` for `MessageFrameCodec` in decoder tests to satisfy `@ConditionalOnBean` dependency. - Enabled `enable.asn1-decoder` property in `@SpringBootTest` annotations across test classes. - Removed unnecessary `@ActiveProfiles` and `@Import` annotations for cleaner configuration. * chore(config): remove zonky reference from comment * refactor(tests): migrate decoder tests to use Mockito and clean up setup - Replaced Spring Boot test annotations with Mockito for faster and lightweight tests. - Refactored test setup to initialize decoders with `@BeforeEach` and mocked dependencies. - Removed unused imports and redundant code in decoder test classes. - Simplified file reading logic within test setup blocks. --------- Co-authored-by: jacob6838 <jfrye@neaeraconsulting.com> Co-authored-by: Jacob Frye <51723791+jacob6838@users.noreply.github.com>
* Adding typescript validation to build scripts and configuring tsconfig * Removing unused imports * Cleaning up tsconfig * Reverting react-hook-form upgrade test * Removing and ignoring tsbuildinfo * Adding test run typechecking * Resolving all typescript errors * Update notifications-table.tsx * Update AdminOrganizationTab.tsx * Updating unit tests * Documenting ts commands * Removing typescript from npm commands * Revert "Removing typescript from npm commands" This reverts commit 5222d1b. * Updating typescript types for organizations * Skipping org queries when no org selected
* Add support for checking the new capitalization for multiple menus that no longer function correctly * Update webapp/src/pages/Map.tsx Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update webapp/src/features/menu/Menu.tsx Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Add support for checking the new capitalization for multiple menus that no longer function correctly * Adding selectIsRoleOrAbove selectors * Removing unused imports * Update webapp/src/generalSlices/userSlice.ts Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update webapp/src/generalSlices/userSlice.ts Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update userSlice.test.ts * Update AdminFormManager.test.tsx.snap * Update DisplayRsuErrors.test.tsx --------- Co-authored-by: Drew Johnston <drewjj901@gmail.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Setting base path to absolute for refresh support * Combining GlobalExceptionHandlers * Fixing merge conflicts * Switching from Zonky to TestContainers * Update services/intersection-api/api/src/test/java/us/dot/its/jpo/ode/api/config/AbstractIntegrationTest.java Co-authored-by: Matt Cook <mattheworion.cook@gmail.com> * Update services/intersection-api/api/src/test/java/us/dot/its/jpo/ode/api/config/AbstractIntegrationTest.java Co-authored-by: Matt Cook <mattheworion.cook@gmail.com> * docs: initial requirements assessment * docs: refine requirements spec * chore: add .idea folder to gitignore * feat: introduce Admin Intersection API models, controller, and service stubs - Added admin intersection models (`AllowedSelections`, `Bbox`, `RefPt`, `IntersectionData`, `IntersectionListResponse`, `IntersectionSingleResponse`, `IntersectionPatch`) to support API functionality. - Created `AdminIntersectionController` with endpoints for managing intersections (`GET`, `PATCH`, `DELETE`). - Defined `AdminIntersectionService` stub with method placeholders for intersection data retrieval and updates. * fix: relocate auth to controller layer * feat(intersection): define missing repositories and methods Define missing relationships in Intersection.java and add new repository for IntersectionOrganizations. Add missing repository method to existing RsuIntersectionRepository * feat(intersection): add repository methods for intersection and RSU data retrieval - Added batch and single-fetch methods in `RsuIntersectionRepository` for RSU IPs by intersection numbers. - Introduced `IntersectionRepository` methods to load intersections with associated organizations for various use cases. - Added method in `OrganizationRepository` to find organizations by name for batch operations. * feat(intersection): replace IntersectionData with IntersectionDto and add IntersectionMapper - Replaced `IntersectionData` with `IntersectionDto` in `IntersectionSingleResponse` and `IntersectionListResponse` models. - Introduced `IntersectionMapper` using MapStruct for entity-to-DTO conversion, including custom mapping for organization names. * feat(intersection): add GeometryMapper for JTS geometry and admin DTO conversion - Introduced `GeometryMapper` using MapStruct for converting between JTS Point/Polygon and admin DTOs (`RefPt`, `Bbox`). - Included methods for transforming JTS geometry to DTO and vice versa with SRID 4326 geometry handling. * feat(intersection): implement service layer with unit tests for AdminIntersectionService methods - Implemented unit tests for methods in `AdminIntersectionService`, including `getAllIntersections`, `getIntersection`, `patchIntersection`, and `deleteIntersection`. - Verified context-based filtering of organizations and RSU IPs, as well as error handling for not found entities and transactional integrity during patch and delete operations. - Used Mockito for mocking dependencies and capturing interactions. * feat(intersection): enforce org restrictions in AdminIntersectionController - Updated `getIntersection` and `patchIntersection` methods to include organization restriction checks: - Enforced access control by validating organizations against the user's roles and permissions. - Added HTTP 403 response for unauthorized org modifications in `patchIntersection`. - Passed `isSuperUser`, `userOrgs`, and `operatorOrgs` to service methods for handling filtered data access. * refactor(intersection): use instance of GeometryMapper in AdminIntersectionService - Replaced static calls to `GeometryMapper` with the instance-based `geometryMapper` in `patchIntersection`. - Updated constructor to include `GeometryMapper` as a dependency. * refactor(tests): group related unit tests using @nested in AdminIntersectionServiceTest - Organized test methods into logical groups (`GetAllIntersections`, `GetIntersection`, `PatchIntersection`) using `@Nested` for improved readability and maintainability. - Removed outdated comments and unnecessary helper factories. * fix(tests): downgrade Testcontainers version and simplify AbstractIntegrationTest config - Downgraded `testcontainers` dependency in `pom.xml` from `2.0.3` to `1.21.4` for compatibility. - Corrected mismatched class and file name - Removed `@ServiceConnection` annotations and reorganized bean definitions. * feat(tests): add integration test profile for Intersection API - Introduced `application-integration-test.yaml` for service-layer integration tests. - Configured test containers for PostgreSQL using `tc:` JDBC URL and MongoDB via `@DynamicPropertySource`. - Defined Kafka topics and schemas, security, email, and other environment-specific settings for testing. * feat(tests): convert AdminIntersectionServiceTest to integration test with Testcontainers - Refactored `AdminIntersectionServiceTest` to use Spring Boot's integration testing annotations (`@SpringBootTest`, `@Testcontainers`, `@Transactional`). - Replaced Mockito mocks with actual repository beans for full stack validation. - Configured MongoDB `@Container` with `@DynamicPropertySource` for integration test environment. - Added helper methods and entity factories for persisted test data setup. - Simplified test cases by removing unnecessary mocking and stubbing. * chore: ignore IntelliJ IDEA `.iml` files in `.gitignore` * chore(tests): remove unused configs from integration test YAML - Deleted `hibernate.dialect` property as it is unused with Testcontainers' `tc:` JDBC driver. - Removed obsolete `haas` and `asn1-decoder` settings to streamline configuration. * fix(tests): downgrade Testcontainers version and simplify AbstractIntegrationTest config - Downgraded `testcontainers` dependency in `pom.xml` from `2.0.3` to `1.21.4` for compatibility. - Corrected mismatched class and file name - Removed `@ServiceConnection` annotations and reorganized bean definitions. * feat(tests): add integration test profile for Intersection API - Introduced `application-integration-test.yaml` for service-layer integration tests. - Configured test containers for PostgreSQL using `tc:` JDBC URL and MongoDB via `@DynamicPropertySource`. - Defined Kafka topics and schemas, security, email, and other environment-specific settings for testing. * refactor(tests): remove AbstractIntegrationTest and migrate tests to use Testcontainers MongoDB configuration - Deleted `AbstractIntegrationTest` class and associated PostgreSQL container setup. - Migrated all test classes to use standalone MongoDB containers with `@Testcontainers` and `@DynamicPropertySource`. - Updated application test profiles to reflect the removal of PostgreSQL references. - Simplified test setup and adjusted annotations for improved maintainability. * fix(tests): update Hibernate `ddl-auto` setting to `none` in integration test config - Changed `ddl-auto` property in `application-integration-test.yaml` to prevent schema recreation during tests. * feat: add conditional bean and property loading for ASN.1 decoders - Applied `@ConditionalOnBean` to all ASN.1 decoders and `DecoderManager` to ensure presence of `MessageFrameCodec` before initialization. - Added `@ConditionalOnProperty` in `CodecConfig` for enabling ASN.1 decoder configuration (`enable.asn1-decoder` flag). - Updated `Asn1Controller` with `@ConditionalOnBean(DecoderManager.class)` to ensure dependency injection. - Modified `application-integration-test.yaml` to include `asn1-decoder` configuration. * feat: centralize Testcontainers version with BOM and add Spring Boot Testcontainers dependency - Added `spring-boot-testcontainers` dependency for enhanced test configuration. - Introduced `dependencyManagement` with `testcontainers-bom` to manage Testcontainers version consistency. - Removed redundant explicit version declarations for Testcontainers dependencies. * fix(ci): still run tests if change limit exceeded This allows us to still see test results when we exceed file change limits in valid cases. * test(containers): use ServiceConnection to speed up integration tests * refactor(tests): migrate decoder tests to Mockito and simplify setup - Replaced Spring Boot test annotations with Mockito for improved test performance. - Adjusted test setup to use `@BeforeEach` for initializing decoders with mocked dependencies. - Updated package paths for decoder test classes. - Removed unused imports and redundant annotations. * Revert "refactor(tests): migrate decoder tests to Mockito and simplify setup" This reverts commit 17932ec. * refactor(tests): update package paths for decoder test classes - Moved decoder test classes from `decoderTests` to `asn1` package for correctness. - Removed unused imports across decoder test files. * refactor(tests): enable ASN.1 decoder property and simplify test setup - Added `@MockitoBean` for `MessageFrameCodec` in decoder tests to satisfy `@ConditionalOnBean` dependency. - Enabled `enable.asn1-decoder` property in `@SpringBootTest` annotations across test classes. - Removed unnecessary `@ActiveProfiles` and `@Import` annotations for cleaner configuration. * fix(tests): replaced constructor injection with `@Autowired` for `HaasController`. * refactor(tests): remove unused dependencies and redundant test setup in `AdminIntersectionServiceTest` - Eliminated `@Transactional`, `@DynamicPropertySource`, and `@Container` annotations. - Removed unused `EntityManager`, `MessageFrameCodec`, and related methods (`flushAndClear`, `uuid`, etc.). - Simplified test logic by eliminating redundant calls and improving clarity. * chore(config): remove zonky reference from comment * refactor(tests): migrate decoder tests to use Mockito and clean up setup - Replaced Spring Boot test annotations with Mockito for faster and lightweight tests. - Refactored test setup to initialize decoders with `@BeforeEach` and mocked dependencies. - Removed unused imports and redundant code in decoder test classes. - Simplified file reading logic within test setup blocks. * chore(tests): update `ddl-auto` property in test configuration - Changed Hibernate `ddl-auto` setting from `none` to `create-only` in `application-integration-test.yaml`. * chore(tests): reformat AdminIntersectionServiceTest * refactor(tests): introduce TestFixtures and replace redundant test setup in `AdminIntersectionServiceTest` - Added `TestFixtures` to centralize object creation for tests, using `javafaker` for randomized test data. - Replaced inline and duplicated setup logic with reusable methods from `TestFixtures`. - Updated `pom.xml` to add `javafaker` as a test dependency. * refactor(tests): add database cleanup before each test in `AdminIntersectionServiceTest` - Introduced `@BeforeEach` method to clear all relevant tables to ensure isolated test data. - Updated `@SpringBootTest` to use `webEnvironment = NONE`. - Removed unused imports and redundant `@Testcontainers` annotation. * feat(api): add ManufacturerRepository and integrate with AdminIntersectionServiceTest - Introduced `ManufacturerRepository` for managing `Manufacturer` entities. - Updated `AdminIntersectionServiceTest` to use `ManufacturerRepository` for saving manufacturers. - Changed `webEnvironment` in test configuration to `MOCK` for more realistic test scenarios. - Adjusted `application.yaml` to replace Hibernate dialect with `database: postgresql` for configuration consistency. * chore(tests): remove unused random object creation methods in TestFixtures - Deleted `randomPoint`, `createRandomIntersection`, `createRandomRsu`, and related unused methods in `TestFixtures`. - Removed redundant variable `orgAName` from `AdminIntersectionServiceTest`. * feat(models): add OpenAPI schema annotations to intersection-related classes - Added `@Schema` annotations with descriptions and examples for intersection domain models. - Introduced new `MessageResponse` class for simple write operation responses. - Improved model documentation for API clarity and OpenAPI spec generation. * feat(api): refactor AdminIntersectionController with optimized endpoints and MessageResponse integration - Replaced `@RequestMapping` with specific HTTP method annotations (`@GetMapping`, `@PatchMapping`, `@DeleteMapping`) for better clarity and alignment with REST conventions. - Added `@Parameter` annotations for improved OpenAPI documentation generation. - Updated method return types to use `MessageResponse` for consistency in write operation responses. - Simplified intersection query handling by separating `getAllIntersections` and `getIntersection` methods. - Refactored `patchIntersection` and `deleteIntersection` to eliminate redundant response entity wrapping. * test(api): add integration tests for AdminIntersectionController - Added comprehensive integration tests to cover `GET`, `PATCH`, and `DELETE` endpoints in `AdminIntersectionController`. - Included tests for authorization, validation, and service delegation. - Verified HTTP status codes and response shapes for various scenarios, including success and failure cases. * fix(api): remove unnecessary `matchIfMissing` in `@ConditionalOnProperty` * chore(docs): remove AdminIntersection migration spec - Deleted `admin_intersection_migration_spec.md` as it is no longer relevant. - The document was removed after the refactor was completed * chore(api): remove redundant endpoint comments in AdminIntersectionController * refactor(api): remove `MessageResponse` usage from AdminIntersectionController and associated services - Deleted `MessageResponse` class as it is no longer required. - Updated `patchIntersection` and `deleteIntersection` methods to return `void` instead of `MessageResponse`. - Adjusted related service methods to eliminate success message returns. - Updated tests to reflect the removal of `MessageResponse` and success message assertions. * test(api): add unit tests for GeometryMapper - Added `GeometryMapperTest` class to test mapping between JTS geometries (`Point`, `Polygon`) and DTOs (`RefPt`, `Bbox`). - Included tests for `toRefPt`, `toPoint`, `toBbox`, and `toPolygon` methods to verify mapping correctness. - Added round-trip tests to ensure data integrity across conversions. - Verified handling of edge cases, including null inputs and negative coordinates. * refactor(api): replace manual mappings in GeometryMapper with MapStruct annotations - Replaced custom `toRefPt` and `toBbox` methods with annotated MapStruct mappings. - Simplified mapping logic by delegating property mappings to `@Mapping` annotations. - Improved consistency and reduced manual code for geometry-to-DTO conversions. * test(api): add unit tests for IntersectionMapper and utility method in TestFixtures - Added `IntersectionMapperTest` to validate entity-to-DTO mappings, including edge cases like null inputs and missing fields. - Tested `toDto` and `mapOrgNames` methods for accuracy and coverage. - Added `createBBox` utility to `TestFixtures` for reusable mock bounding box creation. * chore(api): remove redundant comments from Intersection API models - Deleted outdated and unnecessary comments in `IntersectionListResponse`, `IntersectionPatch`, `IntersectionDto`, and `IntersectionSingleResponse`. - Retained meaningful annotations for clarity and accurate documentation generation. * refactor(api): improve exception handling in AdminIntersectionService - Replaced empty DTO responses with `EntityNotFoundException` for consistency. - Added `AccessDeniedException` for unauthorized access to intersections. - Updated service methods to throw descriptive exceptions instead of returning empty responses. - Adjusted test cases to validate the new exception behavior. * chore(api): remove unused imports and fields in IntersectionMapperTest and AdminIntersectionService * chore(api): remove redundant comments from repository and service classes - Deleted outdated and unnecessary comments across repository interfaces and `AdminIntersectionService`. - Retained meaningful descriptions to ensure clarity and maintain readability. * feat(api): add logging to AdminIntersectionController and AdminIntersectionService - Added `@Slf4j` to `AdminIntersectionService` for centralized logging. - Introduced info, debug, and error logs across controller and service methods to improve traceability. - Enhanced debugging output for intersection queries, patching, and deletions. * chore(api): update GeometryMapper javadoc to simplify description * feat(api): add validation annotations to intersection models and logging updates in AdminIntersectionController - Added `@NotNull`, `@DecimalMin`, and `@DecimalMax` annotations to `Bbox` to ensure latitude and longitude values are within valid ranges. - Applied `@Pattern` annotations for IPv4 validation in `IntersectionPatch`'s RSU IP address lists. - Enhanced AdminIntersectionController with validation for numeric `intersectionId` and organization membership checks. - Introduced logging and authorization validation for GET `/admin-intersection`. * fix(api): add additional API response annotations in AdminIntersectionController - Added `@ApiResponse` annotations for `405 Method Not Allowed` and `404 Not Found` scenarios. - Improved API documentation for intersection endpoints with new response descriptions. * fix(api): remove redundant authorization check in AdminIntersectionController - Eliminated unnecessary organization membership validation block in GET `/admin-intersection`. - Simplified controller logic while retaining existing logging and response mechanisms. * chore(api): fix package structure for ManufacturerRepository and update tests * refactor(api): replace String with Integer for intersectionId in DTOs, services, and tests - Updated `intersectionId` type from `String` to `Integer` across `IntersectionDto` and related models. - Simplified method parameters and repository queries to use `Integer` instead of `String`. - Refactored `AdminIntersectionController` and `AdminIntersectionService` to remove redundant org filtering and authorization logic. - Adjusted test cases to align with updated method signatures and data type changes. * chore(api): remove unused `@Pattern` import from AdminIntersectionController * refactor(tests): update intersectionId assertions to use Integer instead of String - Adjusted test cases in `AdminIntersectionControllerTest` to align with the updated `intersectionId` data type. - Refined `@DisplayName` in non-superuser intersection access test for clarity. * chore(api): add `@Repository` annotation to ManufacturerRepository * refactor(tests): adjust admin intersection service tests to confirm security - Added `@AfterEach` to clear security context after tests. - Introduced `setUpSuperuserContext` and `setUpOperatorContext` for simulating authorization scenarios. - Improved test cases to verify behavior for superuser and operator roles. - Updated assertions to validate intersection organization data. * chore(api): simplify javadoc in IntersectionSingleResponse model * fix(tests): update admin intersection tests to assert 404 for not found scenarios - Adjusted test cases to assert 404 status for non-existent intersections or inaccessible data. - Updated `@DisplayName` annotations for clarity in test descriptions. - Replaced mock service responses with `EntityNotFoundException` for missing data scenarios. * chore(api): remove redundant driver-class-name and improve javadoc in AdminIntersectionService * refactor(api): simplify admin intersection update logic with partial mapping - Replaced manual field mapping in `AdminIntersectionService` with `IntersectionMapper.partialUpdate`. - Added `partialUpdate` method to `IntersectionMapper` using `@BeanMapping` with `NullValuePropertyMappingStrategy.IGNORE`. * chore(tests): migrate test properties file from `.properties` to `.yaml` format * fix(api): enforce RSU permissions in `patchIntersection` endpoint - Added validation to ensure operators have access to RSUs being added or removed. - Introduced `rsuRepository` to support RSU permission checks. - Updated error handling to return 403 when RSU access is unauthorized. * fix(tests): update integration test config with MongoDB properties annotations * fix(tests): add RSU access validation tests for patchIntersection endpoint - Introduced tests to assert 403 responses when RSUs in `rsus_to_add` or `rsus_to_remove` are outside the user's permissions. - Added `RsuRepository` as a mock dependency in `AdminIntersectionControllerTest`. * refactor(api): enforce organization header in admin intersection API - Removed user-specific filtering logic in `AdminIntersectionService` for intersection retrieval. - Updated controller and service methods to require the Organization header for all requests. - Simplified logging and query logic for intersection retrieval. * feat(api): handle MissingRequestHeaderException with detailed error response - Added an exception handler for `MissingRequestHeaderException` in `GlobalExceptionHandler`. - Returns a structured `ProblemDetail` with a 400 status and header-specific details. - Introduced unit tests to validate error handling for missing request headers. * chore(tests): reformat mockMvc calls for readability * refactor(api): update admin intersection endpoint to remove "admin-" prefix - Standardized endpoint paths by removing "admin-" prefix in `AdminIntersectionController`. - Updated associated test cases, logging, and model annotations to reflect new endpoint paths. - Ensured backward compatibility by adjusting test assertions for all CRUD operations. * refactor(api): update intersection API endpoints to use pluralized paths - Changed `/intersection` to `/intersections` in `AdminIntersectionController` for consistency with RESTful conventions. - Updated all related test cases, logging statements, and model annotations to reflect the new endpoint paths. - Ensured proper alignment in test expectations and documentation comments for affected methods and classes. * refactor(api): add "admin/" prefix to intersection API endpoints - Updated endpoint paths in `AdminIntersectionController` to include "admin/" prefix for improved namespace clarity. - Adjusted related test cases, logging statements, and model annotations to reflect the new paths. - Ensured test expectations, comments, and API documentation were aligned with the updated endpoint structure. * refactor(api): update deleteIntersection endpoint to use PathVariable instead of RequestParam - Changed `intersection_id` in `deleteIntersection` endpoint from `RequestParam` to `PathVariable` for improved RESTful design. - Updated related test cases and logging to reflect the new path structure. * chore(api): remove unused imports from AdminIntersectionController * feat(api): prevent duplicate RSU-Intersection associations - Added unique constraint on `rsu_id` and `intersection_id` in `RsuIntersection` entity. - Introduced filter logic in `AdminIntersectionService` to skip creating duplicate RSU-Intersection associations. - Updated `RsuIntersectionRepository` with `existsByRsuAndIntersection` method. - Added a test case to verify that duplicate associations are not created. * feat(sql): add unique constraint for RSU and intersection IDs - Enforced a unique constraint on `(rsu_id, intersection_id)` in `RsuIntersection` table to prevent duplicate associations. --------- Co-authored-by: jacob6838 <jfrye@neaeraconsulting.com> Co-authored-by: Jacob Frye <51723791+jacob6838@users.noreply.github.com>
* Using decoded JWT data for simple user auth * Updating unit tests * Update services/intersection-api/api/src/main/java/us/dot/its/jpo/ode/api/services/PermissionService.java Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Cleaning up token decoding code, adding DecodedTokenTest * Debugging permission service test failures * Fixing PermissionService tests * Adding RequestScopedDecodedToken to cache decoded JWT tokens * Update services/intersection-api/api/src/main/java/us/dot/its/jpo/ode/api/models/keycloak/DecodedToken.java Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update services/intersection-api/api/src/main/java/us/dot/its/jpo/ode/api/models/keycloak/DecodedToken.java Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update services/intersection-api/api/src/main/java/us/dot/its/jpo/ode/api/models/keycloak/DecodedToken.java Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Adding AuthUtils * Update .vscode/launch.json Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update services/intersection-api/api/src/main/java/us/dot/its/jpo/ode/api/models/keycloak/DecodedToken.java Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update services/intersection-api/api/src/main/java/us/dot/its/jpo/ode/api/models/keycloak/RequestScopedDecodedToken.java Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update DecodedTokenTest.java * Documenting fields in DecodedToken * Adding IAPI token expiration validation * Adding hasRoleInOrgs * Update RsuControllerTest.java * Update RsuControllerTest.java * Switching from DecodedToken to CvManagerAuthToken * Combining GlobalExceptionHandlers and re-adding isSuperUser * Converting UserRole to enum * Resolving unit test errors * Adding UserRole enum refs * Re-formatting admin intersection files * Switching string user roles to enum * Potential fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * Resolving unit test warnings --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Setting base path to absolute for refresh support * Combining GlobalExceptionHandlers * Fixing merge conflicts * Switching from Zonky to TestContainers * Update services/intersection-api/api/src/test/java/us/dot/its/jpo/ode/api/config/AbstractIntegrationTest.java Co-authored-by: Matt Cook <mattheworion.cook@gmail.com> * Update services/intersection-api/api/src/test/java/us/dot/its/jpo/ode/api/config/AbstractIntegrationTest.java Co-authored-by: Matt Cook <mattheworion.cook@gmail.com> * docs: initial requirements assessment * docs: refine requirements spec * chore: add .idea folder to gitignore * feat: introduce Admin Intersection API models, controller, and service stubs - Added admin intersection models (`AllowedSelections`, `Bbox`, `RefPt`, `IntersectionData`, `IntersectionListResponse`, `IntersectionSingleResponse`, `IntersectionPatch`) to support API functionality. - Created `AdminIntersectionController` with endpoints for managing intersections (`GET`, `PATCH`, `DELETE`). - Defined `AdminIntersectionService` stub with method placeholders for intersection data retrieval and updates. * fix: relocate auth to controller layer * feat(intersection): define missing repositories and methods Define missing relationships in Intersection.java and add new repository for IntersectionOrganizations. Add missing repository method to existing RsuIntersectionRepository * feat(intersection): add repository methods for intersection and RSU data retrieval - Added batch and single-fetch methods in `RsuIntersectionRepository` for RSU IPs by intersection numbers. - Introduced `IntersectionRepository` methods to load intersections with associated organizations for various use cases. - Added method in `OrganizationRepository` to find organizations by name for batch operations. * feat(intersection): replace IntersectionData with IntersectionDto and add IntersectionMapper - Replaced `IntersectionData` with `IntersectionDto` in `IntersectionSingleResponse` and `IntersectionListResponse` models. - Introduced `IntersectionMapper` using MapStruct for entity-to-DTO conversion, including custom mapping for organization names. * feat(intersection): add GeometryMapper for JTS geometry and admin DTO conversion - Introduced `GeometryMapper` using MapStruct for converting between JTS Point/Polygon and admin DTOs (`RefPt`, `Bbox`). - Included methods for transforming JTS geometry to DTO and vice versa with SRID 4326 geometry handling. * feat(intersection): implement service layer with unit tests for AdminIntersectionService methods - Implemented unit tests for methods in `AdminIntersectionService`, including `getAllIntersections`, `getIntersection`, `patchIntersection`, and `deleteIntersection`. - Verified context-based filtering of organizations and RSU IPs, as well as error handling for not found entities and transactional integrity during patch and delete operations. - Used Mockito for mocking dependencies and capturing interactions. * feat(intersection): enforce org restrictions in AdminIntersectionController - Updated `getIntersection` and `patchIntersection` methods to include organization restriction checks: - Enforced access control by validating organizations against the user's roles and permissions. - Added HTTP 403 response for unauthorized org modifications in `patchIntersection`. - Passed `isSuperUser`, `userOrgs`, and `operatorOrgs` to service methods for handling filtered data access. * refactor(intersection): use instance of GeometryMapper in AdminIntersectionService - Replaced static calls to `GeometryMapper` with the instance-based `geometryMapper` in `patchIntersection`. - Updated constructor to include `GeometryMapper` as a dependency. * refactor(tests): group related unit tests using @nested in AdminIntersectionServiceTest - Organized test methods into logical groups (`GetAllIntersections`, `GetIntersection`, `PatchIntersection`) using `@Nested` for improved readability and maintainability. - Removed outdated comments and unnecessary helper factories. * fix(tests): downgrade Testcontainers version and simplify AbstractIntegrationTest config - Downgraded `testcontainers` dependency in `pom.xml` from `2.0.3` to `1.21.4` for compatibility. - Corrected mismatched class and file name - Removed `@ServiceConnection` annotations and reorganized bean definitions. * feat(tests): add integration test profile for Intersection API - Introduced `application-integration-test.yaml` for service-layer integration tests. - Configured test containers for PostgreSQL using `tc:` JDBC URL and MongoDB via `@DynamicPropertySource`. - Defined Kafka topics and schemas, security, email, and other environment-specific settings for testing. * feat(tests): convert AdminIntersectionServiceTest to integration test with Testcontainers - Refactored `AdminIntersectionServiceTest` to use Spring Boot's integration testing annotations (`@SpringBootTest`, `@Testcontainers`, `@Transactional`). - Replaced Mockito mocks with actual repository beans for full stack validation. - Configured MongoDB `@Container` with `@DynamicPropertySource` for integration test environment. - Added helper methods and entity factories for persisted test data setup. - Simplified test cases by removing unnecessary mocking and stubbing. * chore: ignore IntelliJ IDEA `.iml` files in `.gitignore` * chore(tests): remove unused configs from integration test YAML - Deleted `hibernate.dialect` property as it is unused with Testcontainers' `tc:` JDBC driver. - Removed obsolete `haas` and `asn1-decoder` settings to streamline configuration. * fix(tests): downgrade Testcontainers version and simplify AbstractIntegrationTest config - Downgraded `testcontainers` dependency in `pom.xml` from `2.0.3` to `1.21.4` for compatibility. - Corrected mismatched class and file name - Removed `@ServiceConnection` annotations and reorganized bean definitions. * feat(tests): add integration test profile for Intersection API - Introduced `application-integration-test.yaml` for service-layer integration tests. - Configured test containers for PostgreSQL using `tc:` JDBC URL and MongoDB via `@DynamicPropertySource`. - Defined Kafka topics and schemas, security, email, and other environment-specific settings for testing. * refactor(tests): remove AbstractIntegrationTest and migrate tests to use Testcontainers MongoDB configuration - Deleted `AbstractIntegrationTest` class and associated PostgreSQL container setup. - Migrated all test classes to use standalone MongoDB containers with `@Testcontainers` and `@DynamicPropertySource`. - Updated application test profiles to reflect the removal of PostgreSQL references. - Simplified test setup and adjusted annotations for improved maintainability. * fix(tests): update Hibernate `ddl-auto` setting to `none` in integration test config - Changed `ddl-auto` property in `application-integration-test.yaml` to prevent schema recreation during tests. * feat: add conditional bean and property loading for ASN.1 decoders - Applied `@ConditionalOnBean` to all ASN.1 decoders and `DecoderManager` to ensure presence of `MessageFrameCodec` before initialization. - Added `@ConditionalOnProperty` in `CodecConfig` for enabling ASN.1 decoder configuration (`enable.asn1-decoder` flag). - Updated `Asn1Controller` with `@ConditionalOnBean(DecoderManager.class)` to ensure dependency injection. - Modified `application-integration-test.yaml` to include `asn1-decoder` configuration. * feat: centralize Testcontainers version with BOM and add Spring Boot Testcontainers dependency - Added `spring-boot-testcontainers` dependency for enhanced test configuration. - Introduced `dependencyManagement` with `testcontainers-bom` to manage Testcontainers version consistency. - Removed redundant explicit version declarations for Testcontainers dependencies. * fix(ci): still run tests if change limit exceeded This allows us to still see test results when we exceed file change limits in valid cases. * test(containers): use ServiceConnection to speed up integration tests * refactor(tests): migrate decoder tests to Mockito and simplify setup - Replaced Spring Boot test annotations with Mockito for improved test performance. - Adjusted test setup to use `@BeforeEach` for initializing decoders with mocked dependencies. - Updated package paths for decoder test classes. - Removed unused imports and redundant annotations. * Revert "refactor(tests): migrate decoder tests to Mockito and simplify setup" This reverts commit 17932ec. * refactor(tests): update package paths for decoder test classes - Moved decoder test classes from `decoderTests` to `asn1` package for correctness. - Removed unused imports across decoder test files. * refactor(tests): enable ASN.1 decoder property and simplify test setup - Added `@MockitoBean` for `MessageFrameCodec` in decoder tests to satisfy `@ConditionalOnBean` dependency. - Enabled `enable.asn1-decoder` property in `@SpringBootTest` annotations across test classes. - Removed unnecessary `@ActiveProfiles` and `@Import` annotations for cleaner configuration. * fix(tests): replaced constructor injection with `@Autowired` for `HaasController`. * refactor(tests): remove unused dependencies and redundant test setup in `AdminIntersectionServiceTest` - Eliminated `@Transactional`, `@DynamicPropertySource`, and `@Container` annotations. - Removed unused `EntityManager`, `MessageFrameCodec`, and related methods (`flushAndClear`, `uuid`, etc.). - Simplified test logic by eliminating redundant calls and improving clarity. * chore(config): remove zonky reference from comment * refactor(tests): migrate decoder tests to use Mockito and clean up setup - Replaced Spring Boot test annotations with Mockito for faster and lightweight tests. - Refactored test setup to initialize decoders with `@BeforeEach` and mocked dependencies. - Removed unused imports and redundant code in decoder test classes. - Simplified file reading logic within test setup blocks. * chore(tests): update `ddl-auto` property in test configuration - Changed Hibernate `ddl-auto` setting from `none` to `create-only` in `application-integration-test.yaml`. * chore(tests): reformat AdminIntersectionServiceTest * refactor(tests): introduce TestFixtures and replace redundant test setup in `AdminIntersectionServiceTest` - Added `TestFixtures` to centralize object creation for tests, using `javafaker` for randomized test data. - Replaced inline and duplicated setup logic with reusable methods from `TestFixtures`. - Updated `pom.xml` to add `javafaker` as a test dependency. * refactor(tests): add database cleanup before each test in `AdminIntersectionServiceTest` - Introduced `@BeforeEach` method to clear all relevant tables to ensure isolated test data. - Updated `@SpringBootTest` to use `webEnvironment = NONE`. - Removed unused imports and redundant `@Testcontainers` annotation. * feat(api): add ManufacturerRepository and integrate with AdminIntersectionServiceTest - Introduced `ManufacturerRepository` for managing `Manufacturer` entities. - Updated `AdminIntersectionServiceTest` to use `ManufacturerRepository` for saving manufacturers. - Changed `webEnvironment` in test configuration to `MOCK` for more realistic test scenarios. - Adjusted `application.yaml` to replace Hibernate dialect with `database: postgresql` for configuration consistency. * chore(tests): remove unused random object creation methods in TestFixtures - Deleted `randomPoint`, `createRandomIntersection`, `createRandomRsu`, and related unused methods in `TestFixtures`. - Removed redundant variable `orgAName` from `AdminIntersectionServiceTest`. * feat(models): add OpenAPI schema annotations to intersection-related classes - Added `@Schema` annotations with descriptions and examples for intersection domain models. - Introduced new `MessageResponse` class for simple write operation responses. - Improved model documentation for API clarity and OpenAPI spec generation. * feat(api): refactor AdminIntersectionController with optimized endpoints and MessageResponse integration - Replaced `@RequestMapping` with specific HTTP method annotations (`@GetMapping`, `@PatchMapping`, `@DeleteMapping`) for better clarity and alignment with REST conventions. - Added `@Parameter` annotations for improved OpenAPI documentation generation. - Updated method return types to use `MessageResponse` for consistency in write operation responses. - Simplified intersection query handling by separating `getAllIntersections` and `getIntersection` methods. - Refactored `patchIntersection` and `deleteIntersection` to eliminate redundant response entity wrapping. * test(api): add integration tests for AdminIntersectionController - Added comprehensive integration tests to cover `GET`, `PATCH`, and `DELETE` endpoints in `AdminIntersectionController`. - Included tests for authorization, validation, and service delegation. - Verified HTTP status codes and response shapes for various scenarios, including success and failure cases. * fix(api): remove unnecessary `matchIfMissing` in `@ConditionalOnProperty` * chore(docs): remove AdminIntersection migration spec - Deleted `admin_intersection_migration_spec.md` as it is no longer relevant. - The document was removed after the refactor was completed * chore(api): remove redundant endpoint comments in AdminIntersectionController * refactor(api): remove `MessageResponse` usage from AdminIntersectionController and associated services - Deleted `MessageResponse` class as it is no longer required. - Updated `patchIntersection` and `deleteIntersection` methods to return `void` instead of `MessageResponse`. - Adjusted related service methods to eliminate success message returns. - Updated tests to reflect the removal of `MessageResponse` and success message assertions. * test(api): add unit tests for GeometryMapper - Added `GeometryMapperTest` class to test mapping between JTS geometries (`Point`, `Polygon`) and DTOs (`RefPt`, `Bbox`). - Included tests for `toRefPt`, `toPoint`, `toBbox`, and `toPolygon` methods to verify mapping correctness. - Added round-trip tests to ensure data integrity across conversions. - Verified handling of edge cases, including null inputs and negative coordinates. * refactor(api): replace manual mappings in GeometryMapper with MapStruct annotations - Replaced custom `toRefPt` and `toBbox` methods with annotated MapStruct mappings. - Simplified mapping logic by delegating property mappings to `@Mapping` annotations. - Improved consistency and reduced manual code for geometry-to-DTO conversions. * test(api): add unit tests for IntersectionMapper and utility method in TestFixtures - Added `IntersectionMapperTest` to validate entity-to-DTO mappings, including edge cases like null inputs and missing fields. - Tested `toDto` and `mapOrgNames` methods for accuracy and coverage. - Added `createBBox` utility to `TestFixtures` for reusable mock bounding box creation. * chore(api): remove redundant comments from Intersection API models - Deleted outdated and unnecessary comments in `IntersectionListResponse`, `IntersectionPatch`, `IntersectionDto`, and `IntersectionSingleResponse`. - Retained meaningful annotations for clarity and accurate documentation generation. * refactor(api): improve exception handling in AdminIntersectionService - Replaced empty DTO responses with `EntityNotFoundException` for consistency. - Added `AccessDeniedException` for unauthorized access to intersections. - Updated service methods to throw descriptive exceptions instead of returning empty responses. - Adjusted test cases to validate the new exception behavior. * chore(api): remove unused imports and fields in IntersectionMapperTest and AdminIntersectionService * chore(api): remove redundant comments from repository and service classes - Deleted outdated and unnecessary comments across repository interfaces and `AdminIntersectionService`. - Retained meaningful descriptions to ensure clarity and maintain readability. * feat(api): add logging to AdminIntersectionController and AdminIntersectionService - Added `@Slf4j` to `AdminIntersectionService` for centralized logging. - Introduced info, debug, and error logs across controller and service methods to improve traceability. - Enhanced debugging output for intersection queries, patching, and deletions. * chore(api): update GeometryMapper javadoc to simplify description * feat(api): add validation annotations to intersection models and logging updates in AdminIntersectionController - Added `@NotNull`, `@DecimalMin`, and `@DecimalMax` annotations to `Bbox` to ensure latitude and longitude values are within valid ranges. - Applied `@Pattern` annotations for IPv4 validation in `IntersectionPatch`'s RSU IP address lists. - Enhanced AdminIntersectionController with validation for numeric `intersectionId` and organization membership checks. - Introduced logging and authorization validation for GET `/admin-intersection`. * refactor(api): remove AdminIntersection python feature and tests - Deleted `AdminIntersection` feature and its routes in `main.py`. - Removed `test_admin_intersection.py` and tests related to `AdminIntersection` functionalities. - Updated references across the codebase to reflect the removal. * refactor(api): remove AdminIntersection feature implementation and related logic - Deleted `admin_intersection.py` and all associated helper functions, schemas, and endpoint logic. - Removed database query methods and authorization-related utilities specific to the AdminIntersection feature. - Cleaned up unnecessary imports and code dependencies related to this functionality. * chore(webapp): update package-lock.json to adjust `peer` field entries - Removed redundant `peer` fields across various dependencies. - Added missing `peer` fields for consistency. * fix(api): add additional API response annotations in AdminIntersectionController - Added `@ApiResponse` annotations for `405 Method Not Allowed` and `404 Not Found` scenarios. - Improved API documentation for intersection endpoints with new response descriptions. * fix(api): remove redundant authorization check in AdminIntersectionController - Eliminated unnecessary organization membership validation block in GET `/admin-intersection`. - Simplified controller logic while retaining existing logging and response mechanisms. * chore(api): fix package structure for ManufacturerRepository and update tests * refactor(api): replace String with Integer for intersectionId in DTOs, services, and tests - Updated `intersectionId` type from `String` to `Integer` across `IntersectionDto` and related models. - Simplified method parameters and repository queries to use `Integer` instead of `String`. - Refactored `AdminIntersectionController` and `AdminIntersectionService` to remove redundant org filtering and authorization logic. - Adjusted test cases to align with updated method signatures and data type changes. * chore(api): remove unused `@Pattern` import from AdminIntersectionController * refactor(tests): update intersectionId assertions to use Integer instead of String - Adjusted test cases in `AdminIntersectionControllerTest` to align with the updated `intersectionId` data type. - Refined `@DisplayName` in non-superuser intersection access test for clarity. * chore(api): add `@Repository` annotation to ManufacturerRepository * refactor(tests): adjust admin intersection service tests to confirm security - Added `@AfterEach` to clear security context after tests. - Introduced `setUpSuperuserContext` and `setUpOperatorContext` for simulating authorization scenarios. - Improved test cases to verify behavior for superuser and operator roles. - Updated assertions to validate intersection organization data. * chore(api): simplify javadoc in IntersectionSingleResponse model * fix(tests): update admin intersection tests to assert 404 for not found scenarios - Adjusted test cases to assert 404 status for non-existent intersections or inaccessible data. - Updated `@DisplayName` annotations for clarity in test descriptions. - Replaced mock service responses with `EntityNotFoundException` for missing data scenarios. * chore(api): remove redundant driver-class-name and improve javadoc in AdminIntersectionService * refactor(api): simplify admin intersection update logic with partial mapping - Replaced manual field mapping in `AdminIntersectionService` with `IntersectionMapper.partialUpdate`. - Added `partialUpdate` method to `IntersectionMapper` using `@BeanMapping` with `NullValuePropertyMappingStrategy.IGNORE`. * chore(tests): migrate test properties file from `.properties` to `.yaml` format * fix(api): enforce RSU permissions in `patchIntersection` endpoint - Added validation to ensure operators have access to RSUs being added or removed. - Introduced `rsuRepository` to support RSU permission checks. - Updated error handling to return 403 when RSU access is unauthorized. * fix(tests): update integration test config with MongoDB properties annotations * fix(tests): add RSU access validation tests for patchIntersection endpoint - Introduced tests to assert 403 responses when RSUs in `rsus_to_add` or `rsus_to_remove` are outside the user's permissions. - Added `RsuRepository` as a mock dependency in `AdminIntersectionControllerTest`. * refactor(api): enforce organization header in admin intersection API - Removed user-specific filtering logic in `AdminIntersectionService` for intersection retrieval. - Updated controller and service methods to require the Organization header for all requests. - Simplified logging and query logic for intersection retrieval. * feat(api): handle MissingRequestHeaderException with detailed error response - Added an exception handler for `MissingRequestHeaderException` in `GlobalExceptionHandler`. - Returns a structured `ProblemDetail` with a 400 status and header-specific details. - Introduced unit tests to validate error handling for missing request headers. * chore(tests): reformat mockMvc calls for readability * refactor(api): remove unnecessary query parameters from admin intersection API calls - Updated `_getDataWithCodes` and `_patchData` calls to exclude redundant `query_params`. - Appended `intersection_id` directly to the URL for consistency and simplicity. - Normalized `intersection_id` to string during state updates to ensure uniformity in data handling. * refactor(api): update admin intersection endpoint to remove "admin-" prefix - Standardized endpoint paths by removing "admin-" prefix in `AdminIntersectionController`. - Updated associated test cases, logging, and model annotations to reflect new endpoint paths. - Ensured backward compatibility by adjusting test assertions for all CRUD operations. * refactor(api): update intersection API endpoints to use pluralized paths - Changed `/intersection` to `/intersections` in `AdminIntersectionController` for consistency with RESTful conventions. - Updated all related test cases, logging statements, and model annotations to reflect the new endpoint paths. - Ensured proper alignment in test expectations and documentation comments for affected methods and classes. * refactor(api): add "admin/" prefix to intersection API endpoints - Updated endpoint paths in `AdminIntersectionController` to include "admin/" prefix for improved namespace clarity. - Adjusted related test cases, logging statements, and model annotations to reflect the new paths. - Ensured test expectations, comments, and API documentation were aligned with the updated endpoint structure. * refactor(webapp): update admin intersection endpoint to use pluralized path * refactor(api): update deleteIntersection endpoint to use PathVariable instead of RequestParam - Changed `intersection_id` in `deleteIntersection` endpoint from `RequestParam` to `PathVariable` for improved RESTful design. - Updated related test cases and logging to reflect the new path structure. * chore(api): remove unused imports from AdminIntersectionController * feat(api): prevent duplicate RSU-Intersection associations - Added unique constraint on `rsu_id` and `intersection_id` in `RsuIntersection` entity. - Introduced filter logic in `AdminIntersectionService` to skip creating duplicate RSU-Intersection associations. - Updated `RsuIntersectionRepository` with `existsByRsuAndIntersection` method. - Added a test case to verify that duplicate associations are not created. * feat(sql): add unique constraint for RSU and intersection IDs - Enforced a unique constraint on `(rsu_id, intersection_id)` in `RsuIntersection` table to prevent duplicate associations. * refactor(webapp): update delete intersection API to use path variables - Updated `adminIntersectionTabSlice` to pass `intersection_id` in the URL path instead of query parameters. - Adjusted test cases to align with the new path structure. * refactor(webapp): remove unused async thunks and streamline admin intersection logic - Removed `getIntersectionInfo`, `editIntersection`, and `submitForm` async thunks from `adminEditIntersectionSlice`. - Replaced action thunks with RTK Query APIs for admin intersection CRUD operations. - Refactored related components and tests to align with the new RTK Query integration. - Eliminated redundant `loading` state and selectors for cleaner state management. * chore(webapp): update sample env to use localhost over DOCKER_HOST_IP * feat(webapp): add RTK Query API slice for admin intersection management - Introduced `adminIntersectionApiSlice` for CRUD operations related to admin intersections. - Added queries and mutations: getIntersections, getIntersection, fetch allowed selections, create, patch, and delete intersections. - Implemented tag-based caching and invalidation for efficient state updates. * fix(webapp): ensure modal closure and navigation on successful intersection update * feat(webapp): add organization header to admin intersections API calls - Updated `prepareHeaders` to include the `Organization` header alongside the `Authorization` token. - Refactored `getIntersections` query to remove redundant organization header logic. * fix(webapp): ensure immutability when processing intersection form data * test(webapp): remove eslint-disable directive * test(webapp): remove unused eslint-disable directive in adminIntersectionTabSlice test * test(webapp): update adminEditIntersection tests for form validation and JSON mapping adjustments * refactor(webapp): simplify form validation in adminEditIntersectionSlice * test(webapp): add tests for adminIntersectionApiSlice, covering CRUD operations and cache behavior * refactor(webapp): use constants for adminIntersection API tag and list ID * refactor(webapp): improve toast handling for delete operations in AdminIntersectionTab * refactor(webapp): adjust adminIntersection API to remove organization header from token setup * test(webapp): remove assertion for organization header in adminIntersectionApiSlice test * fix(api): return 200 OK response for empty results from getAllIntersections The 404 response was considered an error by the UI framework, so it prevented the UI from reloading after deleting the last intersection in the list * chore(api): remove unused `/admin-intersection` from middleware.py * feat(api/webapp): enhance error handling for intersections updates - Updated `GlobalExceptionHandler` to categorize errors containing "intersection." - Improved toast notifications in the web app to display specific error details during intersection updates. * feat(api): add content-type and accept headers in adminIntersectionApiSlice - Ensured `Content-Type` and `Accept` headers are set to `application/json` in API requests to standardize data handling and improve compatibility. * fix(api): remove "A" from start of sentence and resource type capitalization in GlobalExceptionHandler - Updated resource type capitalization for consistency. - Removed prefixed "A" from start of sentence to maintain correct grammar in messages when resource starts with a vowel * fix(tests): remove 'A' from string assertion in testRsuDuplicateSerialNumber --------- Co-authored-by: jacob6838 <jfrye@neaeraconsulting.com> Co-authored-by: Jacob Frye <51723791+jacob6838@users.noreply.github.com>
* Setting base path to absolute for refresh support * Combining GlobalExceptionHandlers * Fixing merge conflicts * Switching from Zonky to TestContainers * Update services/intersection-api/api/src/test/java/us/dot/its/jpo/ode/api/config/AbstractIntegrationTest.java Co-authored-by: Matt Cook <mattheworion.cook@gmail.com> * Update services/intersection-api/api/src/test/java/us/dot/its/jpo/ode/api/config/AbstractIntegrationTest.java Co-authored-by: Matt Cook <mattheworion.cook@gmail.com> * docs: initial requirements assessment * docs: refine requirements spec * chore: add .idea folder to gitignore * feat: introduce Admin Intersection API models, controller, and service stubs - Added admin intersection models (`AllowedSelections`, `Bbox`, `RefPt`, `IntersectionData`, `IntersectionListResponse`, `IntersectionSingleResponse`, `IntersectionPatch`) to support API functionality. - Created `AdminIntersectionController` with endpoints for managing intersections (`GET`, `PATCH`, `DELETE`). - Defined `AdminIntersectionService` stub with method placeholders for intersection data retrieval and updates. * fix: relocate auth to controller layer * feat(intersection): define missing repositories and methods Define missing relationships in Intersection.java and add new repository for IntersectionOrganizations. Add missing repository method to existing RsuIntersectionRepository * feat(intersection): add repository methods for intersection and RSU data retrieval - Added batch and single-fetch methods in `RsuIntersectionRepository` for RSU IPs by intersection numbers. - Introduced `IntersectionRepository` methods to load intersections with associated organizations for various use cases. - Added method in `OrganizationRepository` to find organizations by name for batch operations. * feat(intersection): replace IntersectionData with IntersectionDto and add IntersectionMapper - Replaced `IntersectionData` with `IntersectionDto` in `IntersectionSingleResponse` and `IntersectionListResponse` models. - Introduced `IntersectionMapper` using MapStruct for entity-to-DTO conversion, including custom mapping for organization names. * feat(intersection): add GeometryMapper for JTS geometry and admin DTO conversion - Introduced `GeometryMapper` using MapStruct for converting between JTS Point/Polygon and admin DTOs (`RefPt`, `Bbox`). - Included methods for transforming JTS geometry to DTO and vice versa with SRID 4326 geometry handling. * feat(intersection): implement service layer with unit tests for AdminIntersectionService methods - Implemented unit tests for methods in `AdminIntersectionService`, including `getAllIntersections`, `getIntersection`, `patchIntersection`, and `deleteIntersection`. - Verified context-based filtering of organizations and RSU IPs, as well as error handling for not found entities and transactional integrity during patch and delete operations. - Used Mockito for mocking dependencies and capturing interactions. * feat(intersection): enforce org restrictions in AdminIntersectionController - Updated `getIntersection` and `patchIntersection` methods to include organization restriction checks: - Enforced access control by validating organizations against the user's roles and permissions. - Added HTTP 403 response for unauthorized org modifications in `patchIntersection`. - Passed `isSuperUser`, `userOrgs`, and `operatorOrgs` to service methods for handling filtered data access. * refactor(intersection): use instance of GeometryMapper in AdminIntersectionService - Replaced static calls to `GeometryMapper` with the instance-based `geometryMapper` in `patchIntersection`. - Updated constructor to include `GeometryMapper` as a dependency. * refactor(tests): group related unit tests using @nested in AdminIntersectionServiceTest - Organized test methods into logical groups (`GetAllIntersections`, `GetIntersection`, `PatchIntersection`) using `@Nested` for improved readability and maintainability. - Removed outdated comments and unnecessary helper factories. * fix(tests): downgrade Testcontainers version and simplify AbstractIntegrationTest config - Downgraded `testcontainers` dependency in `pom.xml` from `2.0.3` to `1.21.4` for compatibility. - Corrected mismatched class and file name - Removed `@ServiceConnection` annotations and reorganized bean definitions. * feat(tests): add integration test profile for Intersection API - Introduced `application-integration-test.yaml` for service-layer integration tests. - Configured test containers for PostgreSQL using `tc:` JDBC URL and MongoDB via `@DynamicPropertySource`. - Defined Kafka topics and schemas, security, email, and other environment-specific settings for testing. * feat(tests): convert AdminIntersectionServiceTest to integration test with Testcontainers - Refactored `AdminIntersectionServiceTest` to use Spring Boot's integration testing annotations (`@SpringBootTest`, `@Testcontainers`, `@Transactional`). - Replaced Mockito mocks with actual repository beans for full stack validation. - Configured MongoDB `@Container` with `@DynamicPropertySource` for integration test environment. - Added helper methods and entity factories for persisted test data setup. - Simplified test cases by removing unnecessary mocking and stubbing. * chore: ignore IntelliJ IDEA `.iml` files in `.gitignore` * chore(tests): remove unused configs from integration test YAML - Deleted `hibernate.dialect` property as it is unused with Testcontainers' `tc:` JDBC driver. - Removed obsolete `haas` and `asn1-decoder` settings to streamline configuration. * fix(tests): downgrade Testcontainers version and simplify AbstractIntegrationTest config - Downgraded `testcontainers` dependency in `pom.xml` from `2.0.3` to `1.21.4` for compatibility. - Corrected mismatched class and file name - Removed `@ServiceConnection` annotations and reorganized bean definitions. * feat(tests): add integration test profile for Intersection API - Introduced `application-integration-test.yaml` for service-layer integration tests. - Configured test containers for PostgreSQL using `tc:` JDBC URL and MongoDB via `@DynamicPropertySource`. - Defined Kafka topics and schemas, security, email, and other environment-specific settings for testing. * refactor(tests): remove AbstractIntegrationTest and migrate tests to use Testcontainers MongoDB configuration - Deleted `AbstractIntegrationTest` class and associated PostgreSQL container setup. - Migrated all test classes to use standalone MongoDB containers with `@Testcontainers` and `@DynamicPropertySource`. - Updated application test profiles to reflect the removal of PostgreSQL references. - Simplified test setup and adjusted annotations for improved maintainability. * fix(tests): update Hibernate `ddl-auto` setting to `none` in integration test config - Changed `ddl-auto` property in `application-integration-test.yaml` to prevent schema recreation during tests. * feat: add conditional bean and property loading for ASN.1 decoders - Applied `@ConditionalOnBean` to all ASN.1 decoders and `DecoderManager` to ensure presence of `MessageFrameCodec` before initialization. - Added `@ConditionalOnProperty` in `CodecConfig` for enabling ASN.1 decoder configuration (`enable.asn1-decoder` flag). - Updated `Asn1Controller` with `@ConditionalOnBean(DecoderManager.class)` to ensure dependency injection. - Modified `application-integration-test.yaml` to include `asn1-decoder` configuration. * feat: centralize Testcontainers version with BOM and add Spring Boot Testcontainers dependency - Added `spring-boot-testcontainers` dependency for enhanced test configuration. - Introduced `dependencyManagement` with `testcontainers-bom` to manage Testcontainers version consistency. - Removed redundant explicit version declarations for Testcontainers dependencies. * fix(ci): still run tests if change limit exceeded This allows us to still see test results when we exceed file change limits in valid cases. * test(containers): use ServiceConnection to speed up integration tests * refactor(tests): migrate decoder tests to Mockito and simplify setup - Replaced Spring Boot test annotations with Mockito for improved test performance. - Adjusted test setup to use `@BeforeEach` for initializing decoders with mocked dependencies. - Updated package paths for decoder test classes. - Removed unused imports and redundant annotations. * Revert "refactor(tests): migrate decoder tests to Mockito and simplify setup" This reverts commit 17932ec. * refactor(tests): update package paths for decoder test classes - Moved decoder test classes from `decoderTests` to `asn1` package for correctness. - Removed unused imports across decoder test files. * refactor(tests): enable ASN.1 decoder property and simplify test setup - Added `@MockitoBean` for `MessageFrameCodec` in decoder tests to satisfy `@ConditionalOnBean` dependency. - Enabled `enable.asn1-decoder` property in `@SpringBootTest` annotations across test classes. - Removed unnecessary `@ActiveProfiles` and `@Import` annotations for cleaner configuration. * fix(tests): replaced constructor injection with `@Autowired` for `HaasController`. * refactor(tests): remove unused dependencies and redundant test setup in `AdminIntersectionServiceTest` - Eliminated `@Transactional`, `@DynamicPropertySource`, and `@Container` annotations. - Removed unused `EntityManager`, `MessageFrameCodec`, and related methods (`flushAndClear`, `uuid`, etc.). - Simplified test logic by eliminating redundant calls and improving clarity. * chore(config): remove zonky reference from comment * refactor(tests): migrate decoder tests to use Mockito and clean up setup - Replaced Spring Boot test annotations with Mockito for faster and lightweight tests. - Refactored test setup to initialize decoders with `@BeforeEach` and mocked dependencies. - Removed unused imports and redundant code in decoder test classes. - Simplified file reading logic within test setup blocks. * chore(tests): update `ddl-auto` property in test configuration - Changed Hibernate `ddl-auto` setting from `none` to `create-only` in `application-integration-test.yaml`. * chore(tests): reformat AdminIntersectionServiceTest * refactor(tests): introduce TestFixtures and replace redundant test setup in `AdminIntersectionServiceTest` - Added `TestFixtures` to centralize object creation for tests, using `javafaker` for randomized test data. - Replaced inline and duplicated setup logic with reusable methods from `TestFixtures`. - Updated `pom.xml` to add `javafaker` as a test dependency. * refactor(tests): add database cleanup before each test in `AdminIntersectionServiceTest` - Introduced `@BeforeEach` method to clear all relevant tables to ensure isolated test data. - Updated `@SpringBootTest` to use `webEnvironment = NONE`. - Removed unused imports and redundant `@Testcontainers` annotation. * feat(api): add ManufacturerRepository and integrate with AdminIntersectionServiceTest - Introduced `ManufacturerRepository` for managing `Manufacturer` entities. - Updated `AdminIntersectionServiceTest` to use `ManufacturerRepository` for saving manufacturers. - Changed `webEnvironment` in test configuration to `MOCK` for more realistic test scenarios. - Adjusted `application.yaml` to replace Hibernate dialect with `database: postgresql` for configuration consistency. * chore(tests): remove unused random object creation methods in TestFixtures - Deleted `randomPoint`, `createRandomIntersection`, `createRandomRsu`, and related unused methods in `TestFixtures`. - Removed redundant variable `orgAName` from `AdminIntersectionServiceTest`. * feat(models): add OpenAPI schema annotations to intersection-related classes - Added `@Schema` annotations with descriptions and examples for intersection domain models. - Introduced new `MessageResponse` class for simple write operation responses. - Improved model documentation for API clarity and OpenAPI spec generation. * feat(api): refactor AdminIntersectionController with optimized endpoints and MessageResponse integration - Replaced `@RequestMapping` with specific HTTP method annotations (`@GetMapping`, `@PatchMapping`, `@DeleteMapping`) for better clarity and alignment with REST conventions. - Added `@Parameter` annotations for improved OpenAPI documentation generation. - Updated method return types to use `MessageResponse` for consistency in write operation responses. - Simplified intersection query handling by separating `getAllIntersections` and `getIntersection` methods. - Refactored `patchIntersection` and `deleteIntersection` to eliminate redundant response entity wrapping. * test(api): add integration tests for AdminIntersectionController - Added comprehensive integration tests to cover `GET`, `PATCH`, and `DELETE` endpoints in `AdminIntersectionController`. - Included tests for authorization, validation, and service delegation. - Verified HTTP status codes and response shapes for various scenarios, including success and failure cases. * fix(api): remove unnecessary `matchIfMissing` in `@ConditionalOnProperty` * chore(docs): remove AdminIntersection migration spec - Deleted `admin_intersection_migration_spec.md` as it is no longer relevant. - The document was removed after the refactor was completed * chore(api): remove redundant endpoint comments in AdminIntersectionController * refactor(api): remove `MessageResponse` usage from AdminIntersectionController and associated services - Deleted `MessageResponse` class as it is no longer required. - Updated `patchIntersection` and `deleteIntersection` methods to return `void` instead of `MessageResponse`. - Adjusted related service methods to eliminate success message returns. - Updated tests to reflect the removal of `MessageResponse` and success message assertions. * test(api): add unit tests for GeometryMapper - Added `GeometryMapperTest` class to test mapping between JTS geometries (`Point`, `Polygon`) and DTOs (`RefPt`, `Bbox`). - Included tests for `toRefPt`, `toPoint`, `toBbox`, and `toPolygon` methods to verify mapping correctness. - Added round-trip tests to ensure data integrity across conversions. - Verified handling of edge cases, including null inputs and negative coordinates. * refactor(api): replace manual mappings in GeometryMapper with MapStruct annotations - Replaced custom `toRefPt` and `toBbox` methods with annotated MapStruct mappings. - Simplified mapping logic by delegating property mappings to `@Mapping` annotations. - Improved consistency and reduced manual code for geometry-to-DTO conversions. * test(api): add unit tests for IntersectionMapper and utility method in TestFixtures - Added `IntersectionMapperTest` to validate entity-to-DTO mappings, including edge cases like null inputs and missing fields. - Tested `toDto` and `mapOrgNames` methods for accuracy and coverage. - Added `createBBox` utility to `TestFixtures` for reusable mock bounding box creation. * chore(api): remove redundant comments from Intersection API models - Deleted outdated and unnecessary comments in `IntersectionListResponse`, `IntersectionPatch`, `IntersectionDto`, and `IntersectionSingleResponse`. - Retained meaningful annotations for clarity and accurate documentation generation. * refactor(api): improve exception handling in AdminIntersectionService - Replaced empty DTO responses with `EntityNotFoundException` for consistency. - Added `AccessDeniedException` for unauthorized access to intersections. - Updated service methods to throw descriptive exceptions instead of returning empty responses. - Adjusted test cases to validate the new exception behavior. * chore(api): remove unused imports and fields in IntersectionMapperTest and AdminIntersectionService * chore(api): remove redundant comments from repository and service classes - Deleted outdated and unnecessary comments across repository interfaces and `AdminIntersectionService`. - Retained meaningful descriptions to ensure clarity and maintain readability. * feat(api): add logging to AdminIntersectionController and AdminIntersectionService - Added `@Slf4j` to `AdminIntersectionService` for centralized logging. - Introduced info, debug, and error logs across controller and service methods to improve traceability. - Enhanced debugging output for intersection queries, patching, and deletions. * chore(api): update GeometryMapper javadoc to simplify description * feat(api): add validation annotations to intersection models and logging updates in AdminIntersectionController - Added `@NotNull`, `@DecimalMin`, and `@DecimalMax` annotations to `Bbox` to ensure latitude and longitude values are within valid ranges. - Applied `@Pattern` annotations for IPv4 validation in `IntersectionPatch`'s RSU IP address lists. - Enhanced AdminIntersectionController with validation for numeric `intersectionId` and organization membership checks. - Introduced logging and authorization validation for GET `/admin-intersection`. * refactor(api): remove AdminIntersection python feature and tests - Deleted `AdminIntersection` feature and its routes in `main.py`. - Removed `test_admin_intersection.py` and tests related to `AdminIntersection` functionalities. - Updated references across the codebase to reflect the removal. * refactor(api): remove AdminIntersection feature implementation and related logic - Deleted `admin_intersection.py` and all associated helper functions, schemas, and endpoint logic. - Removed database query methods and authorization-related utilities specific to the AdminIntersection feature. - Cleaned up unnecessary imports and code dependencies related to this functionality. * chore(webapp): update package-lock.json to adjust `peer` field entries - Removed redundant `peer` fields across various dependencies. - Added missing `peer` fields for consistency. * fix(api): add additional API response annotations in AdminIntersectionController - Added `@ApiResponse` annotations for `405 Method Not Allowed` and `404 Not Found` scenarios. - Improved API documentation for intersection endpoints with new response descriptions. * fix(api): remove redundant authorization check in AdminIntersectionController - Eliminated unnecessary organization membership validation block in GET `/admin-intersection`. - Simplified controller logic while retaining existing logging and response mechanisms. * chore(api): fix package structure for ManufacturerRepository and update tests * refactor(api): replace String with Integer for intersectionId in DTOs, services, and tests - Updated `intersectionId` type from `String` to `Integer` across `IntersectionDto` and related models. - Simplified method parameters and repository queries to use `Integer` instead of `String`. - Refactored `AdminIntersectionController` and `AdminIntersectionService` to remove redundant org filtering and authorization logic. - Adjusted test cases to align with updated method signatures and data type changes. * chore(api): remove unused `@Pattern` import from AdminIntersectionController * refactor(tests): update intersectionId assertions to use Integer instead of String - Adjusted test cases in `AdminIntersectionControllerTest` to align with the updated `intersectionId` data type. - Refined `@DisplayName` in non-superuser intersection access test for clarity. * chore(api): add `@Repository` annotation to ManufacturerRepository * refactor(tests): adjust admin intersection service tests to confirm security - Added `@AfterEach` to clear security context after tests. - Introduced `setUpSuperuserContext` and `setUpOperatorContext` for simulating authorization scenarios. - Improved test cases to verify behavior for superuser and operator roles. - Updated assertions to validate intersection organization data. * chore(api): simplify javadoc in IntersectionSingleResponse model * fix(tests): update admin intersection tests to assert 404 for not found scenarios - Adjusted test cases to assert 404 status for non-existent intersections or inaccessible data. - Updated `@DisplayName` annotations for clarity in test descriptions. - Replaced mock service responses with `EntityNotFoundException` for missing data scenarios. * chore(api): remove redundant driver-class-name and improve javadoc in AdminIntersectionService * refactor(api): simplify admin intersection update logic with partial mapping - Replaced manual field mapping in `AdminIntersectionService` with `IntersectionMapper.partialUpdate`. - Added `partialUpdate` method to `IntersectionMapper` using `@BeanMapping` with `NullValuePropertyMappingStrategy.IGNORE`. * chore(tests): migrate test properties file from `.properties` to `.yaml` format * fix(api): enforce RSU permissions in `patchIntersection` endpoint - Added validation to ensure operators have access to RSUs being added or removed. - Introduced `rsuRepository` to support RSU permission checks. - Updated error handling to return 403 when RSU access is unauthorized. * fix(tests): update integration test config with MongoDB properties annotations * fix(tests): add RSU access validation tests for patchIntersection endpoint - Introduced tests to assert 403 responses when RSUs in `rsus_to_add` or `rsus_to_remove` are outside the user's permissions. - Added `RsuRepository` as a mock dependency in `AdminIntersectionControllerTest`. * refactor(api): enforce organization header in admin intersection API - Removed user-specific filtering logic in `AdminIntersectionService` for intersection retrieval. - Updated controller and service methods to require the Organization header for all requests. - Simplified logging and query logic for intersection retrieval. * feat(api): handle MissingRequestHeaderException with detailed error response - Added an exception handler for `MissingRequestHeaderException` in `GlobalExceptionHandler`. - Returns a structured `ProblemDetail` with a 400 status and header-specific details. - Introduced unit tests to validate error handling for missing request headers. * chore(tests): reformat mockMvc calls for readability * refactor(api): remove unnecessary query parameters from admin intersection API calls - Updated `_getDataWithCodes` and `_patchData` calls to exclude redundant `query_params`. - Appended `intersection_id` directly to the URL for consistency and simplicity. - Normalized `intersection_id` to string during state updates to ensure uniformity in data handling. * refactor(api): update admin intersection endpoint to remove "admin-" prefix - Standardized endpoint paths by removing "admin-" prefix in `AdminIntersectionController`. - Updated associated test cases, logging, and model annotations to reflect new endpoint paths. - Ensured backward compatibility by adjusting test assertions for all CRUD operations. * refactor(api): update intersection API endpoints to use pluralized paths - Changed `/intersection` to `/intersections` in `AdminIntersectionController` for consistency with RESTful conventions. - Updated all related test cases, logging statements, and model annotations to reflect the new endpoint paths. - Ensured proper alignment in test expectations and documentation comments for affected methods and classes. * refactor(api): add "admin/" prefix to intersection API endpoints - Updated endpoint paths in `AdminIntersectionController` to include "admin/" prefix for improved namespace clarity. - Adjusted related test cases, logging statements, and model annotations to reflect the new paths. - Ensured test expectations, comments, and API documentation were aligned with the updated endpoint structure. * refactor(webapp): update admin intersection endpoint to use pluralized path * refactor(api): update deleteIntersection endpoint to use PathVariable instead of RequestParam - Changed `intersection_id` in `deleteIntersection` endpoint from `RequestParam` to `PathVariable` for improved RESTful design. - Updated related test cases and logging to reflect the new path structure. * chore(api): remove unused imports from AdminIntersectionController * feat(api): prevent duplicate RSU-Intersection associations - Added unique constraint on `rsu_id` and `intersection_id` in `RsuIntersection` entity. - Introduced filter logic in `AdminIntersectionService` to skip creating duplicate RSU-Intersection associations. - Updated `RsuIntersectionRepository` with `existsByRsuAndIntersection` method. - Added a test case to verify that duplicate associations are not created. * feat(sql): add unique constraint for RSU and intersection IDs - Enforced a unique constraint on `(rsu_id, intersection_id)` in `RsuIntersection` table to prevent duplicate associations. * refactor(webapp): update delete intersection API to use path variables - Updated `adminIntersectionTabSlice` to pass `intersection_id` in the URL path instead of query parameters. - Adjusted test cases to align with the new path structure. * refactor(webapp): remove unused async thunks and streamline admin intersection logic - Removed `getIntersectionInfo`, `editIntersection`, and `submitForm` async thunks from `adminEditIntersectionSlice`. - Replaced action thunks with RTK Query APIs for admin intersection CRUD operations. - Refactored related components and tests to align with the new RTK Query integration. - Eliminated redundant `loading` state and selectors for cleaner state management. * chore(webapp): update sample env to use localhost over DOCKER_HOST_IP * feat(webapp): add RTK Query API slice for admin intersection management - Introduced `adminIntersectionApiSlice` for CRUD operations related to admin intersections. - Added queries and mutations: getIntersections, getIntersection, fetch allowed selections, create, patch, and delete intersections. - Implemented tag-based caching and invalidation for efficient state updates. * fix(webapp): ensure modal closure and navigation on successful intersection update * feat(webapp): add organization header to admin intersections API calls - Updated `prepareHeaders` to include the `Organization` header alongside the `Authorization` token. - Refactored `getIntersections` query to remove redundant organization header logic. * fix(webapp): ensure immutability when processing intersection form data * test(webapp): remove eslint-disable directive * test(webapp): remove unused eslint-disable directive in adminIntersectionTabSlice test * test(webapp): update adminEditIntersection tests for form validation and JSON mapping adjustments * refactor(webapp): simplify form validation in adminEditIntersectionSlice * test(webapp): add tests for adminIntersectionApiSlice, covering CRUD operations and cache behavior * docs(api): add specification for Admin New Intersection API - Documented GET and POST endpoints for the `/admin-new-intersection` API. - Included authentication, role-based access control, and validation details. - Described database schema and safe input constraints. - Highlighted test coverage and identified existing gaps. * docs(api): update Admin Intersection API migration spec - Revised documentation for migrating the Python `AdminNewIntersection` API to Java Spring Boot. - Updated specification for GET and POST endpoints, including database schema mappings, validation, and authorization logic. - Added details about Java architecture, DTOs, services, and test coverage requirements. * feat(api): add GET and POST endpoints for Admin Intersection API - Added `GET /admin/intersections/allowed-selections` endpoint to retrieve organizations and RSUs for intersection creation. - Added `POST /admin/intersections` endpoint to create new intersections with validation and role-based access control. - Implemented service and permission checks for organization and RSU authorization. - Added corresponding unit tests to validate scenarios, including edge cases. * feat(api): add IntersectionCreate DTO for POST /admin/intersections - Introduced `IntersectionCreate` DTO to represent the request body for creating new intersections. - Included validation annotations and OpenAPI schema definitions for fields like `intersectionId`, `refPt`, `organizations`, `rsus`, and others. * feat(api): add CreateIntersection and AllowedSelections tests - Added unit tests for `createIntersection`, covering all edge cases like optional fields, duplicate IDs, and non-existent entities. - Added unit tests for `getAllowedSelections`, verifying superuser and operator access control behavior. - Updated service logic to validate and associate RSUs and organizations during intersection creation. * refactor(webapp): use constants for adminIntersection API tag and list ID * refactor(webapp): improve toast handling for delete operations in AdminIntersectionTab * refactor(webapp): adjust adminIntersection API to remove organization header from token setup * test(webapp): remove assertion for organization header in adminIntersectionApiSlice test * feat(api): enforce unique constraint on intersection_number - Added `@Column(unique = true)` to `intersection_number` in the Intersection entity to align with production DB constraints. - Added test to validate behavior when creating intersections with duplicate `intersection_number`. * docs(api): update comments for AdminIntersection API logic * fix(api): return 200 OK response for empty results from getAllIntersections The 404 response was considered an error by the UI framework, so it prevented the UI from reloading after deleting the last intersection in the list * chore(api): remove unused `/admin-intersection` from middleware.py * chore(docs): remove admin new intersection migration spec - Deleted `admin_new_intersection_spec.md` as the migration has been fully implemented in the intersection API. - Cleaned up unused references and outdated documentation. * feat(api/webapp): enhance error handling for intersections updates - Updated `GlobalExceptionHandler` to categorize errors containing "intersection." - Improved toast notifications in the web app to display specific error details during intersection updates. * feat(api): add content-type and accept headers in adminIntersectionApiSlice - Ensured `Content-Type` and `Accept` headers are set to `application/json` in API requests to standardize data handling and improve compatibility. * fix(api): remove "A" from start of sentence and resource type capitalization in GlobalExceptionHandler - Updated resource type capitalization for consistency. - Removed prefixed "A" from start of sentence to maintain correct grammar in messages when resource starts with a vowel * fix(tests): remove 'A' from string assertion in testRsuDuplicateSerialNumber * fix(api): use UserRole enum for operator-qualified org list * fix(tests): use UserRole enum in AdminIntersectionControllerTest --------- Co-authored-by: jacob6838 <jfrye@neaeraconsulting.com> Co-authored-by: Jacob Frye <51723791+jacob6838@users.noreply.github.com>
#321) * Setting base path to absolute for refresh support * Combining GlobalExceptionHandlers * Fixing merge conflicts * Switching from Zonky to TestContainers * Update services/intersection-api/api/src/test/java/us/dot/its/jpo/ode/api/config/AbstractIntegrationTest.java Co-authored-by: Matt Cook <mattheworion.cook@gmail.com> * Update services/intersection-api/api/src/test/java/us/dot/its/jpo/ode/api/config/AbstractIntegrationTest.java Co-authored-by: Matt Cook <mattheworion.cook@gmail.com> * docs: initial requirements assessment * docs: refine requirements spec * chore: add .idea folder to gitignore * feat: introduce Admin Intersection API models, controller, and service stubs - Added admin intersection models (`AllowedSelections`, `Bbox`, `RefPt`, `IntersectionData`, `IntersectionListResponse`, `IntersectionSingleResponse`, `IntersectionPatch`) to support API functionality. - Created `AdminIntersectionController` with endpoints for managing intersections (`GET`, `PATCH`, `DELETE`). - Defined `AdminIntersectionService` stub with method placeholders for intersection data retrieval and updates. * fix: relocate auth to controller layer * feat(intersection): define missing repositories and methods Define missing relationships in Intersection.java and add new repository for IntersectionOrganizations. Add missing repository method to existing RsuIntersectionRepository * feat(intersection): add repository methods for intersection and RSU data retrieval - Added batch and single-fetch methods in `RsuIntersectionRepository` for RSU IPs by intersection numbers. - Introduced `IntersectionRepository` methods to load intersections with associated organizations for various use cases. - Added method in `OrganizationRepository` to find organizations by name for batch operations. * feat(intersection): replace IntersectionData with IntersectionDto and add IntersectionMapper - Replaced `IntersectionData` with `IntersectionDto` in `IntersectionSingleResponse` and `IntersectionListResponse` models. - Introduced `IntersectionMapper` using MapStruct for entity-to-DTO conversion, including custom mapping for organization names. * feat(intersection): add GeometryMapper for JTS geometry and admin DTO conversion - Introduced `GeometryMapper` using MapStruct for converting between JTS Point/Polygon and admin DTOs (`RefPt`, `Bbox`). - Included methods for transforming JTS geometry to DTO and vice versa with SRID 4326 geometry handling. * feat(intersection): implement service layer with unit tests for AdminIntersectionService methods - Implemented unit tests for methods in `AdminIntersectionService`, including `getAllIntersections`, `getIntersection`, `patchIntersection`, and `deleteIntersection`. - Verified context-based filtering of organizations and RSU IPs, as well as error handling for not found entities and transactional integrity during patch and delete operations. - Used Mockito for mocking dependencies and capturing interactions. * feat(intersection): enforce org restrictions in AdminIntersectionController - Updated `getIntersection` and `patchIntersection` methods to include organization restriction checks: - Enforced access control by validating organizations against the user's roles and permissions. - Added HTTP 403 response for unauthorized org modifications in `patchIntersection`. - Passed `isSuperUser`, `userOrgs`, and `operatorOrgs` to service methods for handling filtered data access. * refactor(intersection): use instance of GeometryMapper in AdminIntersectionService - Replaced static calls to `GeometryMapper` with the instance-based `geometryMapper` in `patchIntersection`. - Updated constructor to include `GeometryMapper` as a dependency. * refactor(tests): group related unit tests using @nested in AdminIntersectionServiceTest - Organized test methods into logical groups (`GetAllIntersections`, `GetIntersection`, `PatchIntersection`) using `@Nested` for improved readability and maintainability. - Removed outdated comments and unnecessary helper factories. * fix(tests): downgrade Testcontainers version and simplify AbstractIntegrationTest config - Downgraded `testcontainers` dependency in `pom.xml` from `2.0.3` to `1.21.4` for compatibility. - Corrected mismatched class and file name - Removed `@ServiceConnection` annotations and reorganized bean definitions. * feat(tests): add integration test profile for Intersection API - Introduced `application-integration-test.yaml` for service-layer integration tests. - Configured test containers for PostgreSQL using `tc:` JDBC URL and MongoDB via `@DynamicPropertySource`. - Defined Kafka topics and schemas, security, email, and other environment-specific settings for testing. * feat(tests): convert AdminIntersectionServiceTest to integration test with Testcontainers - Refactored `AdminIntersectionServiceTest` to use Spring Boot's integration testing annotations (`@SpringBootTest`, `@Testcontainers`, `@Transactional`). - Replaced Mockito mocks with actual repository beans for full stack validation. - Configured MongoDB `@Container` with `@DynamicPropertySource` for integration test environment. - Added helper methods and entity factories for persisted test data setup. - Simplified test cases by removing unnecessary mocking and stubbing. * chore: ignore IntelliJ IDEA `.iml` files in `.gitignore` * chore(tests): remove unused configs from integration test YAML - Deleted `hibernate.dialect` property as it is unused with Testcontainers' `tc:` JDBC driver. - Removed obsolete `haas` and `asn1-decoder` settings to streamline configuration. * fix(tests): downgrade Testcontainers version and simplify AbstractIntegrationTest config - Downgraded `testcontainers` dependency in `pom.xml` from `2.0.3` to `1.21.4` for compatibility. - Corrected mismatched class and file name - Removed `@ServiceConnection` annotations and reorganized bean definitions. * feat(tests): add integration test profile for Intersection API - Introduced `application-integration-test.yaml` for service-layer integration tests. - Configured test containers for PostgreSQL using `tc:` JDBC URL and MongoDB via `@DynamicPropertySource`. - Defined Kafka topics and schemas, security, email, and other environment-specific settings for testing. * refactor(tests): remove AbstractIntegrationTest and migrate tests to use Testcontainers MongoDB configuration - Deleted `AbstractIntegrationTest` class and associated PostgreSQL container setup. - Migrated all test classes to use standalone MongoDB containers with `@Testcontainers` and `@DynamicPropertySource`. - Updated application test profiles to reflect the removal of PostgreSQL references. - Simplified test setup and adjusted annotations for improved maintainability. * fix(tests): update Hibernate `ddl-auto` setting to `none` in integration test config - Changed `ddl-auto` property in `application-integration-test.yaml` to prevent schema recreation during tests. * feat: add conditional bean and property loading for ASN.1 decoders - Applied `@ConditionalOnBean` to all ASN.1 decoders and `DecoderManager` to ensure presence of `MessageFrameCodec` before initialization. - Added `@ConditionalOnProperty` in `CodecConfig` for enabling ASN.1 decoder configuration (`enable.asn1-decoder` flag). - Updated `Asn1Controller` with `@ConditionalOnBean(DecoderManager.class)` to ensure dependency injection. - Modified `application-integration-test.yaml` to include `asn1-decoder` configuration. * feat: centralize Testcontainers version with BOM and add Spring Boot Testcontainers dependency - Added `spring-boot-testcontainers` dependency for enhanced test configuration. - Introduced `dependencyManagement` with `testcontainers-bom` to manage Testcontainers version consistency. - Removed redundant explicit version declarations for Testcontainers dependencies. * fix(ci): still run tests if change limit exceeded This allows us to still see test results when we exceed file change limits in valid cases. * test(containers): use ServiceConnection to speed up integration tests * refactor(tests): migrate decoder tests to Mockito and simplify setup - Replaced Spring Boot test annotations with Mockito for improved test performance. - Adjusted test setup to use `@BeforeEach` for initializing decoders with mocked dependencies. - Updated package paths for decoder test classes. - Removed unused imports and redundant annotations. * Revert "refactor(tests): migrate decoder tests to Mockito and simplify setup" This reverts commit 17932ec. * refactor(tests): update package paths for decoder test classes - Moved decoder test classes from `decoderTests` to `asn1` package for correctness. - Removed unused imports across decoder test files. * refactor(tests): enable ASN.1 decoder property and simplify test setup - Added `@MockitoBean` for `MessageFrameCodec` in decoder tests to satisfy `@ConditionalOnBean` dependency. - Enabled `enable.asn1-decoder` property in `@SpringBootTest` annotations across test classes. - Removed unnecessary `@ActiveProfiles` and `@Import` annotations for cleaner configuration. * fix(tests): replaced constructor injection with `@Autowired` for `HaasController`. * refactor(tests): remove unused dependencies and redundant test setup in `AdminIntersectionServiceTest` - Eliminated `@Transactional`, `@DynamicPropertySource`, and `@Container` annotations. - Removed unused `EntityManager`, `MessageFrameCodec`, and related methods (`flushAndClear`, `uuid`, etc.). - Simplified test logic by eliminating redundant calls and improving clarity. * chore(config): remove zonky reference from comment * refactor(tests): migrate decoder tests to use Mockito and clean up setup - Replaced Spring Boot test annotations with Mockito for faster and lightweight tests. - Refactored test setup to initialize decoders with `@BeforeEach` and mocked dependencies. - Removed unused imports and redundant code in decoder test classes. - Simplified file reading logic within test setup blocks. * chore(tests): update `ddl-auto` property in test configuration - Changed Hibernate `ddl-auto` setting from `none` to `create-only` in `application-integration-test.yaml`. * chore(tests): reformat AdminIntersectionServiceTest * refactor(tests): introduce TestFixtures and replace redundant test setup in `AdminIntersectionServiceTest` - Added `TestFixtures` to centralize object creation for tests, using `javafaker` for randomized test data. - Replaced inline and duplicated setup logic with reusable methods from `TestFixtures`. - Updated `pom.xml` to add `javafaker` as a test dependency. * refactor(tests): add database cleanup before each test in `AdminIntersectionServiceTest` - Introduced `@BeforeEach` method to clear all relevant tables to ensure isolated test data. - Updated `@SpringBootTest` to use `webEnvironment = NONE`. - Removed unused imports and redundant `@Testcontainers` annotation. * feat(api): add ManufacturerRepository and integrate with AdminIntersectionServiceTest - Introduced `ManufacturerRepository` for managing `Manufacturer` entities. - Updated `AdminIntersectionServiceTest` to use `ManufacturerRepository` for saving manufacturers. - Changed `webEnvironment` in test configuration to `MOCK` for more realistic test scenarios. - Adjusted `application.yaml` to replace Hibernate dialect with `database: postgresql` for configuration consistency. * chore(tests): remove unused random object creation methods in TestFixtures - Deleted `randomPoint`, `createRandomIntersection`, `createRandomRsu`, and related unused methods in `TestFixtures`. - Removed redundant variable `orgAName` from `AdminIntersectionServiceTest`. * feat(models): add OpenAPI schema annotations to intersection-related classes - Added `@Schema` annotations with descriptions and examples for intersection domain models. - Introduced new `MessageResponse` class for simple write operation responses. - Improved model documentation for API clarity and OpenAPI spec generation. * feat(api): refactor AdminIntersectionController with optimized endpoints and MessageResponse integration - Replaced `@RequestMapping` with specific HTTP method annotations (`@GetMapping`, `@PatchMapping`, `@DeleteMapping`) for better clarity and alignment with REST conventions. - Added `@Parameter` annotations for improved OpenAPI documentation generation. - Updated method return types to use `MessageResponse` for consistency in write operation responses. - Simplified intersection query handling by separating `getAllIntersections` and `getIntersection` methods. - Refactored `patchIntersection` and `deleteIntersection` to eliminate redundant response entity wrapping. * test(api): add integration tests for AdminIntersectionController - Added comprehensive integration tests to cover `GET`, `PATCH`, and `DELETE` endpoints in `AdminIntersectionController`. - Included tests for authorization, validation, and service delegation. - Verified HTTP status codes and response shapes for various scenarios, including success and failure cases. * fix(api): remove unnecessary `matchIfMissing` in `@ConditionalOnProperty` * chore(docs): remove AdminIntersection migration spec - Deleted `admin_intersection_migration_spec.md` as it is no longer relevant. - The document was removed after the refactor was completed * chore(api): remove redundant endpoint comments in AdminIntersectionController * refactor(api): remove `MessageResponse` usage from AdminIntersectionController and associated services - Deleted `MessageResponse` class as it is no longer required. - Updated `patchIntersection` and `deleteIntersection` methods to return `void` instead of `MessageResponse`. - Adjusted related service methods to eliminate success message returns. - Updated tests to reflect the removal of `MessageResponse` and success message assertions. * test(api): add unit tests for GeometryMapper - Added `GeometryMapperTest` class to test mapping between JTS geometries (`Point`, `Polygon`) and DTOs (`RefPt`, `Bbox`). - Included tests for `toRefPt`, `toPoint`, `toBbox`, and `toPolygon` methods to verify mapping correctness. - Added round-trip tests to ensure data integrity across conversions. - Verified handling of edge cases, including null inputs and negative coordinates. * refactor(api): replace manual mappings in GeometryMapper with MapStruct annotations - Replaced custom `toRefPt` and `toBbox` methods with annotated MapStruct mappings. - Simplified mapping logic by delegating property mappings to `@Mapping` annotations. - Improved consistency and reduced manual code for geometry-to-DTO conversions. * test(api): add unit tests for IntersectionMapper and utility method in TestFixtures - Added `IntersectionMapperTest` to validate entity-to-DTO mappings, including edge cases like null inputs and missing fields. - Tested `toDto` and `mapOrgNames` methods for accuracy and coverage. - Added `createBBox` utility to `TestFixtures` for reusable mock bounding box creation. * chore(api): remove redundant comments from Intersection API models - Deleted outdated and unnecessary comments in `IntersectionListResponse`, `IntersectionPatch`, `IntersectionDto`, and `IntersectionSingleResponse`. - Retained meaningful annotations for clarity and accurate documentation generation. * refactor(api): improve exception handling in AdminIntersectionService - Replaced empty DTO responses with `EntityNotFoundException` for consistency. - Added `AccessDeniedException` for unauthorized access to intersections. - Updated service methods to throw descriptive exceptions instead of returning empty responses. - Adjusted test cases to validate the new exception behavior. * chore(api): remove unused imports and fields in IntersectionMapperTest and AdminIntersectionService * chore(api): remove redundant comments from repository and service classes - Deleted outdated and unnecessary comments across repository interfaces and `AdminIntersectionService`. - Retained meaningful descriptions to ensure clarity and maintain readability. * feat(api): add logging to AdminIntersectionController and AdminIntersectionService - Added `@Slf4j` to `AdminIntersectionService` for centralized logging. - Introduced info, debug, and error logs across controller and service methods to improve traceability. - Enhanced debugging output for intersection queries, patching, and deletions. * chore(api): update GeometryMapper javadoc to simplify description * feat(api): add validation annotations to intersection models and logging updates in AdminIntersectionController - Added `@NotNull`, `@DecimalMin`, and `@DecimalMax` annotations to `Bbox` to ensure latitude and longitude values are within valid ranges. - Applied `@Pattern` annotations for IPv4 validation in `IntersectionPatch`'s RSU IP address lists. - Enhanced AdminIntersectionController with validation for numeric `intersectionId` and organization membership checks. - Introduced logging and authorization validation for GET `/admin-intersection`. * refactor(api): remove AdminIntersection python feature and tests - Deleted `AdminIntersection` feature and its routes in `main.py`. - Removed `test_admin_intersection.py` and tests related to `AdminIntersection` functionalities. - Updated references across the codebase to reflect the removal. * refactor(api): remove AdminIntersection feature implementation and related logic - Deleted `admin_intersection.py` and all associated helper functions, schemas, and endpoint logic. - Removed database query methods and authorization-related utilities specific to the AdminIntersection feature. - Cleaned up unnecessary imports and code dependencies related to this functionality. * chore(webapp): update package-lock.json to adjust `peer` field entries - Removed redundant `peer` fields across various dependencies. - Added missing `peer` fields for consistency. * fix(api): add additional API response annotations in AdminIntersectionController - Added `@ApiResponse` annotations for `405 Method Not Allowed` and `404 Not Found` scenarios. - Improved API documentation for intersection endpoints with new response descriptions. * fix(api): remove redundant authorization check in AdminIntersectionController - Eliminated unnecessary organization membership validation block in GET `/admin-intersection`. - Simplified controller logic while retaining existing logging and response mechanisms. * chore(api): fix package structure for ManufacturerRepository and update tests * refactor(api): replace String with Integer for intersectionId in DTOs, services, and tests - Updated `intersectionId` type from `String` to `Integer` across `IntersectionDto` and related models. - Simplified method parameters and repository queries to use `Integer` instead of `String`. - Refactored `AdminIntersectionController` and `AdminIntersectionService` to remove redundant org filtering and authorization logic. - Adjusted test cases to align with updated method signatures and data type changes. * chore(api): remove unused `@Pattern` import from AdminIntersectionController * refactor(tests): update intersectionId assertions to use Integer instead of String - Adjusted test cases in `AdminIntersectionControllerTest` to align with the updated `intersectionId` data type. - Refined `@DisplayName` in non-superuser intersection access test for clarity. * chore(api): add `@Repository` annotation to ManufacturerRepository * refactor(tests): adjust admin intersection service tests to confirm security - Added `@AfterEach` to clear security context after tests. - Introduced `setUpSuperuserContext` and `setUpOperatorContext` for simulating authorization scenarios. - Improved test cases to verify behavior for superuser and operator roles. - Updated assertions to validate intersection organization data. * chore(api): simplify javadoc in IntersectionSingleResponse model * fix(tests): update admin intersection tests to assert 404 for not found scenarios - Adjusted test cases to assert 404 status for non-existent intersections or inaccessible data. - Updated `@DisplayName` annotations for clarity in test descriptions. - Replaced mock service responses with `EntityNotFoundException` for missing data scenarios. * chore(api): remove redundant driver-class-name and improve javadoc in AdminIntersectionService * refactor(api): simplify admin intersection update logic with partial mapping - Replaced manual field mapping in `AdminIntersectionService` with `IntersectionMapper.partialUpdate`. - Added `partialUpdate` method to `IntersectionMapper` using `@BeanMapping` with `NullValuePropertyMappingStrategy.IGNORE`. * chore(tests): migrate test properties file from `.properties` to `.yaml` format * fix(api): enforce RSU permissions in `patchIntersection` endpoint - Added validation to ensure operators have access to RSUs being added or removed. - Introduced `rsuRepository` to support RSU permission checks. - Updated error handling to return 403 when RSU access is unauthorized. * fix(tests): update integration test config with MongoDB properties annotations * fix(tests): add RSU access validation tests for patchIntersection endpoint - Introduced tests to assert 403 responses when RSUs in `rsus_to_add` or `rsus_to_remove` are outside the user's permissions. - Added `RsuRepository` as a mock dependency in `AdminIntersectionControllerTest`. * refactor(api): enforce organization header in admin intersection API - Removed user-specific filtering logic in `AdminIntersectionService` for intersection retrieval. - Updated controller and service methods to require the Organization header for all requests. - Simplified logging and query logic for intersection retrieval. * feat(api): handle MissingRequestHeaderException with detailed error response - Added an exception handler for `MissingRequestHeaderException` in `GlobalExceptionHandler`. - Returns a structured `ProblemDetail` with a 400 status and header-specific details. - Introduced unit tests to validate error handling for missing request headers. * chore(tests): reformat mockMvc calls for readability * refactor(api): remove unnecessary query parameters from admin intersection API calls - Updated `_getDataWithCodes` and `_patchData` calls to exclude redundant `query_params`. - Appended `intersection_id` directly to the URL for consistency and simplicity. - Normalized `intersection_id` to string during state updates to ensure uniformity in data handling. * refactor(api): update admin intersection endpoint to remove "admin-" prefix - Standardized endpoint paths by removing "admin-" prefix in `AdminIntersectionController`. - Updated associated test cases, logging, and model annotations to reflect new endpoint paths. - Ensured backward compatibility by adjusting test assertions for all CRUD operations. * refactor(api): update intersection API endpoints to use pluralized paths - Changed `/intersection` to `/intersections` in `AdminIntersectionController` for consistency with RESTful conventions. - Updated all related test cases, logging statements, and model annotations to reflect the new endpoint paths. - Ensured proper alignment in test expectations and documentation comments for affected methods and classes. * refactor(api): add "admin/" prefix to intersection API endpoints - Updated endpoint paths in `AdminIntersectionController` to include "admin/" prefix for improved namespace clarity. - Adjusted related test cases, logging statements, and model annotations to reflect the new paths. - Ensured test expectations, comments, and API documentation were aligned with the updated endpoint structure. * refactor(webapp): update admin intersection endpoint to use pluralized path * refactor(api): update deleteIntersection endpoint to use PathVariable instead of RequestParam - Changed `intersection_id` in `deleteIntersection` endpoint from `RequestParam` to `PathVariable` for improved RESTful design. - Updated related test cases and logging to reflect the new path structure. * chore(api): remove unused imports from AdminIntersectionController * feat(api): prevent duplicate RSU-Intersection associations - Added unique constraint on `rsu_id` and `intersection_id` in `RsuIntersection` entity. - Introduced filter logic in `AdminIntersectionService` to skip creating duplicate RSU-Intersection associations. - Updated `RsuIntersectionRepository` with `existsByRsuAndIntersection` method. - Added a test case to verify that duplicate associations are not created. * feat(sql): add unique constraint for RSU and intersection IDs - Enforced a unique constraint on `(rsu_id, intersection_id)` in `RsuIntersection` table to prevent duplicate associations. * refactor(webapp): update delete intersection API to use path variables - Updated `adminIntersectionTabSlice` to pass `intersection_id` in the URL path instead of query parameters. - Adjusted test cases to align with the new path structure. * refactor(webapp): remove unused async thunks and streamline admin intersection logic - Removed `getIntersectionInfo`, `editIntersection`, and `submitForm` async thunks from `adminEditIntersectionSlice`. - Replaced action thunks with RTK Query APIs for admin intersection CRUD operations. - Refactored related components and tests to align with the new RTK Query integration. - Eliminated redundant `loading` state and selectors for cleaner state management. * chore(webapp): update sample env to use localhost over DOCKER_HOST_IP * feat(webapp): add RTK Query API slice for admin intersection management - Introduced `adminIntersectionApiSlice` for CRUD operations related to admin intersections. - Added queries and mutations: getIntersections, getIntersection, fetch allowed selections, create, patch, and delete intersections. - Implemented tag-based caching and invalidation for efficient state updates. * fix(webapp): ensure modal closure and navigation on successful intersection update * feat(webapp): add organization header to admin intersections API calls - Updated `prepareHeaders` to include the `Organization` header alongside the `Authorization` token. - Refactored `getIntersections` query to remove redundant organization header logic. * fix(webapp): ensure immutability when processing intersection form data * test(webapp): remove eslint-disable directive * test(webapp): remove unused eslint-disable directive in adminIntersectionTabSlice test * test(webapp): update adminEditIntersection tests for form validation and JSON mapping adjustments * refactor(webapp): simplify form validation in adminEditIntersectionSlice * test(webapp): add tests for adminIntersectionApiSlice, covering CRUD operations and cache behavior * docs(api): add specification for Admin New Intersection API - Documented GET and POST endpoints for the `/admin-new-intersection` API. - Included authentication, role-based access control, and validation details. - Described database schema and safe input constraints. - Highlighted test coverage and identified existing gaps. * docs(api): update Admin Intersection API migration spec - Revised documentation for migrating the Python `AdminNewIntersection` API to Java Spring Boot. - Updated specification for GET and POST endpoints, including database schema mappings, validation, and authorization logic. - Added details about Java architecture, DTOs, services, and test coverage requirements. * feat(api): add GET and POST endpoints for Admin Intersection API - Added `GET /admin/intersections/allowed-selections` endpoint to retrieve organizations and RSUs for intersection creation. - Added `POST /admin/intersections` endpoint to create new intersections with validation and role-based access control. - Implemented service and permission checks for organization and RSU authorization. - Added corresponding unit tests to validate scenarios, including edge cases. * feat(api): add IntersectionCreate DTO for POST /admin/intersections - Introduced `IntersectionCreate` DTO to represent the request body for creating new intersections. - Included validation annotations and OpenAPI schema definitions for fields like `intersectionId`, `refPt`, `organizations`, `rsus`, and others. * feat(api): add CreateIntersection and AllowedSelections tests - Added unit tests for `createIntersection`, covering all edge cases like optional fields, duplicate IDs, and non-existent entities. - Added unit tests for `getAllowedSelections`, verifying superuser and operator access control behavior. - Updated service logic to validate and associate RSUs and organizations during intersection creation. * refactor(webapp): use constants for adminIntersection API tag and list ID * refactor(webapp): improve toast handling for delete operations in AdminIntersectionTab * refactor(webapp): adjust adminIntersection API to remove organization header from token setup * test(webapp): remove assertion for organization header in adminIntersectionApiSlice test * feat(api): enforce unique constraint on intersection_number - Added `@Column(unique = true)` to `intersection_number` in the Intersection entity to align with production DB constraints. - Added test to validate behavior when creating intersections with duplicate `intersection_number`. * docs(api): update comments for AdminIntersection API logic * fix(api): return 200 OK response for empty results from getAllIntersections The 404 response was considered an error by the UI framework, so it prevented the UI from reloading after deleting the last intersection in the list * chore(api): remove unused `/admin-intersection` from middleware.py * chore(docs): remove admin new intersection migration spec - Deleted `admin_new_intersection_spec.md` as the migration has been fully implemented in the intersection API. - Cleaned up unused references and outdated documentation. * feat(sql): enforce uniqueness for `rsu_id` and `intersection_id` associations - Added unique constraint on `(rsu_id, intersection_id)` in the `RsuIntersection` table to prevent duplicate entries. - Introduced `existsByRsuAndIntersection` method in `RsuIntersectionRepository` for validation. - Updated service logic to handle duplicates gracefully. - Added test cases to verify constraint enforcement. * feat(webapp): refactor admin intersection form and API logic - Replaced thunk-based `getIntersectionCreationData` and `submitForm` actions with RTK Query hooks for improved state management. - Simplified `AdminAddIntersection` component by integrating API query/mutation hooks. - Removed unused selectors, reducers, and thunks in `adminAddIntersectionSlice`. - Updated tests to align with the new RTK Query implementation. - Enhanced form validation and submission flow with client-side checks. * refactor(webapp): remove unused types and simplify tests in AdminOrgTabIntersection - Deleted unused types (`AdminOrgIntersectionWithId` and `AdminOrgTabIntersectionAddMultiple`) for clean-up. - Removed redundant selectors, functions, and reducer tests. - Refactored tests to align with updated RTK Query integration, including mock adjustments and streamlined assertions. * refactor(webapp): remove unused admin intersection endpoints from EnvironmentVars * refactor(webapp): remove unused admin intersection endpoints from test environment variables * refactor(webapp): simplify form validation in adminAddIntersectionSlice * refactor(webapp): remove unused adminOrganizationTabIntersectionSlice - Deleted `adminOrganizationTabIntersectionSlice` and its references from the store for clean-up. - Removed redundant reducer and associated logic to streamline the codebase. * refactor(webapp): streamline adminOrganizationTabIntersection tests - Removed redundant reducer tests and selectors. - Refactored async thunks to improve mocking and use `vi.spyOn` for better integration with RTK Query. - Simplified dispatch logic by replacing type-based checks with `__initiate` marker. - Updated test setup/teardown for cleaner mock management. * refactor(api): remove admin new intersection feature and related references * refactor(api): remove admin new intersection module and related tests * refactor(webapp): replace hardcoded admin intersection tags with constants - Updated `invalidateTags` logic to use `ADMIN_INTERSECTION_TAG` and `ADMIN_INTERSECTION_LIST_ID` constants for improved maintainability. * feat(api/webapp): enhance error handling for intersections updates - Updated `GlobalExceptionHandler` to categorize errors containing "intersection." - Improved toast notifications in the web app to display specific error details during intersection updates. * feat(webapp): improve error handling for admin intersection actions - Enhanced error notifications to display detailed messages for add, delete, and bulk delete operations. - Introduced `extractErrorDetail` utility function for consistent error extraction logic. * feat(api): add content-type and accept headers in adminIntersectionApiSlice - Ensured `Content-Type` and `Accept` headers are set to `application/json` in API requests to standardize data handling and improve compatibility. * fix(api): remove "A" from start of sentence and resource type capitalization in GlobalExceptionHandler - Updated resource type capitalization for consistency. - Removed prefixed "A" from start of sentence to maintain correct grammar in messages when resource starts with a vowel * fix(tests): remove 'A' from string assertion in testRsuDuplicateSerialNumber * refactor(webapp): remove redundant admin intersection slice and related tests - Deleted `adminOrganizationTabIntersectionSlice` and corresponding test file. - Replaced async thunk logic with direct API usage in `AdminOrganizationTabIntersection` component. - Improved code clarity and reduced dependency complexity with consolidated logic. * test(webapp): add unit tests for AdminOrganizationTabIntersection actions - Added comprehensive test coverage for delete and bulk delete actions. - Validates functionality for scenarios with multiple and single organization intersections. - Improved mock setups for API calls and UI actions. * feat(api): add endpoint to fetch intersections not in organization - Introduced `/admin/intersections/available` API endpoint to retrieve intersections not associated with a specified organization. - Added service logic and repository query for finding non-associated intersections. - Included authorization check for ADMIN role or superuser. * fix(api): use enum for user role in getQualifiedOrgList call * test(api): add unit tests for intersections not in organization endpoint - Added tests for `/admin/intersections/available` endpoint to validate permissions, headers, and responses. - Covered scenarios for superuser, admin role, and unauthorized access. - Verified service interaction and handling of edge cases. * fix(tests): replace string role with enum in getQualifiedOrgList calls * fix(api): update query for intersections not in organization - Modified query to include `LEFT JOIN FETCH` for fetching related entities. - Improved query performance and ensured proper handling of related collections. * feat(webapp): add multi-add functionality for intersections in AdminOrganizationTabIntersection - Added multiselect dropdown to select intersections not associated with the current organization. - Implemented bulk add action with success/error toast messages and table refresh. - Extended `adminIntersectionApiSlice` to support fetching available intersections. - Updated related tests to cover new actions. * refactor(api): extract method for building IntersectionListResponse - Moved repetitive logic into a dedicated `getIntersectionListResponse` method for improved readability and reusability. - Simplified the response generation in related service methods. --------- Co-authored-by: jacob6838 <jfrye@neaeraconsulting.com> Co-authored-by: Jacob Frye <51723791+jacob6838@users.noreply.github.com>
* Make Docker changes * Intersection API changes * CV Manager web application changes * Add Java tests * Remove unusued value artifacts from porting Python implementation * Remove redundant RSU null and empty list check * Add dedicated test for when hasCompleteRsuData is false. * Remove helper method intended for simplifying the warning suppression * Address pull request comments from mcook42 * Remove the zonky usage from the original UpgradeControllerTest implementation * Have frontend test use new dto response structure * Address upgrade mapper null field behavior * Change upgrade endpoint to properly support multiple RSUs once more * Make rsu_ip plural along with affected classes and scripts * Reduce max pool size for the hikari tests to 2 to increase tests that can be run in parallel * Organization header is no longer used * Converted returned map to the FirmwareUpgradeCheckResponseDto * Refactor the map to be typed instead of just defining it as Object * Remove !==
* feat: add ScmsHealthDto, ScmsHealthMapper, and unit tests for SCMS health data mapping
* feat: add ScmsHealthService with method skeleton for SCMS health status retrieval
* feat: add ScmsHealthController and integration tests for SCMS health status retrieval
* feat: update ScmsHealthDto, mapper, and controller to return SCMS health status as a map keyed by RSU IP addresses
* feat: implement SCMS health status retrieval with repository, service, and integration tests
* test: add test for empty SCMS status response handling
* test: add tests for null and empty inputs in ScmsHealthMapper
* test: ensure stable result when SCMS health records share identical timestamps
* refactor: extract reusable helper methods in ScmsHealthServiceTest to streamline test setup
* feat: update webapp to use new ISS SCMS status endpoint and update related tests
* chore: remove legacy ISS SCMS status endpoint and associated tests, data, and middleware references
* feat: update IAPI SCMS health status handling to use string values ("1"/"0") instead of boolean, adjust models, mappers, and tests accordingly
* feat: revert webapp SCMS status handling to use string values ("1"/"0"), update related components, models, and tests
* test: update RsuMarker tests to use "scms" as displayType instead of "online" for SCMS status
* feat: add ColumnTransformer for SCMS health field to handle type conversion
* feat: add role-based authorization to ScmsHealthController, update tests accordingly
* feat: update ScmsHealth query to ensure deterministic tie-breaking by ID, adjust test to validate exact one-row-per-RSU behavior
* test: expand unit tests for ScmsHealthService to validate edge cases and deterministic query behavior
* test: add `@DisplayName` annotations to ScmsHealthControllerTest for improved test readability and clarity
* test: add `@DisplayName` annotations to ScmsHealthServiceTest for improved test readability and consistency
* feat: ensure deterministic tie-breaking for ScmsHealth query by refining subquery logic
* test: add `@DisplayName` annotations to ScmsHealthMapperTest for improved test readability and refactor variable names for clarity
* feat: simplify ScmsHealth query by using MAX(id) for latest record selection, remove timestamp precedence test
* docs: remove reference to deleted file
* docs: update schema descriptions in ScmsHealthDto class
* docs: update SCMS health record descriptions in ScmsHealthDto class
* feat: update ScmsHealth query to select the latest record by timestamp instead of ID, adjust related tests to reflect new logic
* docs: clarify SCMS health status schema description in ScmsHealthDto
* docs: add `@Operation` annotation to ScmsHealthController.getAllStatuses method and update description of class-level `@Tag` annotation
* refactor: replace `Rsu` entity with direct `InetAddress` reference in ScmsHealthRsuProjection and adapt related queries, mappers, and tests
* Revert "feat: update ScmsHealth query to select the latest record by timestamp instead of ID, adjust related tests to reflect new logic"
This reverts commit 195df88.
* feat(api): handle MissingRequestHeaderException with detailed error response
- Added an exception handler for `MissingRequestHeaderException` in `GlobalExceptionHandler`.
- Returns a structured `ProblemDetail` with a 400 status and header-specific details.
- Introduced unit tests to validate error handling for missing request headers.
* test: update expected error response for missing organization header in ScmsHealthController
* chore: restore dispatch of getIssScmsStatus action on WZDx marker selection
* refactor: extract zone ID as constant, update date time formatter in ScmsHealthMapper and add TODO to make configurable
* chore: update `findLatestScmsHealthByOrganization` to use Spring's `@Transactional(readOnly = true)` annotation to improve performance
* Revert "test: update RsuMarker tests to use "scms" as displayType instead of "online" for SCMS status"
This reverts commit 032573e.
* chore: remove unused Lombok @slf4j annotation from ScmsHealthService
* chore: remove unnecessary newline in ScmsHealthRepository
* refactor: simplify saveRsuWithoutOrg method by reusing saveRsu and adding null check for organization
* chore: update SCMS status endpoint paths to `/scms/status` across API, tests, and webapp
* refactor: replace getIssScmsStatus thunk with RTK Query-based scmsApiSlice, remove legacy state and selectors
* feat: add configurable application timezone support with DateTimeConfig class and related tests
* feat: add SCMS health sample data for RSUs in CVManager_SampleData.sql
* refactor: remove unused Lombok annotations and update timezone exception handling in DateTimeConfig
* refactor: replace ScmsHealth subquery logic with ROW_NUMBER window function and update related tests and projection
* refactor: use ScmsHealthResponse wrapper for SCMS status API, update mappers, controller, tests, and webapp to reflect changes
* chore: enforce compile error for unmapped fields in ScmsHealthMapper
* chore: remove unused CVIZ_API_SERVER_URL environment variable from tests
* docs: update response code order in API response documentation for SCMS status endpoint
* chore: remove SCMS health data insertion from sample data SQL
* refactor: update SCMS API base path to include `/devices` in controllers, tests, and webapp
* refactor: simplify SCMS health service tests by using TestFixtures for data creation
* chore: adjust HikariCP settings in integration tests to manage PostgreSQL connection limits
* feat(tests): update ScmsHealthDtos to use Boolean for health values
- Updated test cases in `ScmsHealthControllerTest` and `ScmsHealthMapperTest` to validate health values as Booleans instead of Strings.
- Modified `ScmsHealthDto` to reflect Boolean type for health field.
- Removed `healthToString` mapper method and updated mappings in `ScmsHealthMapper`.
- Adjusted tests for inactive and active health scenarios to align with Boolean logic.
* chore(tests): remove unused test ManufacturerRepository
* feat(api): remove DateTimeConfig and update SCMS expiration to Instant
- Deleted `DateTimeConfig` and related timezone configuration properties from the application.
- Updated `ScmsHealthDto` to replace `String expiration` with `Instant expiration` for ISO-8601 UTC compatibility.
- Refactored `ScmsHealthMapper` to simplify mapping logic by removing timezone-based date formatting.
- Adjusted tests for SCMS health mappings and projections to accommodate changes in `Instant` expiration type.
* chore(env): remove APP_TIMEZONE configuration from env and compose files
- Eliminated `APP_TIMEZONE` variable from `sample-full.env` and `docker-compose-intersection.yml` as it is no longer required.
* feat(webapp): improve SCMS expiration formatting for RSUs
- Introduced `formatScmsExpiration` utility for human-readable SCMS expiration dates.
- Updated components and selectors to use standardized expiration display logic.
- Replaced raw ISO-8601 strings with localized date formats in SCMS status and expiration views.
- Enhanced maintainability by centralizing date formatting logic in `scmsApiSlice`.
* refactor(api): simplify `prepareHeaders` logic by removing unused endpoint parameter in `scmsApiSlice`
* refactor(webapp): replace `useLazyGetScmsStatusQuery` with `useGetScmsStatusQuery`
- Removed lazy querying logic and updated SCMS status retrieval to auto-fetch based on `organization`.
- Simplified components by eliminating explicit trigger calls for SCMS status data.
- Updated `ScmsHealthDto` to use `boolean` for `health` field instead of `'0' | '1' | null`.
* fix(webapp): update SCMS status to use boolean type
- Replaced `'0' | '1' | null` SCMS status with `boolean | null` for consistency and clarity.
- Updated components, tests, and filters to handle the boolean SCMS status.
- Simplified related logic in `DisplayRsuErrors` and `RsuMarker`.
* test(tests): correct order of database cleanup and security context management
- Added `@BeforeEach` and `@AfterEach` methods for test database cleanup and security context management to prevent test data bleeding from one suite to another.
* chore(api): remove outdated comment from `ScmsHealthDto` definition
* test(webapp): enhance `RsuErrors` and `RsuFirmwareMenu` test coverage
- Added detailed test cases for `DisplayRsuErrors` covering SCMS statuses, snapshots, and edge cases.
- Improved `RsuFirmwareMenu` tests to validate success/error message rendering for single and batch RSU upgrades.
- Introduced mock configurations for realistic test scenarios.
* test(webapp): update `DisplayRsuErrors` snapshot tests
- Regenerated snapshots for `DisplayRsuErrors` to reflect updated SCMS status, UI structure, and table contents.
* chore: move ScmsHealthRsuProjectionImpl to test package
* feat(sql): add index for `timestamp` column in `scms_health` table
- Improves query performance by indexing the `timestamp` field.
* refactor(sql): move `scms_health` index creation to dedicated update script
---------
Co-authored-by: Matt Cook <matt.cook@trihydro.com>
Co-authored-by: Drew Johnston <drewjj901@gmail.com>
* Adding notif email generation and sending to the iapi * Adding notification email examples * Adding iapi email controller * Fixing EmailServiceTest * Update EmailController.java * Update EmailServiceTest.java * Fixing broken config property refs * Updating email generator comments, colors, and content * Updating default iapi env vars to avoid localhost * Update EmailRecipient.java * Adding email frequency to tables and queries * Update launch.json * Improving getCombinedResponseEntity to handle partial failures * Removing secure env vars from application.yaml * Renaming email template to email_template.html * Removing duplicate dependency * Update services/intersection-api/api/src/main/java/us/dot/its/jpo/ode/api/emails/generators/IntersectionNotificationSummaryEmailGenerator.java Co-authored-by: Matt Cook <mattheworion.cook@gmail.com> * Handling unsubscribe URL generation errors * Removing unused methods * Cleaning up email generation code * Writing email provider tests * Adding sql update script * Update email_subscription_type_update.sql * SUpporting authenticated SMTP server * Regenerating email postgres JPA objects * Working email sending with JPA repositories * Temporarily force enabling emailer * Renaming INTERSECTION_SMTP_SERVER_HOST * Adding spring mail ssl trust policy * Adding python EmailApi * Adding IAPI endpoints for pyapi emails * Removing unused environment variables * Reverting pyapi iapi email gen * Revert "Reverting pyapi iapi email gen" This reverts commit c81a1e7. * Removing unnecessary email endpoints * Writing tests for email generators * Adding snapshot tests for email generators * Updating intersectionNotificationSummary snapshot * Improving emailApi non-success handling * Hardcoding gmail for testing * Adding logging for email recipients * Adding additional iapi email logging * Adding transport listener for email send debugging * Adding sentDate header * Reverting add transport listener * Reverting email logging changes * Combining GlobalExceptionHandlers * Update EmailService.java * Update launch.json * Revert "Merge branch 'iapi-email-generation-pyapi' into iapi-email-generation-extra-types" This reverts commit 0c148f8, reversing changes made to a8930e8. * Update services/intersection-api/api/src/main/java/us/dot/its/jpo/ode/api/models/keycloak/TokenPostRequestKeycloak.java Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update services/intersection-api/api/src/test/java/us/dot/its/jpo/ode/api/SnapshotTestUtils.java Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update services/intersection-api/api/src/main/java/us/dot/its/jpo/ode/api/models/emails/contents/serialization/InstantToMillisSerializer.java Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update EmailServiceTest.java * Update SnapshotTestUtils.java * Condensing shared keycloak api code * Using EmailResponseException with email send APIs * Removing Instant to/from Millis de/serializers * Escaping HTML in email templating * Tweaking unit tests * Updating email test snapshot * Reverting unnecessary changes * Adding Schema annotations and removing unused code * Improving mocking in email generator tests * Updating iapi email generator test url * Updating email snapshot formatting * Update intersection_notification_summary_email_snapshot.html * Revert update settings.json * Updating email templates with no max-width * Removing KeycloakApi * Update rsu_error_summary_email_snapshot.html * Updating RsuErrorSummary email to only use requestors email * Updating javadoc comments and handling null emails from token Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update EmailSendResponse.java * Updatin rsu error summary email text * Switching auth token getters to lombok * Updating EmailGeneratorTests with MockitoExtension * Adding conditional unsubscribe url rendering * Update SupportRequestEmailGeneratorTest.java * Adding email generator mocked tests * Tweaking logging in snapshot utils * Parameterize shouldHandleEmailInVariousFormats test Co-authored-by: Brandon Payne <bpayne@trihydro.com> * Update CvManagerAuthTokenTest.java --------- Co-authored-by: Matt Cook <mattheworion.cook@gmail.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Co-authored-by: Brandon Payne <bpayne@trihydro.com>
* Adding notif email generation and sending to the iapi * Adding notification email examples * Adding iapi email controller * Fixing EmailServiceTest * Update EmailController.java * Update EmailServiceTest.java * Fixing broken config property refs * Updating email generator comments, colors, and content * Updating default iapi env vars to avoid localhost * Update EmailRecipient.java * Adding email frequency to tables and queries * Update launch.json * Improving getCombinedResponseEntity to handle partial failures * Removing secure env vars from application.yaml * Renaming email template to email_template.html * Removing duplicate dependency * Update services/intersection-api/api/src/main/java/us/dot/its/jpo/ode/api/emails/generators/IntersectionNotificationSummaryEmailGenerator.java Co-authored-by: Matt Cook <mattheworion.cook@gmail.com> * Handling unsubscribe URL generation errors * Removing unused methods * Cleaning up email generation code * Writing email provider tests * Adding sql update script * Update email_subscription_type_update.sql * SUpporting authenticated SMTP server * Regenerating email postgres JPA objects * Working email sending with JPA repositories * Temporarily force enabling emailer * Renaming INTERSECTION_SMTP_SERVER_HOST * Adding spring mail ssl trust policy * Adding python EmailApi * Adding IAPI endpoints for pyapi emails * Removing unused environment variables * Reverting pyapi iapi email gen * Revert "Reverting pyapi iapi email gen" This reverts commit c81a1e7. * Removing unnecessary email endpoints * Writing tests for email generators * Adding snapshot tests for email generators * Updating intersectionNotificationSummary snapshot * Improving emailApi non-success handling * Hardcoding gmail for testing * Adding logging for email recipients * Adding additional iapi email logging * Adding transport listener for email send debugging * Adding sentDate header * Reverting add transport listener * Reverting email logging changes * Combining GlobalExceptionHandlers * Update EmailService.java * Update launch.json * Revert "Merge branch 'iapi-email-generation-pyapi' into iapi-email-generation-extra-types" This reverts commit 0c148f8, reversing changes made to a8930e8. * Update services/intersection-api/api/src/main/java/us/dot/its/jpo/ode/api/models/keycloak/TokenPostRequestKeycloak.java Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update services/intersection-api/api/src/test/java/us/dot/its/jpo/ode/api/SnapshotTestUtils.java Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update services/intersection-api/api/src/main/java/us/dot/its/jpo/ode/api/models/emails/contents/serialization/InstantToMillisSerializer.java Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update EmailServiceTest.java * Update SnapshotTestUtils.java * Condensing shared keycloak api code * Using EmailResponseException with email send APIs * Removing Instant to/from Millis de/serializers * Escaping HTML in email templating * Tweaking unit tests * Updating email test snapshot * Creating python EmailApi * Removing contact-support and rsu-error-summary python code * Adding email sending env vars to pyapi * Integrating support and rsu error iapi emails into webapp * Reverting unnecessary changes * Adding Schema annotations and removing unused code * Improving mocking in email generator tests * Updating iapi email generator test url * Updating email snapshot formatting * Update intersection_notification_summary_email_snapshot.html * Revert update settings.json * Reverting removal of emailSender * Updating unit tests * Update AdminIntersectionTab.test.tsx.snap * Converting smtp_error_handler to EmailApi * Update test_smtp_error_handler.py * Updating email templates with no max-width * Adding ENABLE_ERROR_EMAILS to python api * Removing KeycloakApi * Adding keycloak_api direct to keycloak * Update rsu_error_summary_email_snapshot.html * Updating unit tests * Potential fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * Cleaning up email_api and keycloak_api * Updating RsuErrorSummary email to only use requestors email * Removing RsuErrorSummary recipients * Fixing python api KC_SA_CLIENT_ID Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * Removing unused env vars from README Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * Updating smtp error handler stack trace processing Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * Improving error handling in email_api Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * Cleaning up email_api documentation Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * Cleaning up email_api env vars and imports * Update test_smtp_error_handler.py * Updating javadoc comments and handling null emails from token Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update EmailSendResponse.java * Removing RsuErrorSummary dialog * Updatin rsu error summary email text * Updating test snapshots * Updating docstring comments throughout emailing code Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * Update DisplayRsuErrors.tsx * Update test_smtp_error_handler.py * Switching auth token getters to lombok * Updating EmailGeneratorTests with MockitoExtension * Adding conditional unsubscribe url rendering * Update SupportRequestEmailGeneratorTest.java * Adding email generator mocked tests * Tweaking logging in snapshot utils * Tweaking logging in snapshot utils * Parameterize shouldHandleEmailInVariousFormats test Co-authored-by: Brandon Payne <bpayne@trihydro.com> * Update CvManagerAuthTokenTest.java * Adding private methods to keycloak_api * Cleaning up unused code * Documenting keycloak_api test fixtures * Reverting AdminIntersectionTab test changes --------- Co-authored-by: Matt Cook <mattheworion.cook@gmail.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Co-authored-by: Brandon Payne <bpayne@trihydro.com>
* Resolving keycloak user deletion error * Adding user creation endpoint and service methods * Adding hasUser and hasUsers checks to PermissionService * Creating create user RTK query endpoint and integrating into webapp * Removing adminAddUserSlice * Adding java unit tests * Removing admin-new-user python code * Updating UserManagementService tests * Removing admin-new-user references * Cleaning up authentication and closing edge cases * Re-adding check_email to admin_new_org * Update UserControllerTest.java * Update services/intersection-api/api/src/main/java/us/dot/its/jpo/ode/api/services/PermissionService.java Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update webapp/src/features/adminAddUser/AdminAddUser.tsx Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update services/intersection-api/api/src/main/java/us/dot/its/jpo/ode/api/services/PermissionService.java Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * removing org findByNameIgnoreCase * Update UserMapper.java * Clening up AdminOrganizationTabUser slice tests * Update UserManagementServiceTest.java * Fixing UserRole enum refs * Removing RsuUpgradeService import from GlobalExceptionHandler * Revert "Removing RsuUpgradeService import from GlobalExceptionHandler" This reverts commit 0d25b9b. * Adding size constraint to UserDto organizations * Removing unnecessary user patch email duplication check * Update services/intersection-api/api/src/main/java/us/dot/its/jpo/ode/api/controllers/users/UserController.java Co-authored-by: Matt Cook <matt.cook@trihydro.com> * Using ResponseStatus Created in createUser controller * Adding UserDto Email validation * Improving error message handling in edit User/RSU * Update UserControllerTest.java * Adding keycloak admin service account user creation * Updating UserManagementService tests with kc admin * Adding userApiSlice tests * Update setupTests.ts * Update setupTests.ts * Removing redundant isAUuthValid checks from PermissionService * Adding and implementing shared checkQualifiedOrgs method * Adding AccessDeniedExceptions to getCvManagerAuthToken * Safely try wrapping keycloak user creation * Update PermissionServiceTest.java --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Co-authored-by: Matt Cook <matt.cook@trihydro.com>
* feat: add Flyway V1 baseline migration and dev seed data Merges CVManager_CreateTables.sql and all 19 update_scripts into a single V1__baseline.sql representing the fully-migrated schema state. Adds rsu_health table and idx_scms_health_timestamp that were present in update scripts but absent from CreateTables.sql. Moves CVManager_SampleData.sql into R__sample_data.sql as a Flyway repeatable migration with ON CONFLICT guards so it is idempotent on re-run. * chore: add Flyway configuration Adds flyway.toml with baselineOnMigrate=true so existing live databases are stamped at V1 without re-execution, outOfOrder=true to handle timestamp-versioned migrations from fork branches, and mixed=true for compatibility with the DDL-only baseline script. * docs: add Flyway migration developer guide Documents the timestamp-based naming convention, how to create and run migrations locally, how baselineOnMigrate works for existing environments, and why outOfOrder is enabled for the CDOT fork workflow. * docs: deprecate manual sql_scripts update_scripts Manual migration scripts in resources/sql_scripts/ are superseded by Flyway-managed migrations in resources/db/migration/. Directory is kept as historical reference only. * feat: integrate Flyway into Docker Compose startup - Add pg_isready healthcheck to cvmanager_postgres so dependent services wait for the database to accept connections - Remove docker-entrypoint-initdb.d volume mount (replaced by Flyway) - Add flyway service that runs migrate on startup and exits cleanly - Wire cvmanager_api to depend on flyway: service_completed_successfully so the API never starts against an un-migrated schema * ci: add Flyway migration validation job Adds validate_migrations job that spins up a PostGIS 15 container, applies all migrations with flyway migrate, and verifies state with flyway info on every PR and push to develop/cdot-release branches. Catches broken migration SQL before it reaches any environment. * fix: remove schemas param incompatible with Flyway 10 Community flyway.schemas is a Teams/Enterprise-only parameter in Flyway 10. Removing it restores compatibility; public is the PostgreSQL default. * docs: update Flyway developer guide with migration guidelines and formatting adjustments * docs(db): improve Flyway migration guide formatting and clarity * refactor(sql): move legacy update scripts to deprecated directory for reference * docs: preserve schema documentation as SQL comments and README reference Adds COMMENT ON TABLE/COLUMN for all 30 tables via a new Flyway migration (V202605211729__add_table_comments.sql), making descriptions queryable in psql/pgAdmin and portable via pg_dump. Also adds a Schema Reference section to resources/db/README.md covering all tables and critical data requirements. Source material: resources/deprecated/sql_scripts/README.md, which documented the original 14 tables but left the 16 tables added by update scripts undocumented. Both sources are now superseded. Each COMMENT statement is guarded with a DO/EXCEPTION block so the migration tolerates databases that were baselined at V1 before all update scripts were manually applied. * docs: improve formatting and table organization in schema reference * fix(docker-compose): disable restart policy for service This prevents infinite restarts of the flyway service during a migration failure * feat(db): add Dockerfile for cvmanager-flyway migration image Builds a self-contained Flyway image from flyway/flyway:10-alpine with versioned migration SQL files baked in. The V*.sql glob excludes R__sample_data.sql (dev-only seed data) without any explicit list to maintain. Adding a new migration only requires adding a file to resources/db/migration/ and rebuilding the image. * feat(k8s): replace pg-init-tables ConfigMap with Flyway migration Job The embedded pg-init-tables ConfigMap was stale (16 tables vs 25+ in the current schema) and used docker-entrypoint-initdb.d, which only runs on an empty data directory and cannot apply incremental schema updates. Remove the ConfigMap and its volume mount from the Postgres Deployment. Add a wait-for-migrations init container to the API Deployment that polls flyway_schema_history before allowing the API to start, ensuring the Job completes before any application traffic reaches the database. * ci: validate Flyway image build on each PR Adds a build_flyway_image job that builds the cvmanager-flyway image from resources/db/Dockerfile on every PR. Catches Dockerfile or SQL syntax errors before they reach a deployment. Extend with push steps and registry credentials to publish the image as part of the release pipeline. * docs(k8s): document Flyway image build and deployment workflow Replaces the old migration section (which referenced the removed pg-init-tables ConfigMap) with instructions for building and pushing the cvmanager-flyway image, the kubectl deployment order, and the steps for adding new migrations going forward. * refactor(k8s): simplify flyway Job to use custom image instead of ConfigMaps Remove the flyway-sql and flyway-conf ConfigMaps that embedded migration SQL inline. The cvmanager-flyway Docker image (resources/db/Dockerfile) now carries both the SQL files and flyway.toml, making the ConfigMaps redundant. The Job shrinks from ~860 lines to 42 with no embedded content to keep in sync. * test(rsu-info-bridge): replace manual SQL init with Flyway migrations Switch Testcontainers setup to use Spring Boot's Flyway autoconfiguration instead of copying CVManager_CreateTables.sql and CVManager_SampleData.sql (which no longer exist after the migration to resources/db/migration/). Flyway runs all V* and R* migrations from resources/db/migration/ against the PostGIS container before the JPA context initializes. No schema logic lives in TestcontainersConfiguration anymore. * fix(rsu-info-bridge): resolve Flyway migration path via Maven testResources Flyway's filesystem: location is silently skipped when the working directory at test time doesn't match expectations (failOnMissingLocations defaults to false). Switch to classpath:db/migration and use Maven testResources to copy resources/db/migration/ onto the test classpath, which is always reliable regardless of JVM working directory. * fix(rsu-info-bridge): wire Flyway explicitly for Spring Boot 4 test context Spring Boot 4.0.3 removed FlywayAutoConfiguration from spring-boot-autoconfigure, so spring.flyway.* properties in application-test.yaml were silently ignored and migrations never ran against the Testcontainer database. Add an explicit @bean Flyway to TestcontainersConfiguration that calls migrate() during context startup. Remove the now-dead spring.flyway.* block from application-test.yaml and the unused @testcontainers annotation from RsuServiceTest (container lifecycle is managed by @Serviceconnection, not @testcontainers). * ci(k8s): publish cvmanager-flyway to GHCR and gate build on passing migrations - build_flyway_image now requires validate_migrations to succeed before running - On push to develop/cdot-release*, push to ghcr.io/<org>/cvmanager-flyway with sha-<short-sha> and branch-name tags via docker/build-push-action - PR builds remain smoke-test-only (no push, no registry credentials needed) - Update K8s YAML image placeholder to GHCR path and annotate PG_DB_NAME - Update K8s README to document CI-driven publish and manual fallback - Add Kubernetes deployment pointer section to resources/db/README.md * Delete services/rsu-info-bridge/CLAUDE.md * Delete CLAUDE.md * docs: update future migration file naming convention * docs(adr): add ADR-0001 for Flyway database migrations Records the decision to adopt Flyway 10 Community Edition, covering rationale, key configuration choices (timestamp versioning, baselineOnMigrate, outOfOrder, repeatable migrations), deployment integration, alternatives considered, and consequences. * docs(readme): add Architecture Decision Records section with index table * style: remove em dashes from ADR and README per style guide * style: update comment formatting in TestcontainersConfiguration Clarify comment about FlywayAutoConfiguration removal in Spring Boot 4 by adjusting punctuation for consistency. * docs(db): update migration naming convention and deprecated scripts reference - Adjust migration file naming convention to use `HHmm` for timestamps. - Update deprecated scripts reference path in the README for clarity. - Revise Kubernetes Flyway YAML comments to reference the README for migration standards. * fix: address Copilot review feedback on migrations PR - Use PGPASSWORD env var in API initContainer psql command so credentials are not exposed in process args - Add PG_DB_NAME fail-fast check to initContainer to surface misconfiguration immediately rather than looping indefinitely - Mount resources/db/flyway.toml in both CI migration steps so validateMigrationNaming, mixed, and other shared settings are enforced consistently with Docker Compose and Kubernetes - Pin CI PostGIS service image to postgis/postgis:15-3.4 (was floating 15-master tag) to match the Testcontainers tag used in rsu-info-bridge - Fix broken relative link in deprecated sql_scripts README * ci: add Flyway migration validation step to CI workflow * refactor(db): rename timestamp migration to sequential V2 The project is switching from timestamp-based to sequential integer versioning. V202605211729__add_table_comments.sql is renamed to V2__add_table_comments.sql while still on the unmerged feature branch, so no applied-migration checksums are at risk. * refactor(db): disable outOfOrder in Flyway configuration Switch to strict in-order migration application. Version conflicts now surface as merge conflicts that must be resolved explicitly, rather than being silently bypassed with outOfOrder execution. * docs(db): update versioning documentation for sequential integer scheme Replace timestamp-based versioning rationale with sequential integer rationale across the ADR, db README, and Kubernetes README. Remove the outOfOrder section from the developer guide and update all naming convention examples and the schema reference table to reflect V{N}. * ci: remove Flyway validation step from CI workflow * ci: add Azure artifact pipeline for Flyway migrations Publishes resources/db/ as the jpo-cvmanager-flyway artifact on develop, enabling the release pipeline to build and push the cvmanager-flyway image to GCR.
* Adding notif email generation and sending to the iapi * Adding notification email examples * Adding iapi email controller * Fixing EmailServiceTest * Update EmailController.java * Update EmailServiceTest.java * Fixing broken config property refs * Updating email generator comments, colors, and content * Updating default iapi env vars to avoid localhost * Update EmailRecipient.java * Adding email frequency to tables and queries * Update launch.json * Improving getCombinedResponseEntity to handle partial failures * Removing secure env vars from application.yaml * Renaming email template to email_template.html * Removing duplicate dependency * Update services/intersection-api/api/src/main/java/us/dot/its/jpo/ode/api/emails/generators/IntersectionNotificationSummaryEmailGenerator.java Co-authored-by: Matt Cook <mattheworion.cook@gmail.com> * Handling unsubscribe URL generation errors * Removing unused methods * Cleaning up email generation code * Writing email provider tests * Adding sql update script * Update email_subscription_type_update.sql * SUpporting authenticated SMTP server * Regenerating email postgres JPA objects * Working email sending with JPA repositories * Temporarily force enabling emailer * Renaming INTERSECTION_SMTP_SERVER_HOST * Adding spring mail ssl trust policy * Adding python EmailApi * Adding IAPI endpoints for pyapi emails * Adding IAPI email generation to addons * Removing unused environment variables * Update docker-compose-addons.yml * Reverting pyapi iapi email gen * Revert "Reverting pyapi iapi email gen" This reverts commit c81a1e7. * Removing unnecessary email endpoints * Revert "Removing unnecessary email endpoints" This reverts commit 2270355. * Writing tests for email generators * Adding snapshot tests for email generators * Updating intersectionNotificationSummary snapshot * Improving emailApi non-success handling * Hardcoding gmail for testing * Adding logging for email recipients * Adding additional iapi email logging * Adding transport listener for email send debugging * Adding sentDate header * Reverting add transport listener * Reverting email logging changes * Combining GlobalExceptionHandlers * Update EmailService.java * Update launch.json * Revert "Merge branch 'iapi-email-generation-pyapi' into iapi-email-generation-extra-types" This reverts commit 0c148f8, reversing changes made to a8930e8. * Moving support-requests endpoint to EmailController * Reducing duplication in KeycloakApi * Resolving unit tests comments * Update services/intersection-api/api/src/main/java/us/dot/its/jpo/ode/api/models/keycloak/TokenPostRequestKeycloak.java Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update services/intersection-api/api/src/test/java/us/dot/its/jpo/ode/api/SnapshotTestUtils.java Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update services/intersection-api/api/src/main/java/us/dot/its/jpo/ode/api/models/emails/contents/serialization/InstantToMillisSerializer.java Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Adding no unsubscribe email template * Update EmailServiceTest.java * Update SnapshotTestUtils.java * Condensing shared keycloak api code * Using EmailResponseException with email send APIs * Removing Instant to/from Millis de/serializers * Escaping HTML in email templating * Tweaking unit tests * Updating email test snapshot * Update EmailServiceTest.java * Updating unit tests * Creating python EmailApi * Removing contact-support and rsu-error-summary python code * Adding email sending env vars to pyapi * Integrating support and rsu error iapi emails into webapp * Reverting unnecessary changes * Adding Schema annotations and removing unused code * Improving mocking in email generator tests * Updating iapi email generator test url * Updating email snapshot formatting * Update intersection_notification_summary_email_snapshot.html * Revert update settings.json * Reverting removal of emailSender * Updating unit tests * Update AdminIntersectionTab.test.tsx.snap * Update IntersectionNotificationSummaryEmailGeneratorTest.java * Adding Schema annotations to Email contents objects * Update MessageCountEmailContents.java * Working firmware and counts email sending * Converting smtp_error_handler to EmailApi * Update test_smtp_error_handler.py * Handling serialization of datetimes in counts emailer * Updating addon email endpoint names * Update email_api.py * Update email_api.py * Updating message counts email recipients * Updating email templates with no max-width * Adding ENABLE_ERROR_EMAILS to python api * Removing KeycloakApi * Adding keycloak_api direct to keycloak * Update rsu_error_summary_email_snapshot.html * Updating unit tests * Potential fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * Cleaning up email_api and keycloak_api * Updating RsuErrorSummary email to only use requestors email * Removing RsuErrorSummary recipients * Update UserController.java * Update EmailServiceTest.java * Fixing python api KC_SA_CLIENT_ID Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * Removing unused env vars from README Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * Updating smtp error handler stack trace processing Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * Improving error handling in email_api Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * Cleaning up email_api documentation Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * Cleaning up email_api env vars and imports * Update test_smtp_error_handler.py * Updating javadoc comments and handling null emails from token Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update EmailSendResponse.java * Removing RsuErrorSummary dialog * Updatin rsu error summary email text * Updating test snapshots * Updating docstring comments throughout emailing code Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * Update DisplayRsuErrors.tsx * Update test_smtp_error_handler.py * Switching auth token getters to lombok * Updating EmailGeneratorTests with MockitoExtension * Adding conditional unsubscribe url rendering * Update SupportRequestEmailGeneratorTest.java * Adding email generator mocked tests * Tweaking logging in snapshot utils * Tweaking logging in snapshot utils * Parameterize shouldHandleEmailInVariousFormats test Co-authored-by: Brandon Payne <bpayne@trihydro.com> * Update CvManagerAuthTokenTest.java * Adding private methods to keycloak_api * Cleaning up unused code * Documenting keycloak_api test fixtures * Adding python-keycloak to addons * Adding missing KC_ENDPOINT and KC_REALM to addons * Tweaking addon test environment variables * Updating addon email formatting * Reverting test exception in yunex_upgrader * Adding deployment title to message counts email subject * Escaping html in email templating, fixing upgrade runner imports * Delete email_template_no_unsubscribe.png * Updating diff_percent schema definition * Adding custom message counts email template * Adding firmware upgrade failure email template * Reverting AdminIntersectionTab test changes * Adding email rest content validation * Create EmailControllerTest.java * Update IntersectionNotificationSummaryEmailContents.java * Adding early returns when recipients are empty --------- Co-authored-by: Matt Cook <mattheworion.cook@gmail.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Co-authored-by: Brandon Payne <bpayne@trihydro.com>
* Adding notif email generation and sending to the iapi * Adding notification email examples * Adding iapi email controller * Fixing EmailServiceTest * Update EmailController.java * Update EmailServiceTest.java * Fixing broken config property refs * Updating email generator comments, colors, and content * Updating default iapi env vars to avoid localhost * Update EmailRecipient.java * Adding email frequency to tables and queries * Update launch.json * Improving getCombinedResponseEntity to handle partial failures * Removing secure env vars from application.yaml * Renaming email template to email_template.html * Removing duplicate dependency * Update services/intersection-api/api/src/main/java/us/dot/its/jpo/ode/api/emails/generators/IntersectionNotificationSummaryEmailGenerator.java Co-authored-by: Matt Cook <mattheworion.cook@gmail.com> * Handling unsubscribe URL generation errors * Removing unused methods * Cleaning up email generation code * Writing email provider tests * Adding sql update script * Update email_subscription_type_update.sql * SUpporting authenticated SMTP server * Regenerating email postgres JPA objects * Working email sending with JPA repositories * Temporarily force enabling emailer * Renaming INTERSECTION_SMTP_SERVER_HOST * Adding spring mail ssl trust policy * Adding python EmailApi * Adding IAPI endpoints for pyapi emails * Adding IAPI email generation to addons * Removing unused environment variables * Update docker-compose-addons.yml * Reverting pyapi iapi email gen * Revert "Reverting pyapi iapi email gen" This reverts commit c81a1e7. * Removing unnecessary email endpoints * Revert "Removing unnecessary email endpoints" This reverts commit 2270355. * Writing tests for email generators * Adding snapshot tests for email generators * Updating intersectionNotificationSummary snapshot * Improving emailApi non-success handling * Hardcoding gmail for testing * Adding logging for email recipients * Adding additional iapi email logging * Adding transport listener for email send debugging * Adding sentDate header * Reverting add transport listener * Reverting email logging changes * Combining GlobalExceptionHandlers * Update EmailService.java * Update launch.json * Revert "Merge branch 'iapi-email-generation-pyapi' into iapi-email-generation-extra-types" This reverts commit 0c148f8, reversing changes made to a8930e8. * Moving support-requests endpoint to EmailController * Reducing duplication in KeycloakApi * Resolving unit tests comments * Update services/intersection-api/api/src/main/java/us/dot/its/jpo/ode/api/models/keycloak/TokenPostRequestKeycloak.java Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update services/intersection-api/api/src/test/java/us/dot/its/jpo/ode/api/SnapshotTestUtils.java Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update services/intersection-api/api/src/main/java/us/dot/its/jpo/ode/api/models/emails/contents/serialization/InstantToMillisSerializer.java Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Adding no unsubscribe email template * Update EmailServiceTest.java * Update SnapshotTestUtils.java * Condensing shared keycloak api code * Using EmailResponseException with email send APIs * Removing Instant to/from Millis de/serializers * Escaping HTML in email templating * Tweaking unit tests * Updating email test snapshot * Update EmailServiceTest.java * Updating unit tests * Creating python EmailApi * Removing contact-support and rsu-error-summary python code * Adding email sending env vars to pyapi * Integrating support and rsu error iapi emails into webapp * Reverting unnecessary changes * Adding Schema annotations and removing unused code * Improving mocking in email generator tests * Updating iapi email generator test url * Updating email snapshot formatting * Update intersection_notification_summary_email_snapshot.html * Revert update settings.json * Reverting removal of emailSender * Updating unit tests * Update AdminIntersectionTab.test.tsx.snap * Update IntersectionNotificationSummaryEmailGeneratorTest.java * Adding Schema annotations to Email contents objects * Update MessageCountEmailContents.java * Working firmware and counts email sending * Converting smtp_error_handler to EmailApi * Update test_smtp_error_handler.py * Handling serialization of datetimes in counts emailer * Updating addon email endpoint names * Update email_api.py * Update email_api.py * Updating message counts email recipients * Updating email templates with no max-width * Adding ENABLE_ERROR_EMAILS to python api * Removing KeycloakApi * Adding keycloak_api direct to keycloak * Update rsu_error_summary_email_snapshot.html * Updating unit tests * Potential fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * Cleaning up email_api and keycloak_api * Updating RsuErrorSummary email to only use requestors email * Removing RsuErrorSummary recipients * Update UserController.java * Update EmailServiceTest.java * Fixing python api KC_SA_CLIENT_ID Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * Removing unused env vars from README Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * Updating smtp error handler stack trace processing Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * Improving error handling in email_api Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * Cleaning up email_api documentation Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * Cleaning up email_api env vars and imports * Update test_smtp_error_handler.py * Updating javadoc comments and handling null emails from token Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update EmailSendResponse.java * Removing RsuErrorSummary dialog * Updatin rsu error summary email text * Updating test snapshots * Updating docstring comments throughout emailing code Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * Update DisplayRsuErrors.tsx * Update test_smtp_error_handler.py * Switching auth token getters to lombok * Updating EmailGeneratorTests with MockitoExtension * Adding conditional unsubscribe url rendering * Update SupportRequestEmailGeneratorTest.java * Adding email generator mocked tests * Tweaking logging in snapshot utils * Tweaking logging in snapshot utils * Parameterize shouldHandleEmailInVariousFormats test Co-authored-by: Brandon Payne <bpayne@trihydro.com> * Update CvManagerAuthTokenTest.java * Adding private methods to keycloak_api * Cleaning up unused code * Documenting keycloak_api test fixtures * Adding python-keycloak to addons * Adding missing KC_ENDPOINT and KC_REALM to addons * Tweaking addon test environment variables * Updating addon email formatting * Reverting test exception in yunex_upgrader * Adding deployment title to message counts email subject * Escaping html in email templating, fixing upgrade runner imports * Delete email_template_no_unsubscribe.png * Updating diff_percent schema definition * Adding custom message counts email template * Adding firmware upgrade failure email template * Reverting AdminIntersectionTab test changes * Adding email rest content validation * Adding bucket4j caching to email controller * Adding rate limit unit tests * Create EmailControllerTest.java * Update IntersectionNotificationSummaryEmailContents.java * Removing default test cache name * Potential fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * Update application.yaml * Renaming cache limit to EMAIL_RATE_LIMIT_PER_INSTANCE * Adding subject extractor to rate limiter * Update application.yaml * Adding iapi email and rate limit documentation --------- Co-authored-by: Matt Cook <mattheworion.cook@gmail.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Co-authored-by: Brandon Payne <bpayne@trihydro.com>
* Adding email_type required_role * Adding subscription retrieval and modification to EmailService * Adding email subscription endpoints * Adding email unsubscribe endpoints * Improving subscription change validation * Handling super users for email subscription auth * Update services/intersection-api/api/src/main/java/us/dot/its/jpo/ode/api/services/EmailService.java Co-authored-by: Matt Cook <matt.cook@trihydro.com> * Updating EmailService dto/mapper and query logic * Removing unused methods/classes * reformatting EmailService * Optimizing updateEmailSubscriptions to reduce queries * Removing int response from updateEmailSubscriptions * Resolving unit test errors * Moving unsub auth token to Authorization header * Removing permissionService from UnsubscribeController * Update UnsubscribeControllerTest.java * Converting sql update script to flyway migration * Tweaking daily message counts sample data * Revert changes to CVManager_SampleData.sql --------- Co-authored-by: Matt Cook <matt.cook@trihydro.com>
…334) * feat: add Flyway V1 baseline migration and dev seed data Merges CVManager_CreateTables.sql and all 19 update_scripts into a single V1__baseline.sql representing the fully-migrated schema state. Adds rsu_health table and idx_scms_health_timestamp that were present in update scripts but absent from CreateTables.sql. Moves CVManager_SampleData.sql into R__sample_data.sql as a Flyway repeatable migration with ON CONFLICT guards so it is idempotent on re-run. * chore: add Flyway configuration Adds flyway.toml with baselineOnMigrate=true so existing live databases are stamped at V1 without re-execution, outOfOrder=true to handle timestamp-versioned migrations from fork branches, and mixed=true for compatibility with the DDL-only baseline script. * docs: add Flyway migration developer guide Documents the timestamp-based naming convention, how to create and run migrations locally, how baselineOnMigrate works for existing environments, and why outOfOrder is enabled for the CDOT fork workflow. * docs: deprecate manual sql_scripts update_scripts Manual migration scripts in resources/sql_scripts/ are superseded by Flyway-managed migrations in resources/db/migration/. Directory is kept as historical reference only. * feat: integrate Flyway into Docker Compose startup - Add pg_isready healthcheck to cvmanager_postgres so dependent services wait for the database to accept connections - Remove docker-entrypoint-initdb.d volume mount (replaced by Flyway) - Add flyway service that runs migrate on startup and exits cleanly - Wire cvmanager_api to depend on flyway: service_completed_successfully so the API never starts against an un-migrated schema * ci: add Flyway migration validation job Adds validate_migrations job that spins up a PostGIS 15 container, applies all migrations with flyway migrate, and verifies state with flyway info on every PR and push to develop/cdot-release branches. Catches broken migration SQL before it reaches any environment. * fix: remove schemas param incompatible with Flyway 10 Community flyway.schemas is a Teams/Enterprise-only parameter in Flyway 10. Removing it restores compatibility; public is the PostgreSQL default. * docs: update Flyway developer guide with migration guidelines and formatting adjustments * docs(db): improve Flyway migration guide formatting and clarity * refactor(sql): move legacy update scripts to deprecated directory for reference * docs: preserve schema documentation as SQL comments and README reference Adds COMMENT ON TABLE/COLUMN for all 30 tables via a new Flyway migration (V202605211729__add_table_comments.sql), making descriptions queryable in psql/pgAdmin and portable via pg_dump. Also adds a Schema Reference section to resources/db/README.md covering all tables and critical data requirements. Source material: resources/deprecated/sql_scripts/README.md, which documented the original 14 tables but left the 16 tables added by update scripts undocumented. Both sources are now superseded. Each COMMENT statement is guarded with a DO/EXCEPTION block so the migration tolerates databases that were baselined at V1 before all update scripts were manually applied. * docs: improve formatting and table organization in schema reference * fix(docker-compose): disable restart policy for service This prevents infinite restarts of the flyway service during a migration failure * feat(db): add Dockerfile for cvmanager-flyway migration image Builds a self-contained Flyway image from flyway/flyway:10-alpine with versioned migration SQL files baked in. The V*.sql glob excludes R__sample_data.sql (dev-only seed data) without any explicit list to maintain. Adding a new migration only requires adding a file to resources/db/migration/ and rebuilding the image. * feat(k8s): replace pg-init-tables ConfigMap with Flyway migration Job The embedded pg-init-tables ConfigMap was stale (16 tables vs 25+ in the current schema) and used docker-entrypoint-initdb.d, which only runs on an empty data directory and cannot apply incremental schema updates. Remove the ConfigMap and its volume mount from the Postgres Deployment. Add a wait-for-migrations init container to the API Deployment that polls flyway_schema_history before allowing the API to start, ensuring the Job completes before any application traffic reaches the database. * ci: validate Flyway image build on each PR Adds a build_flyway_image job that builds the cvmanager-flyway image from resources/db/Dockerfile on every PR. Catches Dockerfile or SQL syntax errors before they reach a deployment. Extend with push steps and registry credentials to publish the image as part of the release pipeline. * docs(k8s): document Flyway image build and deployment workflow Replaces the old migration section (which referenced the removed pg-init-tables ConfigMap) with instructions for building and pushing the cvmanager-flyway image, the kubectl deployment order, and the steps for adding new migrations going forward. * refactor(k8s): simplify flyway Job to use custom image instead of ConfigMaps Remove the flyway-sql and flyway-conf ConfigMaps that embedded migration SQL inline. The cvmanager-flyway Docker image (resources/db/Dockerfile) now carries both the SQL files and flyway.toml, making the ConfigMaps redundant. The Job shrinks from ~860 lines to 42 with no embedded content to keep in sync. * test(rsu-info-bridge): replace manual SQL init with Flyway migrations Switch Testcontainers setup to use Spring Boot's Flyway autoconfiguration instead of copying CVManager_CreateTables.sql and CVManager_SampleData.sql (which no longer exist after the migration to resources/db/migration/). Flyway runs all V* and R* migrations from resources/db/migration/ against the PostGIS container before the JPA context initializes. No schema logic lives in TestcontainersConfiguration anymore. * fix(rsu-info-bridge): resolve Flyway migration path via Maven testResources Flyway's filesystem: location is silently skipped when the working directory at test time doesn't match expectations (failOnMissingLocations defaults to false). Switch to classpath:db/migration and use Maven testResources to copy resources/db/migration/ onto the test classpath, which is always reliable regardless of JVM working directory. * fix(rsu-info-bridge): wire Flyway explicitly for Spring Boot 4 test context Spring Boot 4.0.3 removed FlywayAutoConfiguration from spring-boot-autoconfigure, so spring.flyway.* properties in application-test.yaml were silently ignored and migrations never ran against the Testcontainer database. Add an explicit @bean Flyway to TestcontainersConfiguration that calls migrate() during context startup. Remove the now-dead spring.flyway.* block from application-test.yaml and the unused @testcontainers annotation from RsuServiceTest (container lifecycle is managed by @Serviceconnection, not @testcontainers). * ci(k8s): publish cvmanager-flyway to GHCR and gate build on passing migrations - build_flyway_image now requires validate_migrations to succeed before running - On push to develop/cdot-release*, push to ghcr.io/<org>/cvmanager-flyway with sha-<short-sha> and branch-name tags via docker/build-push-action - PR builds remain smoke-test-only (no push, no registry credentials needed) - Update K8s YAML image placeholder to GHCR path and annotate PG_DB_NAME - Update K8s README to document CI-driven publish and manual fallback - Add Kubernetes deployment pointer section to resources/db/README.md * Delete services/rsu-info-bridge/CLAUDE.md * Delete CLAUDE.md * docs: update future migration file naming convention * fix(db): enforce integrity constraints and cascade deletes - Add UNIQUE on users.keycloak_id (Keycloak lookup index + one-to-one invariant) - Add UNIQUE on rsu_organization(rsu_id, organization_id) with dedup - Add UNIQUE on user_organization(user_id, organization_id) with dedup - Add UNIQUE on intersection_organization(intersection_id, organization_id) with dedup - Add ON DELETE CASCADE on rsu_options and consecutive_firmware_upgrade_failures - Add ON DELETE CASCADE on user_organization(user_id) to fix Keycloak removeUser bug * perf(db): replace redundant indexes with targeted composite indexes - Drop 7 redundant indexes that duplicate UNIQUE constraint or PK indexes - Drop idx_user_organization made redundant by UNIQUE constraint in prior migration - Drop unused idx_scms_health_timestamp (no query uses timestamp-only predicate) - Add (rsu_id, timestamp DESC) composite index on ping, rsu_health, scms_health * docs(db): add schema integrity review summary to README Points readers to migration files for change rationale and documents deferred items (obu_ota_requests FK gap, Keycloak schema review). * docs(adr): add ADR-0001 for Flyway database migrations Records the decision to adopt Flyway 10 Community Edition, covering rationale, key configuration choices (timestamp versioning, baselineOnMigrate, outOfOrder, repeatable migrations), deployment integration, alternatives considered, and consequences. * docs(readme): add Architecture Decision Records section with index table * style: remove em dashes from ADR and README per style guide * style: update comment formatting in TestcontainersConfiguration Clarify comment about FlywayAutoConfiguration removal in Spring Boot 4 by adjusting punctuation for consistency. * docs(db): update migration naming convention and deprecated scripts reference - Adjust migration file naming convention to use `HHmm` for timestamps. - Update deprecated scripts reference path in the README for clarity. - Revise Kubernetes Flyway YAML comments to reference the README for migration standards. * fix: address Copilot review feedback on migrations PR - Use PGPASSWORD env var in API initContainer psql command so credentials are not exposed in process args - Add PG_DB_NAME fail-fast check to initContainer to surface misconfiguration immediately rather than looping indefinitely - Mount resources/db/flyway.toml in both CI migration steps so validateMigrationNaming, mixed, and other shared settings are enforced consistently with Docker Compose and Kubernetes - Pin CI PostGIS service image to postgis/postgis:15-3.4 (was floating 15-master tag) to match the Testcontainers tag used in rsu-info-bridge - Fix broken relative link in deprecated sql_scripts README * ci: add Flyway migration validation step to CI workflow * refactor(db): rename timestamp migration to sequential V2 The project is switching from timestamp-based to sequential integer versioning. V202605211729__add_table_comments.sql is renamed to V2__add_table_comments.sql while still on the unmerged feature branch, so no applied-migration checksums are at risk. * refactor(db): disable outOfOrder in Flyway configuration Switch to strict in-order migration application. Version conflicts now surface as merge conflicts that must be resolved explicitly, rather than being silently bypassed with outOfOrder execution. * docs(db): update versioning documentation for sequential integer scheme Replace timestamp-based versioning rationale with sequential integer rationale across the ADR, db README, and Kubernetes README. Remove the outOfOrder section from the developer guide and update all naming convention examples and the schema reference table to reflect V{N}. * ci: remove Flyway validation step from CI workflow * feat(db): enforce remaining relational constraints Adds a UNIQUE constraint on firmware_upgrade_rules(from_id, to_id) to prevent duplicate upgrade path entries. A self-healing dedup block runs first and reports removed rows via RAISE NOTICE, consistent with the pattern established in V202605221641. Adds ON DELETE CASCADE to junction table FKs (rsu_intersection, rsu_organization, intersection_organization) so that deleting a parent RSU, intersection, or organization automatically removes membership rows that have no meaning without both parents. Previously these deletes were blocked, requiring callers to manually remove children first. Adds ON DELETE CASCADE to structural child tables snmp_msgfwd_config and max_retry_limit_reached_instances, whose rows are RSU-specific and meaningless after RSU deletion. Telemetry tables (ping, rsu_health, scms_health) are intentionally left as RESTRICT — high-volume time-series data should require explicit pruning before a parent RSU can be removed. * feat(db): add indexes for repository query patterns Adds six indexes derived from an audit of all 21 Spring Data repositories. PostgreSQL does not automatically index FK columns; several FK columns used as leading WHERE or JOIN ON predicates had no index at all. - idx_user_organization_organization_id: covers org-first lookups in UserOrganizationRepository (findByOrganization_Name). The existing UNIQUE constraint is user-first and does not serve these queries. - idx_user_email_notification_email_type_id: covers all three bulk notification recipient queries in UserEmailNotificationRepository, each of which filters by email_type_id as the leading predicate. - idx_firmware_upgrade_rules_from_id: covers findFirstByFrom_Id in FirmwareUpgradeRuleRepository. FK column with no index caused a full sequential scan on every upgrade path resolution. - idx_rsu_options_tim_deposit (partial, WHERE tim_deposit = true): covers rsu-info-bridge findByRsuOptionTimDepositIsTrue. Partial index keeps only TIM-deposit-enabled rows, matching the query predicate exactly. - idx_rsus_primary_route: covers SELECT DISTINCT primary_route ORDER BY primary_route used to populate the primary route dropdown. Allows PostgreSQL to read distinct sorted values directly from the index. - idx_rsu_intersection_intersection_id: covers intersection-first lookups and DELETE operations in RsuIntersectionRepository. The UNIQUE constraint (rsu_id, intersection_id) is rsu-first and does not serve these patterns. * refactor(api): sync JPA entity annotations with database schema Add unique = true to @column on all lookup/identity columns that have corresponding UNIQUE constraints in SQL (manufacturers.name, rsu_models.name, firmware_images.name/install_package/version, organizations.name, snmp_protocols.nickname, roles.name, snmp_msgfwd_type.name, email_type.email_type, rsus.ipv4_address/serial_number/iss_scms_id, users.keycloak_id/email). Add composite @UniqueConstraint on Rsu (milepost, primary_route) and on all three junction tables (user_organization, rsu_organization, intersection_organization). Add @onDelete(CASCADE) on FK fields whose SQL constraints were updated to ON DELETE CASCADE in V202605221641 and V202605261241: RsuOption.rsu, ConsecutiveFirmwareUpgradeFailure.rsu, UserOrganization.user, RsuOrganization.rsu/organization, IntersectionOrganization.intersection/organization, RsuIntersection.rsu/intersection, SnmpMsgfwdConfig.rsu, MaxRetryLimitReachedInstance.rsu. RsuCredential.nickname and SnmpCredential.nickname remain unsynced due to file permission restrictions; both need unique = true added. * refactor(api): push validation queries to database layer Replace per-RSU existsByRsuAndIntersection N+1 loop in patchIntersection with a single JPQL subquery (findRsusNotYetAssociated on RsuIntersectionRepository) that filters already-associated RSUs in one round trip. Replace full-entity fetch-then-count in deleteMultipleRsusByIpv4Address and deleteMultipleUsersByEmail with countByIpv4AddressIn / countByEmailIn derived queries. Adds deleteByEmailIn to UserRepository so the user delete path no longer requires loading entity objects. * test(api): update UserManagementService tests for count-based validation deleteMultipleUsersByEmail now uses countByEmailIn + deleteByEmailIn instead of findByEmailIn + deleteAll. Updated four tests to stub the count query, assert the generic "One or more user(s) not found" message, and verify the correct repository methods are called. * fix(db): enforce user_organization org cascade and harden constraint migrations - Add ON DELETE CASCADE to user_organization(organization_id) for parity with rsu_organization and intersection_organization, so deleting an organization with user memberships no longer fails. role_id is intentionally left RESTRICT (roles are reference data and must not be deletable while assigned); UserOrganization.organization gets the matching @onDelete to keep the entity in sync with the schema. - Guard every DROP CONSTRAINT with IF EXISTS so a database whose FK was created under a different name degrades gracefully instead of failing the migration. - Remove explicit BEGIN/COMMIT; Flyway already wraps each migration in its own transaction, and the explicit control statements interfered with that. * docs(db): document remaining migrations and Flyway-owned schema - Add the V202605261241 and V202605261242 migrations to the Schema Integrity Review table (they were previously undocumented). - Note that the schema is Flyway-owned (no ddl-auto), so entity-level unique/@OnDelete/ @UniqueConstraint annotations are documentation only with no runtime effect. - Add a flyway repair caveat for local databases that already applied the edited migrations. * fix(api): restore specific not-found messages and de-duplicate bulk delete input The count-based validation in deleteMultipleUsersByEmail and deleteMultipleRsusByIpv4Address replaced the specific "not found with X" message with a generic one and broke on duplicate input (count of distinct rows != list size). - De-duplicate the input list before comparing against the count, so a request listing the same existing user/RSU twice succeeds instead of failing as not found. - On the not-found path only, load the existing keys via new lightweight projection queries (UserRepository.findExistingEmails, RsuRepository.findExistingIpv4Addresses) and rebuild the exact missing-id list, keeping the cheap count on the happy path. - Add the missing deleteMultipleRsusByIpv4Address test suite (success, some-not-found, empty, invalid IP, duplicates) and a duplicate-email guard for the user path. * fix(db): rename timestamp migrations to sequential V3-V6 The four schema-integrity migrations were named with the timestamp scheme (V202605221641 etc.) that commit 4896d85 retired in favor of sequential integers. This contradicted the documented convention in resources/db/README.md and poisoned the "increment the highest V{N}" workflow: a future V3-V6 would sort below the already-named timestamp versions. Rename to V3-V6, fix each file's header comment and cross-references, and update the schema-evolution table in README.md. These four migrations have not been applied to any shared environment, so renaming carries no Flyway checksum risk. V2 is intentionally left untouched: it is already applied, so changing even its comments would break checksum validation. * fix(api): remove Hibernate @onDelete annotations from intersection-api entities Flyway-managed ON DELETE CASCADE FK constraints make these Hibernate-specific annotations redundant. Relying on the DB-level cascade keeps the entity model portable and avoids the org.hibernate.annotations dependency. * chore(db): increment migration versions from V3-V6 to V4-V7 * chore(db): consolidate constraint migrations V4+V6 into V4__schema_constraints Merges the two interleaved constraint passes (V4__schema_constraint_integrity and V6__remaining_relational_constraints) into a single migration grouped by entity/table. Each section contains UNIQUE constraint work, dedup blocks, and CASCADE FK changes for the same table, making the per-table intent clear in one place. Top-level block headers removed; section-level comments retained with cascade reasoning for non-SQL readers. * chore(db): consolidate index migrations V5+V7 into V5__schema_indexes Merges the two interleaved index passes (V5__schema_index_optimization and V7__service_pattern_indexes) into a single migration. Drops come first (nine redundant or unused indexes), followed by new indexes grouped by table. COMMENT ON INDEX statements retained. Cross-reference to user_organization_unique updated to point to V4 (was stale V3 after prior renumbering). * chore(intersection-api): revert out-of-scope service/repo optimizations These changes (N+1 fix, count-based validation, deduplication, bulk delete) are valid improvements but unrelated to the V4/V5 schema migrations. Reverting to keep this PR focused on DB normalization. * docs(db): remove Schema Integrity Review section from README * test: seed fixtures to exercise V4 constraints and V5 indexes Add a 3rd cascade-demo RSU (full CASCADE children, no telemetry) and seed the previously-empty ping, rsu_health, scms_health, consecutive_firmware_upgrade_failures, and max_retry_limit_reached_instances tables so the new integrity constraints and indexes can be exercised on a local stack. - RSU 1 & 2 gain telemetry rows -> RESTRICT-on-delete demo - RSU 3 has all CASCADE children but no telemetry -> clean cascade on delete - Org 2 wired with all three junction types -> org-level CASCADE demo - Telemetry inserts guarded with WHERE NOT EXISTS to stay idempotent across repeatable-migration re-runs * chore(db): expand local dev seed data for full hybrid org/resource model - Add third organization and ensure all three orgs receive shared and org-specific RSUs, intersections, and users. - Expand manufacturers, RSU models, firmware images, and upgrade rules. - Populate CASCADE and RESTRICT child/telemetry tables for thorough schema integrity testing. - Update insertion logic to ensure idempotency across migration re-runs. * feat(db): enforce frontend data constraints at the database layer Add four CHECK/NOT NULL constraints to V4__schema_constraints.sql that close gaps between application-level validation and the database schema: - rsus.milepost >= 0 (UI enforces non-negative via digits-only regex) - users.first_name / last_name NOT NULL (both required on all admin forms; NULLs backfilled to empty string before tightening) - roles.name IN ('admin','operator','user') (only three role names are recognised by the application; open varchar allowed silent bad inserts) - intersections.intersection_number ~ '^[0-9]+$' (NTCIP IDs are numeric; UI enforces digits-only regex) Each constraint is guarded by a DO block that aborts the migration with RAISE EXCEPTION if non-conforming rows exist, so the migration is safe against pre-existing bad data. Also documents the new constraints and the intentionally app-layer-only min-one-organization rule in resources/db/README.md. * test(rsu-info-bridge): prune telemetry rows before RSU cleanup in integration tests The expanded R__sample_data.sql seeds ping, rsu_health, and scms_health rows, and those tables intentionally keep RESTRICT FKs to rsus (V4), so RsuServiceTest.clearRsuData() could no longer delete all RSUs. Add minimal cleanup-only test entities and JPA repositories for the three telemetry tables and clear them (deleteAllInBatch) before deleting RSUs, mirroring the explicit pruning the RESTRICT FKs require. * fix(db): restrict organization deletion instead of cascading junction rows Resources (RSUs, intersections, users) may belong to multiple organizations, and the minimum-one-organization rule is enforced only in the application layer (admin_org.delete_org_authorized orphan checks). A CASCADE on organization_id would let any code path that bypasses those checks silently orphan resources at the database level. Change the organization_id FKs on rsu_organization, user_organization, and intersection_organization from ON DELETE CASCADE to ON DELETE RESTRICT. CASCADE remains on the resource side (rsu_id, intersection_id, user_id), where deleting the resource legitimately removes all of its memberships. Addresses PR #334 review feedback from @drewjj.
* Fixed issue where dismissing a notification returns a 500 error * Updating Status code to match what the function actually returns * Removing dead code from Unit test * making longitude / latitude required since the code will error without them.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
6 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
PR Details
Description
How Has This Been Tested?
Types of changes
Checklist: