Follow-up to add HLT tracking monitoring in cosmics

[mmusich]

resolves #47662

PR description:

This is a follow-up to PR #47542 and integrates the (tracking) HLT monitoring during cosmics in the DQM Offline sequences used at Tier0 when the cosmics scenario is selected.
This resolves issue #47662.
In addition wf 7.25 that was introduced at #47542 is enhanced in order to actually test the same setup that is foreseen to be used at Tier0.

PR validation:

Run the command:

runTheMatrix.py -l 7.25 -t 4 -j 8 --nEvents 100000

successfully and observed filled DQM plots in the output file.
Also run the following commands:

cmsDriver.py step3 -s RAW2DIGI,L1Reco,RECO,ALCA:MuAlGlobalCosmics,DQM:@HLTMon --conditions 150X_dataRun3_Express_v1 --datatier GEN-SIM-RECO,DQMIO --eventcontent RECOSIM,DQM -n 1000 --era Run3_2025 --scenario cosmics --filein root://eoscms.cern.ch://eos/cms/tier0/store/data/Commissioning2025/CosmicHLTMonitor/RAW/v1/000/389/873/00000/3ea592ff-5490-47ed-8242-fff796875a01.root

cmsDriver.py step5 -s HARVESTING:@HLTMon --conditions 150X_dataRun3_Express_v1 --filein file:step3_RAW2DIGI_L1Reco_RECO_ALCA_DQM_inDQM.root --scenario cosmics --filetype DQM --era Run3_2025 -n 1000

and obtained the following DQM output file: https://cernbox.cern.ch/s/ez9ceBBuCEkjFr2

If this PR is a backport please specify the original PR and why you need to backport that PR. If this PR will be backported please specify to which release cycle the backport is meant for:

Not a backport, but will be backported to CMSSW_15_0_X for 2025 data-taking purposes.

[cmsbuild]

cms-bot internal usage

[mmusich]

test parameters:

• workflow = 7.25

[cmsbuild]

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-47663/44199

• There are other open Pull requests which might conflict with changes you have proposed:
  • File Configuration/PyReleaseValidation/python/relval_steps.py modified in PR(s): Updates for runTheMatrix.py: input checks, GPUs repartition, input recycling #47377, Level-1 Scouting DQM  #47627, Using Offline GT for 2024 Data RelVals Eras B,C,D,E #47639

[cmsbuild]

A new Pull Request was created by @mmusich for master.

It involves the following packages:

• Configuration/PyReleaseValidation (pdmv, upgrade)
• DQMOffline/Configuration (dqm)
• DQMOffline/Trigger (dqm)

@AdrianoDee, @Moanwar, @antoniovagnerini, @cmsbuild, @DickyChant, @miquork, @rseidita, @srimanob, @subirsarkar can you please review it and eventually sign? Thanks.
@Fedespring, @HuguesBrun, @Martin-Grunewald, @cericeci, @fabiocos, @jhgoh, @makortel, @missirol, @mtosi, @rociovilar, @slomeo, @threus, @trocino this is something you requested to watch as well.
@antoniovilela, @mandrenguyen, @rappoccio, @sextonkennedy you are the release manager for this.

cms-bot commands are listed here

[mmusich]

@cmsbuild, please test

[cmsbuild]

+1

Size: This PR adds an extra 76KB to repository
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-2e0b84/45152/summary.html
COMMIT: 9040eee
CMSSW: CMSSW_15_1_X_2025-03-22-1100/el8_amd64_gcc12
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/47663/45152/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

• You potentially removed 140 lines from the logs
• Reco comparison results: 4 differences found in the comparisons
• DQMHistoTests: Total files compared: 51
• DQMHistoTests: Total histograms compared: 3909043
• DQMHistoTests: Total failures: 0
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 3909023
• DQMHistoTests: Total skipped: 20
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: -85662.687 KiB( 50 files compared)
• DQMHistoSizes: changed ( 4.22,... ): 1.668 KiB HLT/Tracking
• DQMHistoSizes: changed ( 7.25 ): -10138.093 KiB SiStrip/MechanicalView
• DQMHistoSizes: changed ( 7.25 ): -6057.653 KiB PixelPhase1/Phase1_MechanicalView
• DQMHistoSizes: changed ( 7.25 ): -4579.888 KiB PixelPhase1/Tracks
• DQMHistoSizes: changed ( 7.25 ): -4512.607 KiB Hcal/DigiTask
• DQMHistoSizes: changed ( 7.25 ): -4211.243 KiB L1T/L1TStage2EMTF
• DQMHistoSizes: changed ( 7.25 ): -3592.058 KiB L1T/L1TStage2BMTF
• DQMHistoSizes: changed ( 7.25 ): -2923.854 KiB CSC/CSCOfflineMonitor
• DQMHistoSizes: changed ( 7.25 ): -2849.033 KiB EcalBarrel/EBTimingTask
• DQMHistoSizes: changed ( 7.25 ): -2629.476 KiB L1TEMU/L1TStage2CaloLayer2
• DQMHistoSizes: changed ( 7.25 ): ...
• Checked 220 log files, 187 edm output root files, 51 DQM output files
• TriggerResults: no differences found

[cmsbuild]

Pull request #47663 was updated. @AdrianoDee, @Moanwar, @antoniovagnerini, @cmsbuild, @DickyChant, @miquork, @rseidita, @srimanob, @subirsarkar can you please check and sign again.

[mmusich]

@cmsbuild, please test

[cmsbuild]

+1

Size: This PR adds an extra 84KB to repository
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-2e0b84/45153/summary.html
COMMIT: 40b7674
CMSSW: CMSSW_15_1_X_2025-03-22-1100/el8_amd64_gcc12
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/47663/45153/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

• You potentially removed 141 lines from the logs
• Reco comparison results: 575 differences found in the comparisons
• DQMHistoTests: Total files compared: 53
• DQMHistoTests: Total histograms compared: 4137261
• DQMHistoTests: Total failures: 2759
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 4134482
• DQMHistoTests: Total skipped: 20
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: -85661.439 KiB( 52 files compared)
• DQMHistoSizes: changed ( 4.22,... ): 1.980 KiB HLT/Tracking
• DQMHistoSizes: changed ( 7.25 ): -10138.093 KiB SiStrip/MechanicalView
• DQMHistoSizes: changed ( 7.25 ): -6057.653 KiB PixelPhase1/Phase1_MechanicalView
• DQMHistoSizes: changed ( 7.25 ): -4579.888 KiB PixelPhase1/Tracks
• DQMHistoSizes: changed ( 7.25 ): -4512.607 KiB Hcal/DigiTask
• DQMHistoSizes: changed ( 7.25 ): -4211.243 KiB L1T/L1TStage2EMTF
• DQMHistoSizes: changed ( 7.25 ): -3592.058 KiB L1T/L1TStage2BMTF
• DQMHistoSizes: changed ( 7.25 ): -2923.854 KiB CSC/CSCOfflineMonitor
• DQMHistoSizes: changed ( 7.25 ): -2849.033 KiB EcalBarrel/EBTimingTask
• DQMHistoSizes: changed ( 7.25 ): -2629.476 KiB L1TEMU/L1TStage2CaloLayer2
• DQMHistoSizes: changed ( 7.25 ): ...
• Checked 228 log files, 197 edm output root files, 53 DQM output files
• TriggerResults: no differences found

[Moanwar]

+Upgrade

[fabiocos]

@cms-sw/dqm-l2 @cms-sw/pdmv-l2 this PR, and its backport #47664 are needed to put in production the CosmicHLTMonitor DQM at T0. Could you please scrutinize them quickly? Fabio (as ORM)

[AdrianoDee]

+pdmv

[rseidita]

+dqm

• spurious differences seen from unstable WF 12834.7 (open issue spurious differences in outputs of Run-3 wfs *.7 #39803)
• spurious differences from unstable WF 24834.911 (open issue Non-reproducibility in TrackerPhase2OTL1Track #47071)
• P5 tests OK

[cmsbuild]

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @rappoccio, @sextonkennedy, @mandrenguyen, @antoniovilela (and backports should be raised in the release meeting by the corresponding L2)

[mmusich]

ping @cms-sw/orp-l2 we'll need a release with the backport of this included. Can you please merge?

[antoniovilela]

+1