Skip to content

Add CLUE clustering #1699

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 21 commits into from
May 15, 2025
Merged

Add CLUE clustering #1699

merged 21 commits into from
May 15, 2025

Conversation

tomeichlersmith
Copy link
Member

@tomeichlersmith tomeichlersmith commented May 5, 2025

I am copying over the CLUE clustering, additional analyzer and update to the SimCalorimeterHit that was originally developed in Summer 2024 to study ECal clustering at Lund by Ella Viirola. The visualization code based on Phoenix I plan to move into its own repository and the JSON-producers that prepare an event to be visualized I plan to put into EventDisplay.

What are the issues that this addresses?

This resolves #1411

Check List

  • I successfully compiled ldmx-sw with my developments.
  • I read, understood and follow the coding rules.
  • I ran my developments and the following shows that they are successful.

@Lysarina originally did these developments and includes evidence of their success in the original PR #1416

To Do

  git switch ella-dev-clustering
  git rebase trunk
  git switch 1411-add-clue-clustering
  git checkout ella-dev-clustering -- Ecal/

Ella Viirola originally authored these files while doing summer research
into ECal clustering at Lund University

Co-authored-by: Lysarina
… than the incident

copied over from work done originally by Ella Viirola during Summer 2024
at Lund.

Co-authored-by: Lysarina
@tomeichlersmith tomeichlersmith requested review from tvami and EinarElen May 5, 2025 17:47
@tomeichlersmith
Copy link
Member Author

@bryngemark The fact that the PR Validations pass completely leads me to believe that the issues you were observing before are not present (at least within the clean CI environment.) The total energy is appropriately shifted high and is not the discrete distribution you've shown before.

image

Since this distribution is cut off at higher energies, I'm re-running the it_pileup job locally so that I can make sure it still works above the 12GeV line.

@tomeichlersmith tomeichlersmith requested a review from bryngemark May 6, 2025 16:07
Copy link
Member

@tvami tvami left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I skimmed thru quickly, I'll have a 2nd read-thru later, but two general comments:

  • the variable names are not according to our coding rules
  • there are couts and debug flags that should go away into the logging system (everything under debug should be trace in our logging system)

@tomeichlersmith
Copy link
Member Author

@bryngemark I have confirmed that, locally and running from scratch, the IT overlay is working as expected. As a double check, I just looked at the EcalVeto_overlay.summedDet_ branch like you have and I see a normal distribution.

image

Without being able to replicate your issue, I would like to ask you to provide more detail on the environment and inputs you used to produce the plot in the original issue. What version of ldmx-sw produced the overlay events? What version produced the "main" events? What version ran overlay? Those sort of questions. If the environment is on SLAC's S3DF, I'd be happy to look at it directly as well if you give me the directory (and I have access to read that directory).

@bryngemark
Copy link
Contributor

that's great, @tomeichlersmith. i was getting hopeful when the validation pileup test went through.

i ran on ella's dev branch (that the PR was made from) locally on my laptop. i confirmed that it was behaving differently than the then-current trunk on the same machine (which showed reasonable energies with overlay); that was sometime early fall. i don't think we have any reason to believe whatever happened is present here (in fact we have reasons to believe it isn't) so i suggest we merge and then if i reproduce the bug with new trunk, i make an issue.

@tomeichlersmith tomeichlersmith requested a review from tvami May 9, 2025 18:24
@tomeichlersmith
Copy link
Member Author

@tvami I have not moved all of the variables in the CLUE implementation to lower_case (amongst other violations of our coding rules) because I fear breaking the implementation when I'm doing it manually. I am currently trying to figure out how to use clang-tidy to try to have it done algorithmically, but I also feel like that is outside the scope of this PR. Can we merge this despite its failure to completely follow the coding rules? I can put my clang-tidy research into an issue and maybe we can have a workflow that enforces more of our coding rules like clang-format does.

@Lysarina
Copy link

Hi! This might already be resolved or might not even be an issue, but as I wrote in my PR, the initial cluster producer used the WorkingCluster class, that took in EcalHits as pointers. However when I started out I was kind of a pointer noob so I created WorkingEcalCluster, which does essentially the same thing (with some additional improvements), but takes in references instead of pointers in the methods. I never had time to fix this (going back to pointers), so I just wanted to point this out again, in case this would cause some issues later down the line.

Thanks again for taking this on @tomeichlersmith !

@tomeichlersmith tomeichlersmith marked this pull request as draft May 13, 2025 17:37
@tomeichlersmith tomeichlersmith marked this pull request as ready for review May 13, 2025 17:37
@tvami tvami marked this pull request as draft May 13, 2025 19:42
@tvami tvami marked this pull request as ready for review May 13, 2025 19:42
Getting factory updates to make sure things still work after trunk
update.
@tomeichlersmith tomeichlersmith requested a review from tvami May 15, 2025 16:59
@tvami tvami marked this pull request as draft May 15, 2025 17:05
@tvami tvami marked this pull request as ready for review May 15, 2025 17:05
Copy link
Member

@tvami tvami left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome, thanks a lot!

@tvami tvami changed the title add clue clustering Add CLUE clustering May 15, 2025
@tomeichlersmith tomeichlersmith merged commit ed7f1f8 into trunk May 15, 2025
12 of 13 checks passed
@tomeichlersmith tomeichlersmith deleted the 1411-add-clue-clustering branch May 15, 2025 21:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Update Ecal clustering with the CLUE algorithm
4 participants