Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add ReactionMiner backend job, skip duplicate emails, fix casing for novoStoic in emails #71

Merged
merged 22 commits into from
Jan 29, 2025

Conversation

bodom0015
Copy link
Member

@bodom0015 bodom0015 commented Jan 23, 2025

Problem

We need a way to run ReactionMiner using mmli-backend

Approach

  • feat: added ReactionMiner as a JobType, added necessary configuration
  • feat: update pvc-explorer pod to mount full mmli1 NFS share (for debugging)
  • fix: use correct casing for novoStoic when sending email notifications
  • fix: use a trigger file in MinIO to test if email has already been sent - skip sending email if this file already exists
  • feat: support uploading file_content as string in MinIOService.upload_file()

How to Test

Currently deployed to mmli-backend-staging

  1. Navigate to https://mmli.fastapi.staging.mmli1.ncsa.illinois.edu/docs
  2. Upload a file to the new reactionminer bucket
    • You can use your own PDF file, or use the example copper_acetate.pdf file that is attached to this PR
    • You can leave job_id blank and it will create + return a new job_id for you to use - save this job_id for later
  3. Create a new job with job_type=reactionminer - make sure to use the same job_id where you uploaded the PDF file
    • You should see that a new Pod is spawned to run this job on the cluster in the staging namespace
  4. Check the status of the job using job_type=reactionminer and the same job_id where you uploaded the PDF file
    • The job should complete after ~5 minutes
  5. When the job is completed, check the output files using bucket_name=reactionminer and the same job_id
    • You should receive a mapping of all output files and their JSON contents

@bodom0015 bodom0015 changed the title [WIP] feat: add ReactionMiner backend job [WIP] feat: add ReactionMiner backend job, skip duplicate emails, fix casing for novoStoic in emails Jan 23, 2025
@bodom0015 bodom0015 changed the title [WIP] feat: add ReactionMiner backend job, skip duplicate emails, fix casing for novoStoic in emails feat: add ReactionMiner backend job, skip duplicate emails, fix casing for novoStoic in emails Jan 24, 2025
@ckouder
Copy link
Contributor

ckouder commented Jan 24, 2025

tested with b59e86d9e23148af9d596e1433ee1787

everything looks good! Thank you Sara! 🎉

@ckouder ckouder self-requested a review January 27, 2025 15:29
Copy link
Contributor

@ckouder ckouder left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good job Sara!

@bodom0015
Copy link
Member Author

Thank you, @ckouder! 😄

@bodom0015 bodom0015 merged commit 26143c8 into main Jan 29, 2025
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants