Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Store log output in datamodel extension #8855

Open
stscijgbot-jp opened this issue Oct 2, 2024 · 3 comments · May be fixed by #8857
Open

Store log output in datamodel extension #8855

stscijgbot-jp opened this issue Oct 2, 2024 · 3 comments · May be fixed by #8857
Labels
Software Affected: datamodels team_periwinkle

Comments

@stscijgbot-jp
Copy link
Collaborator

Issue JP-3773 was created on JIRA by Tyler Pauly:

JWSTDMS-485 documents the desire for an HST-like trailer file which stores the processing output log during science product generation. JSDP-2860 is tracking SDP's work to generate such an external file, while this ticket is to explore the storage of the log output of the CAL pipeline in a datamodel extension/object/attribute. Such an attribute already appears to exist and is in use for exactly this purpose in Roman processing, such that this work should be straightforward to implement on the JWST side; the primary blocker/investigation remaining is in understanding what information needs to be scrubbed to maintain IT security. From DMS planning, scrubbed information should be:

- Node/machine names
- IP addresses
- usernames who have access to those machines
- filesystems paths
@stscijgbot-jp
Copy link
Collaborator Author

Comment by David Law on JIRA:

Adding a note that this is a indeed a feature still requested by users; I was discussing this at a science meeting just this week.

@tapastro tapastro linked a pull request Oct 4, 2024 that will close this issue
Open
10 tasks
@stscijgbot-jp
Copy link
Collaborator Author

Comment by Pey-Lian Lim on JIRA:

Do you dump tokens and such? Might wanna scrub that out too.

Is this new extension name defined already? If so, where is the specs? Jdaviz might be interested because we have a Metadata plugin.

@stscijgbot-jp
Copy link
Collaborator Author

Comment by Katie Kaleida on JIRA:

This approach seems like the best way to meet the need of a trailer file, as described in JWSTDMS-485, but without the potential difficultly of needing to scrub any information that should not be available to the general public that the SDP solution would have.  CAL should proceed forward with working this ticket JP-3773 ticket in priority order in the 11.3 development timeframe.  We can then collect feedback from the community if this is meeting their needs, or if an HST-like trailer file is really needed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Software Affected: datamodels team_periwinkle
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

JP-3773: Store logs from calibration pipeline in datamodel tapastro/jwst
1 participant
@stscijgbot-jp