Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Test Deployment for v1.2.1 #1416

Merged
merged 56 commits into from
Dec 20, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
56 commits
Select commit Hold shift + click to select a range
952ad17
initial commit of claw monitor
bsunnquist Jan 6, 2023
402d763
added mast query, initial stacking and plotting
bsunnquist Jan 6, 2023
0ab25da
stacks and plots now working on test case
bsunnquist Jan 10, 2023
1a87dd2
adding pointing and other info to plots
bsunnquist Jan 11, 2023
86594ee
Merge branch 'develop' of github.com:spacetelescope/jwql into make-cl…
bsunnquist Sep 28, 2023
4fa7b44
Merge branch 'develop' of github.com:spacetelescope/jwql into make-cl…
bsunnquist Sep 28, 2023
89add7a
Merge branch 'develop' of github.com:spacetelescope/jwql into make-cl…
bsunnquist Oct 6, 2023
b06080c
Merge branch 'develop' of github.com:spacetelescope/jwql into make-cl…
bsunnquist Oct 11, 2023
9af5732
Adding nircam claw databases
bsunnquist Oct 12, 2023
d6be2cc
Updates after spot checking new data run
bsunnquist Oct 16, 2023
ab2d803
adding new columns and other database issues
bsunnquist Oct 17, 2023
d8219c7
claw monitor webpages and databases working
bsunnquist Oct 17, 2023
8e9a1dc
added background monitoring plots and webpages
bsunnquist Oct 18, 2023
b2bd048
updates to background trending plots
bsunnquist Oct 18, 2023
b47dadb
combined sw and lw plotting in single loop
bsunnquist Oct 19, 2023
a5ac035
pep8
bsunnquist Oct 19, 2023
5b53632
pep8
bsunnquist Oct 19, 2023
7e3bc88
pep8
bsunnquist Oct 19, 2023
582e2e8
removed testing statements
bsunnquist Oct 19, 2023
056295b
a couple changes for testing purposes
bsunnquist Oct 20, 2023
82109f5
Removed zscaling; fixed potential future plot limit issue
bsunnquist Oct 20, 2023
fd08a85
Updated readnoise database entry and retrieval
york-stsci Nov 3, 2023
8f7a65c
PEP8
york-stsci Nov 6, 2023
481e8ab
Merge branch 'develop' of github.com:spacetelescope/jwql into make-cl…
bsunnquist Nov 8, 2023
744d801
addressed review comments; changed output directory for display
bsunnquist Nov 8, 2023
9bf381b
pep8
bsunnquist Nov 8, 2023
88dd012
Merge branch 'develop' into issue_1378
mfixstsci Nov 13, 2023
0ac9d5b
Merge branch 'develop' into make-claw-monitor
mfixstsci Nov 21, 2023
e7af577
Bump cryptography from 41.0.4 to 41.0.6
dependabot[bot] Nov 29, 2023
a4bacec
Merge branch 'develop' into issue_1378
mfixstsci Dec 7, 2023
a785009
filter thumbnails on visit
BradleySappington Dec 14, 2023
6f8054b
remove personal TODO
BradleySappington Dec 14, 2023
71ecba1
file path needs to be set before calling monitor methods
mfixstsci Dec 15, 2023
25490ec
Updating static path for files
mfixstsci Dec 15, 2023
7f29685
Adding server name to path for data access
mfixstsci Dec 18, 2023
75de145
Changing variable name to make it more descriptive
mfixstsci Dec 18, 2023
37bed91
Merge pull request #1394 from york-stsci/issue_1378
BradleySappington Dec 18, 2023
53c08ee
Merge branch 'develop' into visit_filter
BradleySappington Dec 18, 2023
4765df0
Merge branch 'develop' into make-claw-monitor
mfixstsci Dec 18, 2023
8087142
pep8 fixes
BradleySappington Dec 18, 2023
4c57c27
updating static file path for claw monitor
mfixstsci Dec 18, 2023
8b4c0e8
Merge branch 'develop' into dependabot/pip/cryptography-41.0.6
BradleySappington Dec 18, 2023
af58b4d
Merge pull request #1404 from spacetelescope/dependabot/pip/cryptogra…
BradleySappington Dec 18, 2023
04ba4ba
Bump scipy from 1.9.3 to 1.11.4
dependabot[bot] Dec 18, 2023
31a3416
removed testing statements
bsunnquist Dec 18, 2023
577543c
Merge branch 'develop' into make-claw-monitor
mfixstsci Dec 18, 2023
f5e3af1
Update requirements.txt
BradleySappington Dec 18, 2023
427b071
Merge branch 'develop' into visit_filter
BradleySappington Dec 18, 2023
e6023ec
Merge pull request #1412 from BradleySappington/visit_filter
BradleySappington Dec 18, 2023
2fae472
Merge branch 'develop' into dependabot/pip/scipy-1.11.4
BradleySappington Dec 18, 2023
cf2e6c2
Merge pull request #1405 from spacetelescope/dependabot/pip/scipy-1.11.4
BradleySappington Dec 18, 2023
bc49728
skipping claw stacks that are missing cal files
bsunnquist Dec 18, 2023
c79ec39
Merge branch 'develop' into make-claw-monitor
mfixstsci Dec 18, 2023
a6d6079
Merge pull request #1152 from bsunnquist/make-claw-monitor
mfixstsci Dec 18, 2023
c3c6b88
Merge pull request #1414 from spacetelescope/develop
mfixstsci Dec 19, 2023
3c27420
Updating CHANGES
mfixstsci Dec 20, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 13 additions & 0 deletions CHANGES.rst
Original file line number Diff line number Diff line change
@@ -1,5 +1,18 @@
## What's Changed

1.2.1 (2023-12-20)
==================

Web Application
~~~~~~~~~~~~~~~
- Visit Filter on query and archive thumbnails pages by @BradleySappington in https://github.com/spacetelescope/jwql/pull/1412

Project & API Documentation
~~~~~~~~~~~~~~~~~~~~~~~~~~~
- NIRCam Claw Monitor by @bsunnquist in https://github.com/spacetelescope/jwql/pull/1152
- Stop storing absolute paths in the database by @york-stsci in https://github.com/spacetelescope/jwql/pull/1394


1.2.0 (2023-11-21)
==================

Expand Down
7 changes: 5 additions & 2 deletions jwql/database/database_interface.py
Original file line number Diff line number Diff line change
Expand Up @@ -509,14 +509,16 @@ class : obj
NIRSpecCosmicRayStats = monitor_orm_factory('nirspec_cosmic_ray_stats')
NIRSpecTAQueryHistory = monitor_orm_factory('nirspec_ta_query_history')
NIRSpecTAStats = monitor_orm_factory('nirspec_ta_stats')
NIRCamClawQueryHistory = monitor_orm_factory('nircam_claw_query_history')
NIRCamClawStats = monitor_orm_factory('nircam_claw_stats')

INSTRUMENT_TABLES = {
'nircam': [NIRCamDarkQueryHistory, NIRCamDarkPixelStats, NIRCamDarkDarkCurrent,
NIRCamBiasQueryHistory, NIRCamBiasStats, NIRCamBadPixelQueryHistory,
NIRCamBadPixelStats, NIRCamReadnoiseQueryHistory, NIRCamReadnoiseStats,
NIRCamAnomaly, NIRCamCosmicRayQueryHistory, NIRCamCosmicRayStats,
NIRCamEDBDailyStats, NIRCamEDBBlockStats, NIRCamEDBTimeIntervalStats,
NIRCamEDBEveryChangeStats],
NIRCamEDBEveryChangeStats, NIRCamClawQueryHistory, NIRCamClawStats],
'niriss': [NIRISSDarkQueryHistory, NIRISSDarkPixelStats, NIRISSDarkDarkCurrent,
NIRISSBiasQueryHistory, NIRISSBiasStats, NIRISSBadPixelQueryHistory,
NIRISSBadPixelStats, NIRISSReadnoiseQueryHistory, NIRISSReadnoiseStats,
Expand Down Expand Up @@ -570,7 +572,8 @@ class : obj
MIRIEDBBlockStats, MIRIEDBTimeIntervalStats, MIRIEDBEveryChangeStats,
NIRSpecEDBDailyStats, NIRSpecEDBBlockStats, NIRSpecEDBTimeIntervalStats,
NIRSpecEDBEveryChangeStats, FGSEDBDailyStats, FGSEDBBlockStats,
FGSEDBTimeIntervalStats, FGSEDBEveryChangeStats]}
FGSEDBTimeIntervalStats, FGSEDBEveryChangeStats],
'claw': [NIRCamClawQueryHistory, NIRCamClawStats], }

if __name__ == '__main__':
base.metadata.create_all(engine)
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
INSTRUMENT, string
START_TIME_MJD, float
END_TIME_MJD, float
RUN_MONITOR, bool
ENTRY_DATE, datetime
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
FILENAME, string
PROPOSAL, string
OBS, string
DETECTOR, string
FILTER, string
PUPIL, string
EXPSTART, string
EXPSTART_MJD, float
EFFEXPTM, float
RA, float
DEC, float
PA_V3, float
MEAN, float
MEDIAN, float
STDDEV, float
FRAC_MASKED, float
SKYFLAT_FILENAME, string
ENTRY_DATE, datetime
13 changes: 8 additions & 5 deletions jwql/instrument_monitors/common_monitors/readnoise_monitor.py
Original file line number Diff line number Diff line change
Expand Up @@ -413,7 +413,7 @@ def process(self, file_list):
processed_file = file.replace("uncal", "refpix")
if not os.path.isfile(processed_file):
files_to_calibrate.append(file)

# Run the files through the necessary pipeline steps
outputs = run_parallel_pipeline(files_to_calibrate, "uncal", "refpix", self.instrument)

Expand All @@ -422,7 +422,7 @@ def process(self, file_list):

# Get relevant header information for this file
self.get_metadata(filename)

if filename in outputs:
processed_file = outputs[filename]
else:
Expand Down Expand Up @@ -498,20 +498,23 @@ def process(self, file_list):
# Construct new entry for this file for the readnoise database table.
# Can't insert values with numpy.float32 datatypes into database
# so need to change the datatypes of these values.
readnoise_db_entry = {'uncal_filename': filename,
#
# Store files as file name only (file path will be built at retrieval based
# on runtime configuration)
readnoise_db_entry = {'uncal_filename': os.path.basename(filename),
'aperture': self.aperture,
'detector': self.detector,
'subarray': self.subarray,
'read_pattern': self.read_pattern,
'nints': self.nints,
'ngroups': self.ngroups,
'expstart': self.expstart,
'readnoise_filename': readnoise_outfile,
'readnoise_filename': os.path.basename(readnoise_outfile),
'full_image_mean': float(full_image_mean),
'full_image_stddev': float(full_image_stddev),
'full_image_n': full_image_n.astype(float),
'full_image_bin_centers': full_image_bin_centers.astype(float),
'readnoise_diff_image': readnoise_diff_png,
'readnoise_diff_image': os.path.basename(readnoise_diff_png),
'diff_image_mean': float(diff_image_mean),
'diff_image_stddev': float(diff_image_stddev),
'diff_image_n': diff_image_n.astype(float),
Expand Down
Loading
Loading