diff --git a/METIS/METIS_DET_IFU.yaml b/METIS/METIS_DET_IFU.yaml index cbc4930b..b236de19 100644 --- a/METIS/METIS_DET_IFU.yaml +++ b/METIS/METIS_DET_IFU.yaml @@ -30,6 +30,11 @@ properties: file_name: "QE_detector_H2RG_METIS.dat" linearity: file_name: "FPA_linearity_HxRG.dat" + border: + 1: [32, 0, 32, 64] + 2: [32, 64, 32, 0] + 3: [32, 64, 32, 0] + 4: [32, 0, 32, 64] effects: - name: detector_array_list @@ -52,6 +57,13 @@ effects: kwargs: fill_frac: "!OBS.auto_exposure.fill_frac" + - name: reference_pixel_mask + description: Mask for reference pixels + class: ReferencePixelBorder + include: True + kwargs: + border: "!DET.border" + - name: exposure_integration description: Summing up sky signal for all DITs and NDITs class: ExposureIntegration diff --git a/METIS/METIS_DET_IMG_LM.yaml b/METIS/METIS_DET_IMG_LM.yaml index 8ad6a95d..1fef13a7 100644 --- a/METIS/METIS_DET_IMG_LM.yaml +++ b/METIS/METIS_DET_IMG_LM.yaml @@ -25,6 +25,7 @@ properties: file_name: "QE_detector_H2RG_METIS.dat" linearity: file_name: "FPA_linearity_HxRG.dat" + border: [64, 64, 64, 64] effects: - name: detector_array @@ -70,6 +71,13 @@ effects: full_well: "!DET.full_well" mindit: "!DET.mindit" + - name: reference_pixel_mask + description: Mask for reference pixels + class: ReferencePixelBorder + include: True + kwargs: + border: "!DET.border" + - name: exposure_integration description: Summing up sky signal for all DITs and NDITs class: ExposureIntegration diff --git a/METIS/METIS_DET_IMG_N_GeoSnap.yaml b/METIS/METIS_DET_IMG_N_GeoSnap.yaml index 803dcdaf..86dfce0c 100644 --- a/METIS/METIS_DET_IMG_N_GeoSnap.yaml +++ b/METIS/METIS_DET_IMG_N_GeoSnap.yaml @@ -21,6 +21,7 @@ properties: temperature: -240 dit: "!OBS.dit" ndit: "!OBS.ndit" + border: [28, 28, 28, 28] layout: file_name: "FPA_metis_img_n_geosnap_layout.dat" qe_curve: @@ -74,6 +75,13 @@ effects: full_well: "!DET.full_well" mindit: "!DET.mindit" + - name: reference_pixel_mask + description: Mask for reference pixels + class: ReferencePixelBorder + include: True + kwargs: + border: "!DET.border" + - name: exposure_integration description: Summing up sky signal for all DITs and NDITs class: ExposureIntegration diff --git a/METIS/docs/example_notebooks/demos/demo_reference_pixel_mask.ipynb b/METIS/docs/example_notebooks/demos/demo_reference_pixel_mask.ipynb new file mode 100644 index 00000000..a33d3532 --- /dev/null +++ b/METIS/docs/example_notebooks/demos/demo_reference_pixel_mask.ipynb @@ -0,0 +1,301 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "1e941464-1404-46ab-a397-80a5ebee1cdf", + "metadata": {}, + "source": [ + "The METIS detectors have a number of masked rows and pixels around their borders. These pixels do not see any signal but do have dark current, readout noise and other detector effects. This notebook shows what the effect `ReferencePixelBorder` does and how it is set up for METIS. It also explains how the dimensions of the mask can be changed and of course how the mask can be switched off entirely." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "f4e714c2-78d5-4b59-ac37-b543361931f9", + "metadata": {}, + "outputs": [], + "source": [ + "from matplotlib import pyplot as plt" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "1a60f1fb-6829-432b-a175-f636f0a65306", + "metadata": {}, + "outputs": [], + "source": [ + "import scopesim as sim" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "bd5bfa01-10a2-4b89-9ee6-0cf3803bb7a6", + "metadata": {}, + "outputs": [], + "source": [ + "# Edit this path if you have a custom install directorym otherwise comment it out\n", + "sim.link_irdb(\"../../../../\")\n", + "\n", + "# If you haven't got the instrument packages yet, uncomment the following line\n", + "# sim.download_packages([\"METIS\", \"ELT\", \"Armazones\"])" + ] + }, + { + "cell_type": "markdown", + "id": "dd1a598c-be59-4be9-99a4-f931de79593e", + "metadata": {}, + "source": [ + "## Imager detectors" + ] + }, + { + "cell_type": "markdown", + "id": "f44877ee-ea96-4272-8675-414b4eefa44f", + "metadata": {}, + "source": [ + "The Imager detectors (H2RG for IMG-LM, Geosnap for IMG-N) have equal width masks all around, with 64 pixels for the H2RG and 28 pixels for the Geosnap. We look at the H2RG here and simulate a WCU flat field for simplicity." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "7e1959ef-5d74-4adc-a1fd-f4abb8c77aff", + "metadata": {}, + "outputs": [], + "source": [ + "cmd = sim.UserCommands(use_instrument=\"METIS\", set_modes=[\"wcu_img_lm\"])\n", + "metis = sim.OpticalTrain(cmd)\n", + "metis.observe()\n", + "readout = metis.readout(ndit=1, dit=1.)[0]" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "db135971-904e-4efb-aa8c-dab38ecc1553", + "metadata": {}, + "outputs": [], + "source": [ + "_, (ax1, ax2) = plt.subplots(1, 2, figsize=(10, 4))\n", + "ax1.imshow(readout[1].data)\n", + "ax1.set_title(\"IMG-LM flat field\")\n", + "ax2.plot(readout[1].data[250,])\n", + "ax2.set_title(\"Row 250\");" + ] + }, + { + "cell_type": "markdown", + "id": "a9c3d3b2-6763-4603-ac7c-9819e92462ef", + "metadata": {}, + "source": [ + "The dimensions of the mask are recorded the header of the image extension of the readout (the order is **b**ottom, **l**eft, **t**op, **r**ight):" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "487a48ed-07b0-4582-b36e-e125d11d02f5", + "metadata": {}, + "outputs": [], + "source": [ + "print(readout[1].header[\"HIERARCH ESO DET1 CHIP REFPIX\"]) # keyword value itself\n", + "print(readout[1].header.comments[\"HIERARCH ESO DET1 CHIP REFPIX\"]) # keyword comment" + ] + }, + { + "cell_type": "markdown", + "id": "f6357f23-b282-4115-83c0-661715e64097", + "metadata": {}, + "source": [ + "To demonstrate that the masked pixels indeed get dark current, we increase the latter to an appreciable value." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "66c39cb3-c635-47c9-b45d-b6432cade4c2", + "metadata": {}, + "outputs": [], + "source": [ + "metis[\"dark_current\"].meta[\"value\"] = 10000.\n", + "readout = metis.readout(ndit=1, dit=1.)[0]" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "096020fc-1a2b-48b7-8174-28f62c784190", + "metadata": {}, + "outputs": [], + "source": [ + "_, (ax1, ax2) = plt.subplots(1, 2, figsize=(10, 4))\n", + "ax1.imshow(readout[1].data)\n", + "ax1.set_title(\"IMG-LM flat field\")\n", + "ax2.plot(readout[1].data[250,])\n", + "ax2.set_ylim(0, 26000)\n", + "ax2.set_ylabel(\"ADU\")\n", + "ax2.set_title(\"Row 250\");" + ] + }, + { + "cell_type": "markdown", + "id": "1943d886-f425-4ec5-a44b-5a28a684a638", + "metadata": {}, + "source": [ + "Note that with a gain of 4 e/ADU, the reference level of 2500 ADU corresponds to 10000 electrons, as expected." + ] + }, + { + "cell_type": "markdown", + "id": "74d14fff-1335-49d8-a052-40c95ca984c1", + "metadata": {}, + "source": [ + "## LMS detector array\n", + "\n", + "The LMS detector array consists of 4 H2RG detectors. In the y-direction all detectors have 32 reference rows at the bottom and top. In the x-direction (the dispersion direction) only the outsides are masked with 64 columns, while there are no reference pixels on the inside. (The simulation takes several minutes.)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "b6048c19-cd3e-40a1-ac50-ea9a4ec31057", + "metadata": {}, + "outputs": [], + "source": [ + "cmd = sim.UserCommands(use_instrument=\"METIS\", set_modes=[\"wcu_lms\"])\n", + "metis = sim.OpticalTrain(cmd)\n", + "metis['psf'].include = False # saves some simulation time\n", + "metis.observe()\n", + "readout = metis.readout(ndit=1, dit=300)[0]" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "155d8739-8d6c-4f71-af4b-67a6cbc04242", + "metadata": {}, + "outputs": [], + "source": [ + "_, [[ax2, ax1], [ax3, ax4]] = plt.subplots(2, 2, layout=\"tight\", sharex=True, sharey=True)\n", + "ax1.imshow(readout[1].data)\n", + "ax2.imshow(readout[2].data)\n", + "ax3.imshow(readout[3].data)\n", + "ax4.imshow(readout[4].data)" + ] + }, + { + "cell_type": "markdown", + "id": "2f459223-3ddf-4ecf-9606-a8b5aaaf337c", + "metadata": {}, + "source": [ + "## Modifying the behaviour of `ReferencePixelBorder`\n", + "When setting up the instrument, the detector mask can be defined with the parameter `!DET.border`:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "c002ffa9-5769-465f-8b05-064b2af4b683", + "metadata": {}, + "outputs": [], + "source": [ + "cmd = sim.UserCommands(use_instrument=\"METIS\", set_modes=[\"wcu_img_lm\"])\n", + "cmd[\"!DET.border\"] = [0, 40, 80, 120] # bottom left top right\n", + "metis = sim.OpticalTrain(cmd)\n", + "metis.observe()\n", + "readout = metis.readout(dit=1., ndit=1)[0]" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "ad2a1ae9-1d6f-4cb8-94bd-776e79085836", + "metadata": {}, + "outputs": [], + "source": [ + "plt.imshow(readout[1].data, origin='lower')" + ] + }, + { + "cell_type": "markdown", + "id": "763cbbb5-bcd4-404f-94ff-1aaf6889b149", + "metadata": {}, + "source": [ + "When the `OpticalTrain` already exists and has `observe`d a scene, the mask can be modified by changing the effect parameter:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "799e53a4-3f64-43bb-8d4d-5fa180f8d729", + "metadata": {}, + "outputs": [], + "source": [ + "metis['reference_pixel_mask'].meta[\"border\"] = [300, 300, 300, 300]\n", + "readout = metis.readout(dit=1, ndit=1)[0]" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "5a67c02e-d7f4-4beb-b479-973de6624f22", + "metadata": {}, + "outputs": [], + "source": [ + "plt.imshow(readout[1].data, origin='lower')" + ] + }, + { + "cell_type": "markdown", + "id": "dd892e89-05bf-4c53-8f74-d5dd72d5cb79", + "metadata": {}, + "source": [ + "As always, the mask can be switched off entirely by setting" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "340dcb82-abda-44ca-b4da-4703c21a16a4", + "metadata": {}, + "outputs": [], + "source": [ + "metis['reference_pixel_mask'].include = False" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "49d8cd62-912c-4756-b8de-99bcd31f378f", + "metadata": {}, + "outputs": [], + "source": [ + "readout = metis.readout(dit=1., ndit=1)[0]\n", + "plt.imshow(readout[1].data, origin='lower')" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.12.9" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/METIS/headers/FITS_det_ifu_keywords.yaml b/METIS/headers/FITS_det_ifu_keywords.yaml index 975a5e5e..9d599af0 100644 --- a/METIS/headers/FITS_det_ifu_keywords.yaml +++ b/METIS/headers/FITS_det_ifu_keywords.yaml @@ -3,7 +3,7 @@ HIERARCH: ESO: DET3: - DIT: "!OBS.dit" + DIT: ["!OBS.dit", "[s]"] NDIT: "!OBS.ndit" CUBE: MODE: false @@ -16,11 +16,12 @@ DET3: CHIP: ID: "!DET.detector" - MINDIT: "!DET.mindit" - FULLWELL: "!DET.full_well" - RON: "!DET.readout_noise" - DARK: "!DET.dark_current" - GAIN: "!DET.gain.1" + MINDIT: ["!DET.mindit", "[s]"] + FULLWELL: ["!DET.full_well", "[e]"] + RON: ["!DET.readout_noise", "[e] per CDS readout"] + DARK: ["!DET.dark_current", "[e/s]"] + GAIN: ["!DET.gain.1", "[e/ADU]"] + REFPIX: ["!DET.border.1", "b l t r"] - ext_number: [2] keywords: @@ -30,11 +31,12 @@ DET3: CHIP: ID: "!DET.detector" - MINDIT: "!DET.mindit" - FULLWELL: "!DET.full_well" - RON: "!DET.readout_noise" - DARK: "!DET.dark_current" - GAIN: "!DET.gain.2" + MINDIT: ["!DET.mindit", "[s]"] + FULLWELL: ["!DET.full_well", "[e]"] + RON: ["!DET.readout_noise", "[e] per CDS readout"] + DARK: ["!DET.dark_current", "[e/s]"] + GAIN: ["!DET.gain.2", "[e/ADU]"] + REFPIX: ["!DET.border.2", "b l t r"] - ext_number: [3] keywords: @@ -44,11 +46,12 @@ DET3: CHIP: ID: "!DET.detector" - MINDIT: "!DET.mindit" - FULLWELL: "!DET.full_well" - RON: "!DET.readout_noise" - DARK: "!DET.dark_current" - GAIN: "!DET.gain.3" + MINDIT: ["!DET.mindit", "[s]"] + FULLWELL: ["!DET.full_well", "[e]"] + RON: ["!DET.readout_noise", "[e] per CDS readout"] + DARK: ["!DET.dark_current", "[e/s]"] + GAIN: ["!DET.gain.3", "[e/ADU]"] + REFPIX: ["!DET.border.3", "b l t r"] - ext_number: [4] keywords: @@ -58,8 +61,9 @@ DET3: CHIP: ID: "!DET.detector" - MINDIT: "!DET.mindit" - FULLWELL: "!DET.full_well" - RON: "!DET.readout_noise" - DARK: "!DET.dark_current" - GAIN: "!DET.gain.4" + MINDIT: ["!DET.mindit", "[s]"] + FULLWELL: ["!DET.full_well", "[e]"] + RON: ["!DET.readout_noise", "[e] per CDS readout"] + DARK: ["!DET.dark_current", "[e/s]"] + GAIN: ["!DET.gain.4", "[e/ADU]"] + REFPIX: ["!DET.border.4", "b l t r"] diff --git a/METIS/headers/FITS_det_lm_keywords.yaml b/METIS/headers/FITS_det_lm_keywords.yaml index 26bb2247..b6845d34 100644 --- a/METIS/headers/FITS_det_lm_keywords.yaml +++ b/METIS/headers/FITS_det_lm_keywords.yaml @@ -3,7 +3,7 @@ HIERARCH: ESO: DET1: - DIT: "!OBS.dit" + DIT: ["!OBS.dit", "[s]"] NDIT: "!OBS.ndit" MODE: "!DET.mode" CUBE: @@ -17,9 +17,10 @@ DET1: CHIP: ID: "!DET.detector" - MODE: "!DET.mode" - MINDIT: "!DET.mindit" - FULLWELL: "!DET.full_well" - RON: "!DET.readout_noise" - DARK: "!DET.dark_current" - GAIN: "!DET.gain" + MODE: ["!DET.mode", "Readout mode"] + MINDIT: ["!DET.mindit", "[s]"] + FULLWELL: ["!DET.full_well", "[e]"] + RON: ["!DET.readout_noise", "[e] per CDS readout"] + DARK: ["!DET.dark_current", "[e/s]"] + GAIN: ["!DET.gain", "[e/ADU]"] + REFPIX: ["#reference_pixel_mask.border", "b l t r"] diff --git a/METIS/headers/FITS_det_n_keywords.yaml b/METIS/headers/FITS_det_n_keywords.yaml index 6a886612..ef4b8b09 100644 --- a/METIS/headers/FITS_det_n_keywords.yaml +++ b/METIS/headers/FITS_det_n_keywords.yaml @@ -3,7 +3,7 @@ HIERARCH: ESO: DET2: - DIT: "!OBS.dit" + DIT: ["!OBS.dit", "[s]"] NDIT: "!OBS.ndit" MODE: "!DET.mode" CUBE: @@ -22,9 +22,10 @@ DET2: CHIP: ID: "!DET.detector" - MODE: "!DET.mode" - MINDIT: "!DET.mindit" - FULLWELL: "!DET.full_well" - RON: "!DET.readout_noise" - DARK: "!DET.dark_current" - GAIN: "!DET.gain" + MODE: ["!DET.mode", "Readout mode"] + MINDIT: ["!DET.mindit", "[s]"] + FULLWELL: ["!DET.full_well", "[e]"] + RON: ["!DET.readout_noise", "[e] per CDS readout"] + DARK: ["!DET.dark_current", "[e/s]"] + GAIN: ["!DET.gain", "[e/ADU]"] + REFPIX: ["#reference_pixel_mask.border", "b l t r"] diff --git a/METIS/tests/test_readout_exptime.py b/METIS/tests/test_readout_exptime.py index a9d461e1..bf905980 100644 --- a/METIS/tests/test_readout_exptime.py +++ b/METIS/tests/test_readout_exptime.py @@ -36,6 +36,7 @@ def test_readout_exptime(): # The first readout might ignore the exptime. metis_l = sim.OpticalTrain(cmd_l) metis_l["exposure_output"].set_mode("sum") + metis_l["reference_pixel_mask"].include = False metis_l.observe(star, update=True) result_first = metis_l.readout(exptime=0.1)[0] # We need to copy the first readeout, because @@ -45,6 +46,7 @@ def test_readout_exptime(): # The second readout might not properly have the detector reset. metis_l = sim.OpticalTrain(cmd_l) metis_l["exposure_output"].set_mode("sum") + metis_l["reference_pixel_mask"].include = False metis_l.observe(star, update=True) result_temp = metis_l.readout(exptime=.1)[0] result_temp_copy = copy.deepcopy(result_temp)