Detections Database and API

This notebook demonstrates how to access the HDF5 container for the HETDEX line detections database. This database is a catalog of line emission detections and their associated 1D, aperture summed, psf-weighted spectra. There are three tables contained within this HDF5 file:

  1. Detections - this is the main database of line detection sources. It provides the position and central wavelength of each detection and corresponding line fluxes. A source detection corresponds to an emission line so it is possible to have multiple line detections at different wavelengths for a single source. There can also be multiple observations of the same line if it has been observed in multiple shots or if it is associated with a large source.

  2. Fibers - for each source detection, this table lists information about each fiber used to extract the flux measurment and weighted spectrum. This allows a user to return to the processed data products (ie. the shot HDF5 files) to investigate the source further.

  3. Spectra - for each source, this table contains arrays of wavelength and 1D flux-weighted aperture summed spectral data and corresponding errors. Non-calibrated spectra is also provided in counts

%matplotlib inline

import numpy as np
import tables as tb
import matplotlib.pyplot as plt

from astropy.table import Table, Column, join
from astropy.coordinates import SkyCoord
import astropy.units as u

from hetdex_api.config import HDRconfig
from hetdex_api.detections import Detections
from hetdex_api.elixer_widget_cls import ElixerWidget

New for HDR4: We have the Detection Index file to index HDR3/HDR4 continuum and emisison line databases. This file and API provides limited detection info and guides the user on which Detections version to call to grab detection info such as basic line fit parameters, fiber information and spectra. You can query by detectid or sky coordinates for example.

To query the full Detection Index by coordinates

DI = Detections( catalog_type='index', searchable=True)

# putting in a random coordinate where I know there is a detection
coord = SkyCoord( ra=10.792455, dec=1.055791, unit='deg')

matches = DI.query_by_coord( coord, radius=10.0*u.arcsec)

matches

# Note not all detections will have an Elixer Report as we downselect from the +10 Million row raw detection file to a curated list
ElixerWidget(detectlist=matches['detectid'])

To get specific info for a single detection

detectid = 3001001637

DI = Detections( catalog_type='index')

det_info = DI.get_detection_info(detectid)

# this contains some basic detection info, coordinate info, healpix integer, survey and catalog type 
det_info

print("detectid={} is located on shot={} and is survey={} and of catalog type={}".format(
    detectid, det_info['shotid'][0], det_info['survey'][0].decode(), det_info['det_type'][0].decode()))

# Once you know what survey and catalog type you can then access the Detections API for the specific survey/det_type

survey = det_info['survey'][0].decode()
det_type = det_info['det_type'][0].decode()

if det_type == 'cont':
    catalog_type='continuum'
elif det_type == 'line':
    catalog_type='lines'

Pull up relevent Detections Database/API

D = Detections(survey, catalog_type = catalog_type)

#spectra can be accessed using get_spectrum()
spec = D.get_spectrum(detectid)

spec

Get updated Detection Info

det_info = D.get_detection_info(detectid)

det_info

Get Fiber Info:

Get Fiber Info

fib_info = D.get_fiber_info(detectid)

fib_info

### Get Survey Info

survey_info = D.get_survey_info(detectid)
survey_info

To load everything in the detection index

DI = Detections( catalog_type='index', loadtable=True)

DI.hdfile.root.DetectIndex

#detections are in DI.detectid
#coords are in DI.coords

Close the Detection Index and Detections H5 file when done

DI.close()

D.close()