J/A+A/611/A53 Redshift reliability flags (VVDS data) (Jamal+, 2018)
Automated reliability assessment for spectroscopic redshift measurements.
Jamal S., Le Brun V., Le Fevre O., Vibert D., Schmitt A., Surace C.,
Copin Y., Garilli B., Moresco M., Pozzetti L.
<Astron. Astrophys. 611, A53 (2018)>
=2018A&A...611A..53J 2018A&A...611A..53J (SIMBAD/NED BibCode)
ADC_Keywords: Galaxies, spectra ; Redshifts
Keywords: methods: data analysis - methods: statistics -
techniques: spectroscopic - galaxies: distance and redshift - surveys
Abstract:
Future large-scale surveys, as the ESA Euclid mission, will produce a
large set of galaxy redshifts (≥10^6) that will require fully
automated data-processing pipelines to analyze the data, extract
crucial information and ensure that all requirements are met.
A fundamental element in these pipelines is to associate to each
galaxy redshift measurement a quality, or reliability, estimate.
In this work, we introduce a new approach to automate the
spectroscopic redshift reliability assessment based on machine
learning (ML) and characteristics of the redshift probability density
function.
We propose to rephrase the spectroscopic redshift estimation into a
Bayesian framework, in order to incorporate all sources of information
and uncertainties related to the redshift estimation process and
produce a redshift posterior probability density function (PDF).
To automate the assessment of a reliability flag, we exploit key
features in the redshift posterior PDF and machine learning
algorithms.
Description:
The VIMOS VLT Deep Survey (Le Fevre et al. 2013A&A...559A..14L 2013A&A...559A..14L) is a
combination of 3 i-band magnitude limited surveys: Wide
(17.5≤iAB≤22.5; 8.6deg2), Deep (17.5≤iAB≤24; 0.6deg2) and
Ultra-Deep (23≤iAB≤24.75; 512arcmin2), that produced a total of
35526 spectroscopic galaxy redshifts between 0 and 6.7 (22434 in Wide,
12051 in Deep and 1041 in UDeep).
We supplement spectra of the VIMOS VLT Deep Survey (VVDS) with
newly-defined redshift reliability flags obtained from clustering
(unsupervised classification in Machine Learning) a set of descriptors
from individual zPDFs.
In this paper, we exploit a set of 24519 spectra from the VVDS
database. After computing zPDFs for each individual spectrum, a set of
(8) descriptors of the zPDF are extracted to build a feature matrix X
(dimension = 24519 rows, 8 columns). Then, we use a clustering
(unsupervised algorithms in Machine Learning) algorithm to partition
the feature space into distinct clusters (5 clusters: C1,C2,C3,C4,C5),
each depicting a different level of confidence to associate with the
measured redshift zMAP (Maximum-A-Posteriori estimate that corresponds
to the maximum of the redshift PDF).
The clustering results (C1,C2,C3,C4,C5) reported in the table are
those used in the paper (Jamal et al, 2017) to present the new
methodology of automating the zspec reliability assessment. In
particular, we would like to point out that they were obtained from
first tests conducted on the VVDS spectroscopic data (end of 2016).
Therefore, the table does not depict immutable results (on-going
improvements). Future updates of the VVDS redshift reliability flags
can be expected.
File Summary:
--------------------------------------------------------------------------------
FileName Lrecl Records Explanations
--------------------------------------------------------------------------------
ReadMe 80 . This file
table.dat 96 24519 Catalog of VVDS data and new zReliability labels
--------------------------------------------------------------------------------
See also:
III/250 : The VIMOS VLT deep survey (VVDS-DEEP) (Le Fevre+ 2005)
Byte-by-byte Description of file: table.dat
--------------------------------------------------------------------------------
Bytes Format Units Label Explanations
--------------------------------------------------------------------------------
1- 5 A5 --- Survey Survey (1)
7- 19 A13 --- Field VVDS field (VVDS-FNNNN-NN or VVDS-CDFS)
21- 29 I09 --- ObsID Observation identification
31- 39 F9.5 deg RAdeg Right ascension (J2000)
41- 49 F9.5 deg DEdeg Declination (J2000)
51- 58 F8.5 mag Imag AB magnitude in I filter
60- 66 F7.5 --- zsp Spectroscopic redshift
68 I1 --- q_zsp [1/9] Initial VVDS z quality flags (2)
70- 71 A2 --- Relzsp [C1 C2 C3 C4 C5] New VVDS z reliability
flags (3)
73- 96 A24 --- IAUName Name based on J2000 position
(VVDS-JHHMMSS.ss+DDMMSS.s)
--------------------------------------------------------------------------------
Note (1): The VIMOS VLT Deep Survey is a combination of 3 surveys as follows:
DEEP = VIMOS VLT Deep Survey
UDEEP = VIMOS VLT Ultra-Deep Survey
WIDE = VIMOS VLT Wide Survey
(The table contains a subset of spectroscopic data from the 3 surveys)
Note (2): The quality of a redshift is determined through quality flags
(cf. Le Fevre et al., 2013A&A...559A..14L 2013A&A...559A..14L), as follows:
1 = Unreliable redshift
2 = Reliable redshift
9 = Reliable redshift, detection of a single emission line
3 = Very reliable redshift with strong spectral features
4 = Very reliable redshift with obvious spectral features
Note (3): Newly-defined redshift reliability clusters refers to distinct
partitions as follow:
C1 = Highly dispersed PDFs with multiple equiprobable modes,
P(zMAP)∼0.028±0.023
C2 = Less dispersed PDFs, with few modes and low probabilities
P(zMAP)∼0.087±0.033
C3 = Low dispersion (σ), intermediate probabilities
P(zMAP)∼0.166±0.035
C4 = Unimodal PDFs with low dispersion, higher probabilities
P(zMAP)∼0.290±0.059
C5 = Strong unimodal PDFs with extremely low dispersion, better probabilities
P(zMAP)∼0.618±0.204
--------------------------------------------------------------------------------
Acknowledgements:
Sara Jamal, sara.jamal(at)lam.fr
Vincent Le Brun, vincent.lebrun(at)lam.fr
References:
Le Fevre et al., 2013A&A...559A..14L 2013A&A...559A..14L, http://cesam.lam.fr/vvds
(End) Sara Jamal [LAM, France], Patricia Vannier [CDS] 15-Sep-2017