J/A+A/616/A97 SDSS QSO DR7 and DR9 (D'Isanto+, 2018)
Return of the features. Efficient feature selection and interpretation for
photometric redshifts.
D'Isanto A., Cavuoti S., Gieseke F., Polsterer K.L.
<Astron. Astrophys. 616, A97 (2018)>
=2018A&A...616A..97D 2018A&A...616A..97D (SIMBAD/NED BibCode)
ADC_Keywords: QSOs ; Surveys
Keywords: methods: data analysis - methods: statistical -
galaxies: distances and redshifts - quasars: general
Abstract:
The explosion of data in recent years has generated an increasing need
for new analysis techniques in order to extract knowledge from massive
data-sets. Machine learning has proved particularly useful to perform
this task. Fully automatized methods (e.g. deep neural networks) have
recently gathered great popularity, even though those methods often
lack physical interpretability. In contrast, feature based approaches
can provide both well-performing models and understandable causalities
with respect to the correlations found between features and physical
processes.
Efficient feature selection is an essential tool to boost the
performance of machine learning models. In this work, we propose a
forward selection method in order to compute, evaluate, and
characterize better performing features for regression and
classification problems. Given the importance of photometric redshift
estimation, we adopt it as our case study.
We synthetically created 4520 features by combining magnitudes,
errors, radii, and ellipticities of quasars, taken from the Sloan
Digital Sky Survey (SDSS). We apply a forward selection process, a
recursive method in which a huge number of feature sets is tested
through a k-Nearest-Neighbours algorithm, leading to a tree of feature
sets. The branches of the feature tree are then used to perform
experiments with the random forest, in order to validate the best set
with an alternative model.
We demonstrate that the sets of features determined with our approach
improve the performances of the regression models significantly when
compared to the performance of the classic features from the
literature. The found features are unexpected and surprising, being
very different from the classic features. Therefore, a method to
interpret some of the found features in a physical context is
presented. The feature selection methodology described here is very
general and can be used to improve the performance of machine learning
models for any regression or classification task.
Description:
The SDSS object IDs and coordinates of the extracted quasars for the
three catalogues used for the experiments. The catalogue dr7a.dat
contains quasars from SDSS DR7 using photometric flags in the query.
The catalogue dr7b.dat is retrieved using the same query but without
photometric flags. The catalogue dr7_9.dat contains quasars from DR7
and DR9, with a wider redshift distribution.
File Summary:
--------------------------------------------------------------------------------
FileName Lrecl Records Explanations
--------------------------------------------------------------------------------
ReadMe 80 . This file
dr7a.dat 61 83982 SDSS object IDs and coordinates of the quasars
for experiment DR7a
dr7b.dat 45 97041 SDSS object IDs and coordinates of the quasars
for experiment DR7b
dr7_9.dat 47 152137 SDSS object IDs and coordinates of the quasars
for experiment DR7+9
--------------------------------------------------------------------------------
See also:
II/294 : The SDSS Photometric Catalog, Release 7 (Adelman-McCarthy+, 2009)
V/139 : The SDSS Photometric Catalog, Release 9 (Adelman-McCarthy+, 2012)
VII/260 : The SDSS-DR7 quasar catalog, DR7Q (Schneider+, 2010)
VII/269 : The SDSS-DR9 quasar catalog, DR9Q (Paris+, 2012)
http://www.sdss.org/dr7 : SDSS DR7 home page
http://www.sdss3.org/dr9 : SDSS DR9 home page
Byte-by-byte Description of file: dr7a.dat
--------------------------------------------------------------------------------
Bytes Format Units Label Explanations
--------------------------------------------------------------------------------
1- 18 A18 --- objID SDSS DR7 objID
20- 40 F21.17 deg RAdeg Right ascension (J2000.0)
42- 61 F20.16 deg DEdeg Declination (J2000.0)
--------------------------------------------------------------------------------
Byte-by-byte Description of file: dr7b.dat dr7_9.dat
--------------------------------------------------------------------------------
Bytes Format Units Label Explanations
--------------------------------------------------------------------------------
1- 18 A18 --- objID SDSS DR7 objID
20- 32 F13.9 deg RAdeg Right ascension (J2000.0)
34- 47 F14.10 deg DEdeg Declination (J2000.0)
--------------------------------------------------------------------------------
Acknowledgements:
Antonio D'Isanto, antonio.disanto(at)h-its.org
(End) Antonio D'Isanto [HITS, Germany], Patricia Vannier [CDS] 11-May-2018