\documentclass[11pt,twoside]{article}  % Leave intact
\usepackage{adassconf}

\begin{document}

\paperID{P1-5}
%%%% ID=P1-5

\title{The Chandra Multiwavelength Project (ChaMP): Optical Data Processing and Catalog Generation}
\titlemark{The ChaMP: Optical Data Processing and Catalog Generation}

\author{Robert A. Cameron, Wayne A. Barkhouse, Paul J. Green, Amy E. Mossman, 
John D. Silverman, Belinda J. Wilkes and the ChaMP Collaboration}
\affil{Harvard-Smithsonian Center for Astrophysics, 60 Garden Street, Cambridge, MA 02138}

\contact{Robert Cameron}
\email{rcameron@cfa.harvard.edu}

\paindex{Cameron, R. A.}
\aindex{Barkhouse, W. A.}
\aindex{Green, P. J.}
\aindex{Mossman, A. E.}
\aindex{Silverman, J. D.}
\aindex{Wilkes, B. J.}

\authormark{Cameron, Barkhouse, Green, Mossman, Silverman \& Wilkes}

\keywords{Chandra, ChaMP, catalogs: multiwavelength} 

\begin{abstract}
One of the principal objectives of the Chandra Multiwavelength Project (ChaMP)
is the optical identification and cataloging of serendipitously detected
background X-ray sources in {\slshape Chandra} archival data. The ChaMP uses
a program of multi-filter optical imaging of observed {\slshape Chandra} 
fields to detect optical counterparts to X-ray sources. We describe the
methods used for reduction, analysis and cataloging of optical sources
in the ChaMP fields. Automated pipeline processing of the optical data 
includes source extraction, photometric calibration and optical to X-ray 
source matching. Visual inspection tools have been developed for quality 
control of the resultant source lists and for identification of interesting 
objects for follow-up spectroscopic observations. Methods and tools for 
management, presentation and access of the ChaMP catalogs are also described.
\end{abstract}

\section{Introduction}
The {\slshape Chandra} Multiwavelength Project (ChaMP) is a serendipitous
X-ray source survey based on archival {\slshape Chandra} AO1 and AO2 data.
The ACIS data cover approximately 14 square degrees of sky, and are expected 
to provide $\sim\!\!8000$ serendipitous X-ray sources, (Kim et al. 2004a, 
2004b). The sensitive, wide-area ChaMP survey provides a X-ray source sample 
significantly more sensitive than previous ROSAT and ASCA sky surveys, and a 
survey with  greater sky coverage than the {\slshape Chandra} Deep Field 
surveys is the only way to compile a significant sample of high-redshift QSOs.

Chandra's sub-arcsecond angular resolution and $~1''$ celestial location capability 
(Aldcroft et al. 2000) is ideal for a corresponding optical survey, to allow 
unambiguous optical identification of the majority of the X-ray sources. A key component 
of the ChaMP is deep, wide-field optical imaging of the fields. We use the SDSS $g', r', i'$ 
filters, to provide good object classification and photometric redshift determination. 
We use the NOAO 4m telescopes (KPNO and CTIO) with Mosaic CCD detectors to optically image 
the deeper ChaMP fields and SAO's FLWO 1.2m telescope with the 4Shooter camera to image 
northern shallow fields and to measure the brighter objects in the deeper fields. Each 
camera field of view is well matched to the ACIS-I and ACIS-S fields of view.

We scale our optical exposure times to the {\slshape Chandra} X-ray exposure times, to 
provide a uniform sensitivity to X-ray/optical flux ratios. The optical magnitude limit 
for each observation is scaled to the expected X-ray flux limit for each field, to 
include $90\%$ of the ROSAT sky survey AGN at the X-ray flux limit. Individual CCD 
exposures are adjusted according to moon phase to limit background contribution to the 
exposure. Multiple exposures are stacked with median filtering to produce single night
images in each filter for analysis. Total exposure times on each ChaMP field are tallied 
in database tables, using only photometric or near-photometric data, to track imaging 
completeness across multiple observing runs. We expect to match $\sim\!4000$ sources to 
$r' \simeq 25$, matching 90\% of X-ray sources with log\,$f_x>-14.8$. Together with 
optical imaging, optical spectroscopy observations are being carried out to gather an 
AGN and QSO sample, using the FLWO 1.5m FAST, WIYN/HYDRA, CTIO4m/HYDRA, Magellan/LDSS-2 
and MMT/BCS spectrographs. Green et al. (2004) present optical imaging and spectroscopy 
details and results for six fields from the ChaMP. For these six fields, using single-night 
stacked data, 55\% to 78\% of the X-ray sources in each field have optical matches.

\section{Data Management}
Two key issues drive the design of the data management system implemented
for the ChaMP: (i) the large number of {\slshape Chandra}
fields in the survey, and the associated large amount of optical data,
and large numbers of detected optical sources and measured spectra,
(ii) the primary project requirement to provide full, simple access
to the X-ray and optical data and results.

To meet these requirements, standard data products and database tools
are necessary, to provide pipelined data processing, automated database
construction and retrieval and statistical analysis.
The key features of data management in the ChaMP are:
\begin{itemize}
\item Pipeline processing for both X-ray and optical imaging data.
The X-ray data are mainly with CIAO tools. The optical data are
processed with standard IRAF tools, SExtractor, IDL and Perl/PDL scripts.
\item Pipeline processing is driven by Perl and shell scripts and controlled
by input parameter files.
\item Intermediate data files generated by the pipeline processing
are transported in RDB format between tasks.
\item Final X-ray and optical pipeline products go through Verification
and Validation (V\&V) inspection by users before ingest into the final
database archive tables.
\item Product tables are archived in SYBASE. Separate database tables
are defined for X-ray observation data, optical observation data,
basic X-ray source data and optical source data,
and optical to X-ray source cross-identifications.
\item WWW access to the database tables is provided through scripts
that generate HTML. Tools are being developed to provide database
interaction for dynamic data selection through the WWW.
\end{itemize}

\section{Data Reduction and Analysis}
To efficiently process the large number of {\slshape Chandra} observations in ChaMP, and
the corresponding large number of optical imaging datasets, we have implemented
pipeline processing techniques. Pipeline processing automates the data reduction
and analysis to the maximum possible extent and operates with standard data products
for compatability with database management techniques.

Similar but not identical data reduction operations are applied to the Mosiac and
4Shooter imaging data. Standard IRAF tools from the {\sf mscred} (v4.8) {\sf nproto} and {\sf crutil}
packages are used for the data reduction. Basic reduction operations are applied to 
(i) correct crosstalk and remove bias, (ii) flat field with dome flats and super sky
flats, (iii) remove pupil images (NOAO 4m), (iv) refine WCS J2000 astrometry, (v) 
filter cosmic rays, (vi) project multiple CCDs and stack multiple exposures into
single images.

The standardized output products from data the reduction (a merged, stacked CCD image and
bad pixel list) are identical for both program field and standard star field observations,
and for Mosaic and 4Shooter observations. A common source extraction and photometric calibration 
pipeline is used for all subsequent data analysis. Source extraction is based on SExtractor 
(Bertin \& Arnouts 1996). Because we want a good measure of stellarity for each source, we process 
the images through SExtractor twice, to first estimate and then use the correct field FWHM. 
The pipeline is controlled by a modular Perl script. Program fields and standard star fields are 
identified in an input parameter file to select the appropriate processing stream within the pipeline.
The processing tasks are:
\begin{itemize}
\item Generate bad pixel mask image from bad/saturated pixel list
\item Initial SExtractor source extraction with default FWHM
\item Determine FWHM from objects with high stellarity
\item Second SExtractor source extraction with correct FWHM
\item Rejection of objects with invalid flux or FWHM measures
\item Estimate of background rms for each object in the program fields
\item Determine global background rms statistics for each program field
\item Position match $g', r', i'$ object detections ($1''$ tolerance)
\item Identify detected standard stars from a master standard list ($1''$ tolerance)
\item Perform interative photometric calibration
\item Apply photometric calibration to program field objects
\item Position match optical and X-ray source lists for the program fields
\item Estimate program field magnitude limits
\end{itemize}

An iterative photometric calibration of the standard stars is used.
Standard stars calibrated for the SDSS (Smith et al. 2002) plus standard
stars from Landolt (1992) transformed to the SDSS system are used.
Four coefficents (color coefficent, zero point, $k_0$ and $k_1$ extinction
coefficents) are solved for, alternately freezing and solving for two
pairs of coefficients. In addition, a $\sigma$-clip (typically 2 or 3 $\sigma$)
is applied to remove outlier stars from the solution.
Typical rms errors are $\sim\!0.03$ mag from $>30$ stars.

Within the master optical pipeline, X-ray source products are imported from
the X-ray data processing pipeline, and the master optical pipeline performs
position matching of optical and X-ray source lists to provide the best
(possibly multiple) candidate optical matches for each X-ray source.

All intermediate data products generated within the pipeline are transported and 
preserved in RDB files for simple user inspection and analysis. Diagnostic plots 
are also generated at intermediate processing stages to monitor pipeline performance. 
The master pipeline logfile tracks key statistics including source counts, FWHM, 
$g', r', i'$ matching statistics, photometric calibration rms, and optical to X-ray 
source matching statistics. Finally, software scripts are used for automated ingest 
of the pipeline products into the master optical SYBASE tables.
\subsection{Visual Inspection}
Visual inspection of each ChaMP field is performed as the final step of discovering optical 
counterparts to each X-ray source. An IDL tool, {\tt vi}, provides an interactive environment 
to inspect and assess the quality of the optical and X-ray data for each source, and to verify
optical to X-ray source matches. For each X-ray source, {\tt vi} displays optical (typically $r'$ 
filter) and smoothed X-ray images of the source field, together with summary data for the X-ray 
source, and best-match optical source. Overlays on the optical and X-ray images indicate
detected source positions, sizes and processing flags, and candidate matches.
\section{WWW Access}
All ChaMP data are being made available through the ChaMP web 
\htmladdnormallinkfoot{site}{http://hea-www.harvard.edu/CHAMP}, including
X-ray and optical field lists and field images, X-ray and optical source lists with
associated source images, and optical spectra. Also available are refereed papers 
describing the X-ray and optical datasets and analysis. Interactive database query 
tools are being developed to assist data selection.
\acknowledgments
This work was supported in part by NASA contract NAS8-39073 and by Chandra
grants AR1-2003X and AR3-4018X. Optical data for the ChaMP are obtained in
part through the National Optical Astronomy Observatory (NOAO), operated by
the Association of Universities for Research in Astronomy, Inc. (AURA) under
cooperative agreement with the National Science Foundation.
\begin{references}
\reference Aldcroft, T. L. et al. 2000, \spie 71, 276
\reference Bertin, E. \& Arnouts, S. 1996, \aap, 117, 393
\reference Green, P.G. et al. 2004, \apjs, 150, in press (astro-ph/0308506)
\reference Kim, D.-W. et al. 2004a, \apjs, 150, in press (astro-ph/0308492)
\reference Kim, D.-W. et al. 2004b, \apj, 600, in press (astro-ph/0308493)
\reference Landolt, A.U. 1992, \aj, 104 340
\reference Smith, J.A. et al. 2002, \aj, 123, 2121
\end{references}

\end{document}