Difference between revisions of "Bandmerging"

From CoolWiki
Jump to navigationJump to search
(Created page with "The goal of bandmerging is to assemble a catalog where there is one line (or entry, or virtual index card, or however you choose to think of it) per unique object. This line c...")
 
m
 
(One intermediate revision by the same user not shown)
Line 1: Line 1:
 +
=Goal and Definition=
 +
 
The goal of bandmerging is to assemble a catalog where there is one line (or entry, or virtual index card, or however you choose to think of it) per unique object. This line contains all the measurements at all the available wavelengths and any other relevant information all collected on the same line, tied to the same unique object. We need such a catalog to make color-magnitude and color-color diagrams, SEDs, etc.
 
The goal of bandmerging is to assemble a catalog where there is one line (or entry, or virtual index card, or however you choose to think of it) per unique object. This line contains all the measurements at all the available wavelengths and any other relevant information all collected on the same line, tied to the same unique object. We need such a catalog to make color-magnitude and color-color diagrams, SEDs, etc.
  
 
The definition of bandmerging is to take the set of detections at one band and merge the unique sources to the set of unique detections at another band. You can do this by hand, one by one, looking for the same object in multiple images or catalogs, or you can have the computer take a first pass at this (though having said that, even with the computer doing it, often there are a handful of sources that take manual fiddling to get right).
 
The definition of bandmerging is to take the set of detections at one band and merge the unique sources to the set of unique detections at another band. You can do this by hand, one by one, looking for the same object in multiple images or catalogs, or you can have the computer take a first pass at this (though having said that, even with the computer doing it, often there are a handful of sources that take manual fiddling to get right).
  
You can bandmerge catalogs you create, or catalogs you pull out of the literature.
+
=The Process=
 +
 
 +
You can bandmerge catalogs you create, or catalogs you pull out of the literature, or both. (We need to do both.)
  
 
Catalogs you pull out of the literature may already have done some source matching for you. Some of these existing catalogs have “every source we detected in this region”; some have “just the things we think are interesting.” Note that not all catalogs will “see” the same objects because not all surveys go to the same integration depth, and the objects themselves change with wavelength (e.g., stars can be much fainter in IR than optical - think about the SED shape).
 
Catalogs you pull out of the literature may already have done some source matching for you. Some of these existing catalogs have “every source we detected in this region”; some have “just the things we think are interesting.” Note that not all catalogs will “see” the same objects because not all surveys go to the same integration depth, and the objects themselves change with wavelength (e.g., stars can be much fainter in IR than optical - think about the SED shape).
Line 17: Line 21:
 
*Picking a good radius over which to do the match. In general, for modern catalogs, I look for things within a 1 arcsec radius.
 
*Picking a good radius over which to do the match. In general, for modern catalogs, I look for things within a 1 arcsec radius.
 
*Calculate δRA, δDec with spherical trigonometry and make sure you are really measuring angle subtended on the sky.
 
*Calculate δRA, δDec with spherical trigonometry and make sure you are really measuring angle subtended on the sky.
 +
 +
=Example=
  
 
Luisa's most common approach, using BRC38 as an example:
 
Luisa's most common approach, using BRC38 as an example:
Obtain 20 arcmin radius catalog from 2MASS (because it is likely to contain a decent fraction of sources seen at optical and farther IR, and because it has really good coordinates).
+
*Obtain 20 arcmin radius catalog from 2MASS (because it is likely to contain a decent fraction of sources seen at optical and farther IR, and because it has really good coordinates over a large area).
Read it in, including JHK, errors, data quality flags. Remove the photometry for those with bad data quality flags but keep the coordinates.
+
*Read it in, including names, positions, JHK mags, errors, data quality flags. Remove the photometry for those with bad data quality flags but keep the coordinates.
Read in Ogura+2002 catalog. Look for matches in 2MASS catalog. For each match, copy over cross-id, Halpha, indication that it is a YSO. If there is no match, copy over as new source (name, RA, Dec, information from paper). Keep indication that this source has its RA, Dec from a different source than 2MASS.
+
*Read in oldest literature catalog, in this case Ogura+2002 catalog. Look for matches in 2MASS catalog. For each match, copy over cross-id, Halpha, indication that it is a YSO. If there is no match, copy over as new source (name, RA, Dec, information from paper). Keep indication that this source has its RA, Dec from a different source than 2MASS.
Read in Getman+2007 catalog. Retain Getman name, Nisini name, JHK, I1I2I3, Getman class, indication of which objects are in globule, which are the only ones explicitly tagged as YSOs in the paper.
+
*Read in next oldest literature catalog, in this case Getman+2007 catalog. Retain Getman name, Nisini name, JHK, I1I2I3, Getman class, indication of which objects are in globule, which are the only ones explicitly tagged as YSOs in the paper. For each Getman source, look for matches in 2MASS+Ogura catalog. For each match, copy over information from Getman catalog, including cross-id. If there is no match, copy over as new source (name, ra, dec, information from paper, and indication that coordinates come from a different source than 2MASS).
For each Getman source, look for matches in 2MASS+Ogura catalog. For each match, copy over information from Getman catalog, including cross-id. If there is no match, copy over as new source (name, ra, dec, information from paper).
+
*Read in Beltran+2009 catalog. Repeat. NB: Many sources not in 2MASS because Beltran is deeper in that central region.
Read in Beltran+2009 catalog. Repeat. Many sources not in 2MASS because Beltran is deeper in that central region.
+
*Read in Chauhan+2009. Repeat.
Read in Chauhan+2009. Repeat.
+
*Read in Choudhury+2010. Repeat. NB: some problems with T2, a few sources.
Read in Choudhury+2010. Repeat.
+
*Read in Barentsen+2011. Repeat. Only YSOs reported.
 +
*Read in Nakano+2012. Repeat. Only YSOs reported. NB: Nakano's coordinates not as good as 2MASS coordinates. Use 2MASS name and decrypt the position from that.
 +
*Read in WISE catalog. Repeat.
 +
*Read in file from Xavier, including the flags for YSOcandidates. Repeat. No new sources should be added at this step, since all should have been read in during the WISE step.
 +
*Read in Akari catalog. Repeat.
 +
*Read in full IPHAS catalog we got from Barentsen. Repeat. Do not necessarily add new rows for sources not seen to this point, because if the source is too faint in the IR for WISE or Spitzer, we won’t care about it, and it would just add more rows to the catalog.
 +
*Identify which objects may have Spitzer data so we can go get that photometry and merge it in.

Latest revision as of 19:26, 3 July 2013

Goal and Definition

The goal of bandmerging is to assemble a catalog where there is one line (or entry, or virtual index card, or however you choose to think of it) per unique object. This line contains all the measurements at all the available wavelengths and any other relevant information all collected on the same line, tied to the same unique object. We need such a catalog to make color-magnitude and color-color diagrams, SEDs, etc.

The definition of bandmerging is to take the set of detections at one band and merge the unique sources to the set of unique detections at another band. You can do this by hand, one by one, looking for the same object in multiple images or catalogs, or you can have the computer take a first pass at this (though having said that, even with the computer doing it, often there are a handful of sources that take manual fiddling to get right).

The Process

You can bandmerge catalogs you create, or catalogs you pull out of the literature, or both. (We need to do both.)

Catalogs you pull out of the literature may already have done some source matching for you. Some of these existing catalogs have “every source we detected in this region”; some have “just the things we think are interesting.” Note that not all catalogs will “see” the same objects because not all surveys go to the same integration depth, and the objects themselves change with wavelength (e.g., stars can be much fainter in IR than optical - think about the SED shape).

The fundamental approach one uses for bandmerging is the following. For each catalog, and for each object in that catalog... “Sit” on that object and look for any matches in another catalog within a certain radius on the sky.

  • Did you find one? Good, record a match, copy the information.
  • Did you NOT find one? Are you using the right radius? What are the true uncertainties in position in the two catalogs you are using? Make sure of this and look for another match. If you still don't find a match, this is still useful information, and you should make a note of this.
  • Did you find more than one match? Is this source really more than one source, or is this a coordinate issue? Are you using the right radius? How well-matched are the survey resolutions? Can you check images? Try to pick the right one to make the match, or split the original source into two in your accounting.

Potential hiccups include:

  • Different catalogs using different coordinate systems (J2000 vs. B1950, tied to 2MASS or, e.g., pulsars). This may only happen with older papers/catalogs. (In the case of BRC 38, this only matters for one paper, and you watched me fix it in real time on a telecon.)
  • Different catalogs have different resolution and coordinate uncertainties. (e.g., Chandra changes resolution over FOV.)
  • Picking a good radius over which to do the match. In general, for modern catalogs, I look for things within a 1 arcsec radius.
  • Calculate δRA, δDec with spherical trigonometry and make sure you are really measuring angle subtended on the sky.

Example

Luisa's most common approach, using BRC38 as an example:

  • Obtain 20 arcmin radius catalog from 2MASS (because it is likely to contain a decent fraction of sources seen at optical and farther IR, and because it has really good coordinates over a large area).
  • Read it in, including names, positions, JHK mags, errors, data quality flags. Remove the photometry for those with bad data quality flags but keep the coordinates.
  • Read in oldest literature catalog, in this case Ogura+2002 catalog. Look for matches in 2MASS catalog. For each match, copy over cross-id, Halpha, indication that it is a YSO. If there is no match, copy over as new source (name, RA, Dec, information from paper). Keep indication that this source has its RA, Dec from a different source than 2MASS.
  • Read in next oldest literature catalog, in this case Getman+2007 catalog. Retain Getman name, Nisini name, JHK, I1I2I3, Getman class, indication of which objects are in globule, which are the only ones explicitly tagged as YSOs in the paper. For each Getman source, look for matches in 2MASS+Ogura catalog. For each match, copy over information from Getman catalog, including cross-id. If there is no match, copy over as new source (name, ra, dec, information from paper, and indication that coordinates come from a different source than 2MASS).
  • Read in Beltran+2009 catalog. Repeat. NB: Many sources not in 2MASS because Beltran is deeper in that central region.
  • Read in Chauhan+2009. Repeat.
  • Read in Choudhury+2010. Repeat. NB: some problems with T2, a few sources.
  • Read in Barentsen+2011. Repeat. Only YSOs reported.
  • Read in Nakano+2012. Repeat. Only YSOs reported. NB: Nakano's coordinates not as good as 2MASS coordinates. Use 2MASS name and decrypt the position from that.
  • Read in WISE catalog. Repeat.
  • Read in file from Xavier, including the flags for YSOcandidates. Repeat. No new sources should be added at this step, since all should have been read in during the WISE step.
  • Read in Akari catalog. Repeat.
  • Read in full IPHAS catalog we got from Barentsen. Repeat. Do not necessarily add new rows for sources not seen to this point, because if the source is too faint in the IR for WISE or Spitzer, we won’t care about it, and it would just add more rows to the catalog.
  • Identify which objects may have Spitzer data so we can go get that photometry and merge it in.