C-CWEL Source Matching Work - Revision history

Rebull at 19:44, 7 March 2013

2013-03-07T19:44:47Z

Rebull at 18:21, 7 March 2013

2013-03-07T18:21:24Z

Rebull at 00:32, 7 March 2013

2013-03-07T00:32:22Z

Rebull at 18:27, 22 February 2013

2013-02-22T18:27:01Z

Rebull at 18:17, 22 February 2013

2013-02-22T18:17:34Z

Rebull at 18:16, 22 February 2013

2013-02-22T18:16:58Z

Rebull: /* The mechanics of what we need to do */

2013-02-22T18:11:46Z

The mechanics of what we need to do

Rebull at 18:11, 22 February 2013

2013-02-22T18:11:18Z

Rebull at 18:11, 22 February 2013

2013-02-22T18:11:00Z

Rebull at 18:09, 22 February 2013

2013-02-22T18:09:07Z

← Older revision		Revision as of 19:44, 7 March 2013
Line 1:		Line 1:
		+	This is NOT HARD, at least I don't think it should be. But it seems to trip someone up, often multiple someones, every single year. So here I have started to collect all of my verbose explanations and notes. It might just click with you right away. Or it might not. Either way is ok. This is not meant to be intimidating, just complete. :)
		+
	=Big Picture Introduction -- an analogy that might be too fanciful?=		=Big Picture Introduction -- an analogy that might be too fanciful?=

@@ Line 130: / Line 130: @@
 =The mechanics of what we need to do=
 '''Links of interest:'''
 *http://irsa.ipac.caltech.edu/applications/finderchart/ - Finder Chart
@@ Line 137: / Line 136: @@
 *obtained by typing in the coordinates of the objects in our fields from these older papers and then getting updated coordinates.
-=STILL NEEDS UPDATING AFTER HERE=
+What we need to do for each Ogura+ 2002 BRC 38 object is : type the Ogura+ 2002 coordinates into FinderChart. Get 2mass images. Overlay a 2mass catalog. Look at what comes back. Compare to finding charts in Ogura+ 2002. Identify the object that Ogura was talking about. Make a note of the new coordinates.
-[http://www.youtube.com/watch?v=fR58i8zvMwQ Here] is a video I made for my 2011 team on getting updated coordinates. This approach SHOULD work in MOST cases but did not work in every case; more on this momentarily.  What we need to do is :
+The other papers all have pretty cood coordinates.
-Then we will have 5 lists of UPDATED, HIGH QUALITY coordinates, one per paper, and we can let the computer run through the list, finding the matches between papers. We then can generate one file that purports to have one line per literature object, with all the relevant data on that line. The difficulty comes in that inevitably, a few sources during this process end up tied to the same object, or identified in other ways as duplicates or incorrect matches.
+Then we will have a set of UPDATED, HIGH QUALITY coordinates, one per paper, and we can let the computer run through the list, finding the matches between papers. We then can generate one file that purports to have one line per literature object, with all the relevant data on that line. The difficulty comes in that inevitably, a few sources during this process end up tied to the same object, or identified in other ways as duplicates or incorrect matches.
-The approach above to get updated coordinates for targets works ON THE ASSUMPTION THAT THERE ARE NOT "TOO MANY" SOURCES NEARBY, AND THAT THE PRIOR SURVEYS, BEING SHALLOW, ARE MOST LIKELY TO MATCH TO THE BRIGHT 2MASS SOURCE. As I say, this should work in MOST cases but not ALL of them.
+The approach above to get updated coordinates for targets works ON THE ASSUMPTION THAT THERE ARE NOT "TOO MANY" SOURCES NEARBY. Some of this gets into resolution issues, which we will talk about in the future. As I say, this should work in MOST cases but not ALL of them.

@@ Line 72: / Line 72: @@
 The source lists include:
 *All "bright enough" sources seen in the WISE maps (a conceptual list only)
-*Sources in the WISE catalog of photometry (to which we will add photometry from 2MASS, Haleakala, and Spitzer in the places where we have that data)
+*Sources in the WISE catalog of photometry (to which we will add photometry from 2MASS, IPHAS, and Spitzer (and Akari) in the places where we have that data)
 *Sources in this general direction studied by anyone else, ever (the majority of those reported are also YSOs, but not all of them) -- this is the list we are trying to assemble here.

@@ Line 165: / Line 165: @@
 | X-rays. coordinates likely just fine.
 | got all tables from journal. LMR has merged t1,t2,t3 together.
-|need to merge it to rest of catalogs.<BR> '''does it really have all of [http://web.ipac.caltech.edu/staff/rebull/working/cwayspapers/3-oktoskip/nisini01.pdf Nisini et al. (2001)] already in t3? - YES, ''' ''Getman Table 3 contains K, 2MASS and Spitzer data for all 20 ''Embedded Young objects'' from Nisini Table 2 <BR> Getman Tables 1 and 2 contain X-Ray and MIR data for Nisini stars 2, 3, 10, 11<BR>--Peggy Piper 20:56, 10 June 2012 (PDT)'' <br> <font color="green">NOW MERGED INTO BIG ARRAY-7/3/12 --LMR  NB: all sources detected are tabulated; the globule sources are the only ones clearly asserted as likely young, so i've just tagged those as YSOs.</font>
+|need to merge it to rest of catalogs.<BR> '''does it really have all of [http://web.ipac.caltech.edu/staff/rebull/working/cwayspapers/3-oktoskip/nisini01.pdf Nisini et al. (2001)] already in t3? - YES, ''' ''Getman Table 3 contains K, 2MASS and Spitzer data for all 20 ''Embedded Young objects'' from Nisini Table 2 <BR> Getman Tables 1 and 2 contain X-Ray and MIR data for Nisini stars 2, 3, 10, 11<BR>--Peggy Piper 20:56, 10 June 2012 (PDT)'' <br> NOW MERGED INTO BIG ARRAY-7/3/12 --LMR  NB: all sources detected are tabulated; the globule sources are the only ones clearly asserted as likely young, so i've just tagged those as YSOs.
 |-
 | Beltran et al. 2009
 | NIR. coordinates likely just fine.
 | got all tables from journal. ([http://web.ipac.caltech.edu/staff/rebull/working/cwayspapers/1-essential/beltran09data/ link])
-| individual tables have been merged together. merged into big array --7/3/12 --LMR.  NB: all sources detected are tabulated! text talks a lot about how there are YSOs there, but THERE IS NO TABLE OF JUST YSOs. Sigh. Merged photometry in but cannot of course tag the ones they think are young.</font>
+| individual tables have been merged together. merged into big array --7/3/12 --LMR.  NB: all sources detected are tabulated! text talks a lot about how there are YSOs there, but THERE IS NO TABLE OF JUST YSOs. Sigh. Merged photometry in but cannot of course tag the ones they think are young.
 |-
 |Choudhury et al. 2010
 | MIR. coordinates likely just fine.
 | got all tables from journal. ([http://web.ipac.caltech.edu/staff/rebull/working/cwayspapers/1-essential/choud10_data/ link])
-| tables from paper have been merged together. merged into big array --7/3/12 --LMR. NB: not entirely clear if all objects are being asserted as YSOs or not. Includes stuff in later table with a spectral type for an object that I can't find in Ogura (Ogura-BRC38-17). Merged photometry in but cannot of course tag the ones they think are young.</font>
+| tables from paper have been merged together. merged into big array --7/3/12 --LMR. NB: not entirely clear if all objects are being asserted as YSOs or not. Includes stuff in later table with a spectral type for an object that I can't find in Ogura (Ogura-BRC38-17). Merged photometry in but cannot of course tag the ones they think are young.
 |-
 | Chauhan et al. 2009
 | NIR, MIR (IRAC only). 27 and 38. coordinates ... odd. look at carefully.
 | got all tables from journal. ([http://web.ipac.caltech.edu/staff/rebull/working/cwayspapers/1-essential/chauhan09data/ link])
-| need to merge their tables together (will take some care! - do this if the rest of the bold stuff here is done and you're chomping at the bit for more), then merge that to rest of catalogs. <BR>Chauhan table 6 has been merged with table 3.  Sources within .13 seconds for right ascension and 2 arcseconds for declination were considered matches (2 asec = ~.13 sec).  All 2MASS table 3 to ra/dec table 6 were much smaller than this.  Three 2MASS table 3 to ra/dec table 3 had slightly larger mismatch (0.15 and 0.17 sec mismatch).  Also checked 2MASS names against Ogura list and all are correct.  Origin/meaning of non matching? Megeath or Allen et al. 2004?  [[Media:ChauhanTableMerged1.xls]]  [[Media:ChauhanTableMerged1.txt]] <font color="green">NOW MERGED INTO BIG ARRAY-7/3/12 --LMR.  NB: just YSOs tabulated, though for 27 there are indeed sources that appear in the IRAC table that do not appear in the JHK table.</font>
+| need to merge their tables together (will take some care! - do this if the rest of the bold stuff here is done and you're chomping at the bit for more), then merge that to rest of catalogs. <BR>Chauhan table 6 has been merged with table 3.  Sources within .13 seconds for right ascension and 2 arcseconds for declination were considered matches (2 asec = ~.13 sec).  All 2MASS table 3 to ra/dec table 6 were much smaller than this.  Three 2MASS table 3 to ra/dec table 3 had slightly larger mismatch (0.15 and 0.17 sec mismatch).  Also checked 2MASS names against Ogura list and all are correct.  Origin/meaning of non matching? Megeath or Allen et al. 2004?  [[Media:ChauhanTableMerged1.xls]]  [[Media:ChauhanTableMerged1.txt]] NOW MERGED INTO BIG ARRAY-7/3/12 --LMR.  NB: just YSOs tabulated, though for 27 there are indeed sources that appear in the IRAC table that do not appear in the JHK table.
 |-
 | Barentsen et al. 2011
 | iprime,rprime, halpha.
 | got all tables from journal.
-| LMR has already merged it to rest of catalogs. -7/3/12 --LMR.  NB: just YSOs tabulated</font>
+| LMR has already merged it to rest of catalogs. -7/3/12 --LMR.  NB: just YSOs tabulated
 |-
 | Nakano et al. 2012
 | iprime, halpha, some AKARI.
 | got all tables from journal. ([http://web.ipac.caltech.edu/staff/rebull/working/cwayspapers/1-essential/nakano12data/ link])
-| LMR has already merged it to rest of catalogs. -7/3/12 --LMR.  NB: just YSOs tabulated</font>
+| LMR has already merged it to rest of catalogs. -7/3/12 --LMR.  NB: just YSOs tabulated
 |-
 | Ogura et al. 2002
 | '''there are finding charts!!!'''
 | 38 done at least first pass by cways. [http://web.ipac.caltech.edu/staff/rebull/working/cwayspapers/1-essential/group%20S/ogura02data/ link] for data files as downloaded from journal
-| '''COORD NEED **UPDATING**'''  need to obtain list of objects as published, find match in 2mass, assemble list of matches betw this and 2mass. you probably will need the finding charts in the paper to sort out the matches.<BR>Ogura list updated 20120625 (sorry, ignore the date in the name)<BR>text [[File:CWAYSBRC273438Oguraids_20120614.txt]]<BR>excel[[File:CWAYSBRC273438Oguraids_20120614.xls]]<BR>coordinates entered and matched, Peggy's groups cross checked within group, with Rebull 2012,and with Jackie's group. waiting Lauren and Jackie's groups cross check.  MERGED INTO BIG ARRAY-7/3/12 --LMR.  NB: just YSOs tabulated</font>
+| '''COORD NEED **UPDATING**'''  need to obtain list of objects as published, find match in 2mass, assemble list of matches betw this and 2mass. you probably will need the finding charts in the paper to sort out the matches.<BR>Ogura list updated 20120625 (sorry, ignore the date in the name)<BR>text [[File:CWAYSBRC273438Oguraids_20120614.txt]]<BR>excel[[File:CWAYSBRC273438Oguraids_20120614.xls]]<BR>coordinates entered and matched, Peggy's groups cross checked within group, with Rebull 2012,and with Jackie's group. waiting Lauren and Jackie's groups cross check.  MERGED INTO BIG ARRAY-7/3/12 --LMR.  NB: just YSOs tabulated
 |}

@@ Line 137: / Line 137: @@
 *obtained by typing in the coordinates of the objects in our fields from these older papers and then getting updated coordinates.
-=STOPPED UPDATING HERE=
+=STILL NEEDS UPDATING AFTER HERE=
 [http://www.youtube.com/watch?v=fR58i8zvMwQ Here] is a video I made for my 2011 team on getting updated coordinates. This approach SHOULD work in MOST cases but did not work in every case; more on this momentarily.  What we need to do is :

@@ Line 133: / Line 133: @@
 *http://irsa.ipac.caltech.edu/applications/finderchart/ - Finder Chart
-For each of these papers, we need a machine-readable (read as "plain text file that the computer can parse into individual numbers rather than images of numbers or gobbledegook from microsoft") version of the relevant data tables. This was either:
+For each of these papers, we need a machine-readable (read as "plain text file that the computer can parse into individual numbers rather than images of numbers or gobbledegook from microsoft") version of the relevant data tables. This is either:
 *obtained from the journal itself, in which case the data table is typically much longer than we need
 *obtained by typing in the coordinates of the objects in our fields from these older papers and then getting updated coordinates.

@@ Line 124: / Line 124: @@
 *Chauhan et al. 2009 - BVI, NIR, IRAC. coord ok. though some inconsistencies. NEED TO CHECK FOR BRC 38. got all tables.
 *Barentsen et al. 2011 - optical (r,i,Ha). coord ok. got all tables. Would be nice to get table of everything in region we care about, not just that which they are reporting on. Need to go to main IPHAS archive.
-*Nakano et al. 2012 - optical (r,i,Ha); IR (AKARI). coord ok. got all tables, i think. NEED TO CHECK if i have AKARI too. Might as well go and get table from AKARI of everything in the region, though I think WISE will be more powerful.
+*Nakano et al. 2012 - optical (r,i,Ha); IR (AKARI). coord ok. got all tables, i think. Might as well go and get table from AKARI of everything in the region, though I think WISE will be more powerful.
@@ Line 131: / Line 131: @@
 '''Links of interest:'''
 *http://irsa.ipac.caltech.edu/applications/Gator/ - Gator
-*http://irsa.ipac.caltech.edu/applications/FinderChart/ - Finder Chart
+*http://irsa.ipac.caltech.edu/applications/finderchart/ - Finder Chart
 For each of these papers, we need a machine-readable (read as "plain text file that the computer can parse into individual numbers rather than images of numbers or gobbledegook from microsoft") version of the relevant data tables. This was either:
-*obtained from the journal itself, in which case the data table is much longer than we need
+*obtained from the journal itself, in which case the data table is typically much longer than we need
 *obtained by typing in the coordinates of the objects in our fields from these older papers and then getting updated coordinates.
-[http://www.youtube.com/watch?v=fR58i8zvMwQ Here] is a video I made for last year's team on getting updated coordinates. This approach SHOULD work in MOST cases but did not work in every case; more on this momentarily.  What we need to do is :
+STOPPED UPDATING HERE
 *type the 1950 coordinates into the 2mass point source archive, making sure that it knows that it is 1950 coordinates; or type in the 2000 coordinates (and it will assume that they are 2000 coordinates).
 *look at what comes back, and take the closest *bright* object.

@@ Line 81: / Line 81: @@
 The Venn diagram is even trying to correctly represent the relative sizes of the circles in that "all bright enough sources" ought to be darn close to "sources in the catalog" and that there will be some "sources in this general direction..." not in the regions we care about, and some of those sources that do not have IR excesses.
-NOW we are going to work on the list of "Sources in this general direction studied by anyone else, ever" for BRC27 as an example.
+NOW we are going to work on the list of "Sources in this general direction studied by anyone else, ever".
@@ Line 88: / Line 88: @@
 For this diagram, I tried to spatially represent the concepts behind what we're doing now (on this page), but I admit the circles are not as carefully constructed/laid out as the first one!
-Each of the 5 papers studying things in the region of BRC27 (Ogura et al 2002, Gregorio Hetem et al. 2009, Chauhan et al 2009, Shevchenko et al 1999 and Wiramihardja et al 1986) looked in the direction of BRC27. Surely, then, they saw some of the same sources as each other, and as what we are seeing. For example, the Gregorio-Hetem survey covered a HUGE area, and we care only about a part of it. Ogura saw some of the same sources that Gregorio-Hetem did, but not all of them -- they were not covering the same areas, but even within the same area, they did not see the same sources, because one survey was X-ray driven, and one was Halpha driven. They '''*will*''' see different sources, not only because they're looking at different wavelengths, but also because each survey is not infinitely deep -- the sensitivity of the surveys is limited, and as such will not see every source in this direction. Same for each other pair of papers, and our survey.
+Each of the 7 papers studying things in the region of BRC38 (Ogura et al. 2002, Getman et al. 2007, Beltran et al. 2009, Choudhury et al. 2010, Chauhan et al. 2009, Barentsen et al. 2011, Nakano et al. 2012) looked in the direction of BRC38. Surely, then, they saw some of the same sources as each other, and as what we are seeing. For example, the Getman survey with Chandra and the Choudhury Spitzer data covered comparable regions but not exactly the same. Ogura saw some of the same sources that Getman did, but not all of them -- even within the same area, they did not see the same sources, because one survey was X-ray driven, and one was Halpha driven. They '''*will*''' see different sources, not only because they're looking at different wavelengths, but also because each survey is not infinitely deep -- the sensitivity of the surveys is limited, and as such will not see every source in this direction. (For example, Beltran et al. is working in JHK but deeper than 2MASS.) Same for each other pair of papers, and our survey.
@@ Line 97: / Line 97: @@
 *'''Which objects from paper x are also seen in paper y?'''
 and then, the next step we will take is
-*Which of these objects are seen in the WISE data?
+*Which of these objects are seen in the WISE data? (for those that are missing, why are they missing?)
 =The Challenges=
@@ Line 113: / Line 113: @@
 There are plenty of sources that are very close together. Even among just the YSO candidates, some are very close to each other, closer than 5 arcseconds.
-If we had fabulous coordinates for everything, we could let the computer match them all up and not worry about it. But we don't. And it's not just a matter of precessing the B1950 ones to J2000 ones either - there are inherent errors in those old coordinates which will not necessarily smoothly map into a clean match to other objects.
+If we had fabulous coordinates for everything, we could let the computer match them all up and not worry about it. It turns out that for most of these literature surveys in BRC 38, they are pretty well-matched, and the computer's automatic merging gets us a long way. But it's always good to remember that just because the computer says it doesn't mean it's right.
 If we had images, we could line them up by eye and identify the same objects in each frame. I don't necessarily mean "line them up in ds9" (which would be the ideal case). But also, you can identify the objects simply by comparison between images they publish and images to which you have access (IRAC, 2MASS, POSS). This is what we are going to have to retreat to, in the tough cases.
-Here are some notes on the 5 BRC 27 papers, in no particular order:
+Here are some notes on the 7 BRC 38 papers, in chronological order:
-*Gregorio-Hetem - finding charts for a few complex fields, which may or may not be in our region. J2000 coordinates tied to 2mass, though, so less worried about these. note that their discussion includes this very conceptual problem -- they are trying to tie X-ray sources to optical sources.
+*Ogura et al. 2002 - COORD NEED UPDATING, but CWAYS did this for you. Probably good to at least take a quick look at these, just in case. there are finding charts.
-*Wiramihardja - no finding charts; a few very coarse-scale ones, one with just YSOs. Not a lot of help here. Also just 1950 coordinates read off a photographic plate. Likely to be large and unsystematic errors.
+*Getman et al. 2007 - x-rays. got all tables. includes all relevant results from Nisini et al. coord ok because they are from Chandra. Good to remember that Chandra's PSF changes the further you go off-axis - objects on the edge of the Chandra field are much poorer resolution than near the center.
-*Shevchenko - finding chart provided (fig 1). 1950 coordinates. big field (bigger than the region we care about).
+*Beltran et al. 2009 - NIR. got all tables. coord should be ok.
-*Ogura - finding chart provided; J2000 coordinates but not necessarily tied to 2MASS.
+*Choudhury et al. 2010 - IRAC, MIPS, optical. got all tables. coord should be ok.
-*Chauhan - no real finding charts provided; there are a few coarse-scale ones. J2000 coordinates tied to 2mass, though, so less worried about these.
+*Chauhan et al. 2009 - BVI, NIR, IRAC. coord ok. though some inconsistencies. NEED TO CHECK FOR BRC 38. got all tables.
 =The mechanics of what we need to do=