Difference between revisions of "C-WAYS source matching work"

From CoolWiki
Jump to navigationJump to search
m
m
 
(71 intermediate revisions by 5 users not shown)
Line 1: Line 1:
 
=Big Picture Introduction -- an analogy that might be too fanciful?=
 
=Big Picture Introduction -- an analogy that might be too fanciful?=
  
[[image:oldmap1.png|left]] [[image:oldmap2.png|right]] When westerners first discovered the Americas, they were coming at it from the perspective of someone in a boat, with minimal information about what the landforms really were. Their maps look strange to those of us used to seeing images of these landforms from space, but we have a whole lot more information now than they did then.  
+
[[image:oldmap1.png|left]] [[image:oldmap2.png|right]] When westerners first discovered the Americas, they had largely set out with the goal of finding gold or other treasure.  They were coming at the new continent from the perspective of someone in a boat, with minimal information about what the landforms really were, except for what they could see with their own eyes. Their maps look strange to those of us used to seeing images of these landforms from space, but we have a whole lot more information now than they did then.  
  
The first thing that these early Western explorers were able to attempt to map was the coasts, because that's what they had the most information about, and the most immediate need to know.  They needed to know where coral reefs were that might damage their ships, and where the big rivers emptied into the sea so that they could take on more fresh water.  They also could learn about more land faster when boating up the rivers rather than walking.
+
The first thing that these early Western explorers were able to attempt to map was the coasts, because that's what they had the most information about...and the most immediate need to know.  They needed to know where coral reefs were that might damage their ships, and where the big rivers emptied into the sea so that they could take on more fresh water.  They also could learn about more land faster when boating up the rivers rather than walking.
  
 
As more and more boats explored the coasts, the maps got better, but they still seem distorted compared to the landforms we know today. In some of these early maps of the Americas, when Europe and Africa were included, even the African coast on the Mediterranean side doesn't look all that realistic, compared to what we know now.  
 
As more and more boats explored the coasts, the maps got better, but they still seem distorted compared to the landforms we know today. In some of these early maps of the Americas, when Europe and Africa were included, even the African coast on the Mediterranean side doesn't look all that realistic, compared to what we know now.  
  
As the westerners pushed further and further into the land (American or for that matter African) from the coasts, their knowledge became deeper and deeper about what the continent actually looked like. Their knowledge was first clustered around the rivers, again because that's what they needed that best enabled them to travel the furthest, but it expanded as fast as they could expand.  
+
As the westerners pushed further and further into the land (in the Americas or for that matter Africa) from the coasts, their knowledge deepened about what the continent actually looked like, aided by improvements in technology (such as more accurate ways of measuring longitude). Their knowledge of the land started in clumps around the rivers, again because that's what they needed that best enabled them to travel the furthest.  But, their knowledge expanded as fast as they could expand. And their goals changed too -- certainly some were still looking for treasure (or freedom from persecution, religious or otherwise), but more in the earliest years were just trying to survive (here I'm thinking of Jamestown or Roanoke). They explored to find more food to eat (critters or plants). 
  
The Native Americans, of course, had a perfectly good understanding of what their land looked like, but there too, one tribe only knew the land near them -- my guess is that the Powhatan tribe (in VA) had no idea whatsoever what the Sioux tribe's lands looked like. But the Native Americans were observing the land in a different way, having lived there for a while and having their own methods of exploration.  Once westerners realized that they could learn from the Native Americans (here I'm thinking of Lewis and Clark), their knowledge could expand even faster once they paid attention to what was already known.
+
The Native Americans, of course, had a perfectly good understanding of what their land looked like, but even so, most likely, one tribe only knew the land near them -- my guess is that the Powhatan tribe (in VA) had no idea whatsoever what the Sioux tribe's lands looked like, even if trade routes were such that items could move from the Dakotas to Virginia. But the Native Americans were observing the land in a different way, having lived there for a while and having their own methods of exploration.  Once westerners realized that they could learn from the Native Americans (here I'm thinking of Lewis and Clark), their knowledge could expand even faster once they paid attention to what was already known.
  
There were, at nearly every stage of these early maps, regions that were sort of hazy and unexplored, e.g., "here be dragons". Someone might have a vague idea of what was there, but maybe only on the edges; no one known to the map maker had any detailed knowledge of what was there.
+
There were, at nearly every stage of these early maps, regions that were sort of hazy and unexplored, e.g., "here be dragons". Someone might have a vague idea of what was there, but maybe only on the edges; no one (at least no one known to the map maker) had any detailed knowledge of what was there.
  
SO NOW.... Bringing this closer to our work... Some people have spent some time exploring parts of this region before. It will make our knowledge expand faster if we pay attention to what is already known before setting off on our own journey. Others have mapped different parts of the region using different methods of exploration before -- some on foot and some on horseback, and still others in boats.  Some people just ran through this region identifying the big trees and big rocks.  Some people wrote what they learned down in Algonquin (what the Powhatan spoke) and we need to translate it before it makes sense in the language we speak.  Some people cared a lot about a tiny patch next to the river and know that region really well, but beyond the borders of their camp, "here be dragons." There is ''some'' information about the area around the camp that we can obtain from other people and from exploring on our own, but we can come back to this very well-known region and check what we think we know about the rest of the area by testing it on the well-known region. The people who know the region right next to the river really well also know that this critter or that plant is good food and won't make us sick. When we go exploring out further from the camp, if we find a critter or a plant that we think is the same as the stuff that the people next to the river know is ok to eat, we can bring it back to that camp to compare it and see if it is, in fact, the same or something new.  
+
SO NOW.... here is a perhaps tortured analogy... We would like to go exploring in a particular region, making note of where the big landforms are, and we have a specific goal of finding edible animals and plants to support our efforts in further explorations. Some people have spent some time exploring parts of this region before. It will make our knowledge expand faster if we pay attention to what is already known before setting off on our own journey. Others have mapped different parts of the region using different methods of exploration before -- some on foot and some on horseback, and still others in boats.  Some people just ran through this region identifying the big trees and big rocks.  Some people wrote down what they learned in Algonquin (what the Powhatan spoke) and we need to translate it before it makes sense in the language we speak.  Some people cared a lot about their tiny camp next to the river and they know that region really well, but beyond the borders of their camp, "here be dragons." There is ''some'' information about the area around the camp that we can obtain from other people and from exploring on our own, but we can also come back to this very well-known region and check what we think we know about the rest of the area by testing it on the well-known region. The people who know the region right next to the river really well also know that this animal or that plant is good food and won't make us sick. When we go exploring out further from the camp, if we find a critter or a plant that we think is the same as the stuff that the people next to the river know is ok to eat, we can bring it back to that camp to compare it and see if it is, in fact, the same or something new.  We can also, among the animals and plants we find, put them in groups of apparently similar things -- these are all 4-footed furry critters, and those have feathers and wings.  
  
 
Explicitly drawing lines between that analogy and reality:
 
Explicitly drawing lines between that analogy and reality:
Line 20: Line 20:
 
|'''reality translation'''
 
|'''reality translation'''
 
|-  
 
|-  
 +
|We would like to go exploring in a particular region, making note of where the big landforms are, and we have a specific goal of finding edible animals and plants to support our efforts in further explorations. 
 +
|We have a goal of exploring a region (well, really 3) on the sky, specifically looking for young stars.
 +
|-
 
|Some people have spent some time exploring parts of this region before. It will make our knowledge expand faster if we pay attention to what is already known before setting off on our own journey.
 
|Some people have spent some time exploring parts of this region before. It will make our knowledge expand faster if we pay attention to what is already known before setting off on our own journey.
 
| We need to read and understand the literature.
 
| We need to read and understand the literature.
Line 43: Line 46:
 
| The people who know the region right next to the river really well also know that this critter or that plant is good food and won't make us sick.  When we go exploring out further from the camp, if we find a critter or a plant that we think is the same as the stuff that the people next to the river know is ok to eat, we can bring it back to that camp to compare it and see if it is, in fact, the same or something new.  
 
| The people who know the region right next to the river really well also know that this critter or that plant is good food and won't make us sick.  When we go exploring out further from the camp, if we find a critter or a plant that we think is the same as the stuff that the people next to the river know is ok to eat, we can bring it back to that camp to compare it and see if it is, in fact, the same or something new.  
 
| We have a region of space that is very well studied with Spitzer, and serendipitous Spitzer data nearby. We will use WISE over the ''whole region'' to find things that we think might be YSOs. We can check our hunch that some of the objects are in fact YSOs by comparing what we get to the Spitzer data where we can, and including that data in our analysis.  
 
| We have a region of space that is very well studied with Spitzer, and serendipitous Spitzer data nearby. We will use WISE over the ''whole region'' to find things that we think might be YSOs. We can check our hunch that some of the objects are in fact YSOs by comparing what we get to the Spitzer data where we can, and including that data in our analysis.  
 +
|-
 +
|We can also, among the animals and plants we find, put them in groups of apparently similar things -- these are all 4-footed furry critters, and those have feathers and wings.
 +
|Among the objects we find, we can put them in groups based on the shape of their SED, from 'really embedded' (class 0-I) down to 'not much of an IR excess' (class II-III).
 
|}
 
|}
  
We will not be able to get a comprehensive end-all-be-all understanding of the region (e.g., in the analogy, we will not go straight from Lewis and Clark to weather satellite views of the continent). We can, however, do the best that we can with the information we have, by learning from those who have gone before, learning as we go, and making intelligent guesses about what we don't know.
+
We will not be able to get a comprehensive be-all-end-all understanding of the region (e.g., in the analogy, we will not go straight from Columbus or even Lewis and Clark to weather satellite views of the continent with a GPS in our car as we drive). We can, however, do the best that we can with the information we have, by learning from those who have gone before, learning as we go, and making intelligent guesses about what we don't know.
 +
 
 +
And, of course, we need to do this in all of the regions we care about.  
  
 
=More specific introduction to source matching from the literature=
 
=More specific introduction to source matching from the literature=
  
Several people have done prior studies in BRC 27, BRC 34, and BRC 38 before. You worked hard to find all of these articles in the context of the proposal. Certainly investigator A working in BRC 27 in year X saw some of the same sources as investigator B working in that same region in year Y, as did investigator C in year Z.  Now we actually have to do the work of figuring out which specific sources are which in all the papers - are the sources called out in paper 1 the same or different sources as paper 2?, etc., until all the papers are exhausted, and we have a '''complete catalog of all the previously studied sources in the region'''.
+
Several people have done prior studies in BRC 27, BRC 34, and BRC 38 before, but they have NOT found all the young stars!  You worked hard to find all of these articles in the context of the proposal. Certainly investigator A working in BRC 27 in year X saw some of the same sources as investigator B working in that same region in year Y, as did investigator C in year Z.  Now we actually have to do the work of figuring out which specific sources are which in all the papers - are the sources called out in paper 1 the same or different sources as paper 2?, etc., until all the papers are exhausted, and we have a '''complete catalog of all the previously studied sources in the region'''.
  
 
The thing that makes this complicated is that, even though everyone is reporting in RA and Dec, not everyone is using exactly the same system (some are 1950 coordinates and some are 2000 coordinate), and not everyone has the same coordinate accuracies (some are working off of photographic plates, and some are working off large-format CCDs). And, what does it mean to have "the same" coordinates -- is within an arcsecond ok? 5 arcseconds? an arcminute?  This is where it gets tricky, and where you have to apply your brain! Spitzer, WISE, and 2MASS are all using exactly the same, high-accuracy coordinate system -- it's all tied to 2MASS's J2000 coordinates -- but even then the position of the same object will not be EXACTLY the same in each image, in each catalog, because there is a limit to the precision with which we can identify the coordinates.  Where possible, we need to update the old coordinates by comparing what the old papers say to the 2MASS data. Then we need to fold in the objects with newer coordinates into our collection of sources.
 
The thing that makes this complicated is that, even though everyone is reporting in RA and Dec, not everyone is using exactly the same system (some are 1950 coordinates and some are 2000 coordinate), and not everyone has the same coordinate accuracies (some are working off of photographic plates, and some are working off large-format CCDs). And, what does it mean to have "the same" coordinates -- is within an arcsecond ok? 5 arcseconds? an arcminute?  This is where it gets tricky, and where you have to apply your brain! Spitzer, WISE, and 2MASS are all using exactly the same, high-accuracy coordinate system -- it's all tied to 2MASS's J2000 coordinates -- but even then the position of the same object will not be EXACTLY the same in each image, in each catalog, because there is a limit to the precision with which we can identify the coordinates.  Where possible, we need to update the old coordinates by comparing what the old papers say to the 2MASS data. Then we need to fold in the objects with newer coordinates into our collection of sources.
Line 55: Line 63:
 
Part of the challenge here is bookkeeping -- writing down coordinates correctly, keeping track of which sources are which, and getting the correct data matched to the correct source.
 
Part of the challenge here is bookkeeping -- writing down coordinates correctly, keeping track of which sources are which, and getting the correct data matched to the correct source.
  
Last year, I thought this would be a relatively simple project that could be done before the summer visit. However, it turned out not to be the case. Part of this is, I think, the complexity of the region, BRC 27 in particular (which is of course still a problem this year), and part of it was I think my ability to explain it!  I've pulled out and updated all of my best(?) explanations and descriptions here. IF IT DOESN'T MAKE SENSE, PLEASE ASK QUESTIONS. If this is done wrong, or only halfway done, it will make for a LOT of pain downstream.  
+
Last year, I thought this would be a relatively simple project that could be done before the summer visit. However, it turned out not to be the case. Part of this is, I think, the complexity of the region, BRC 27 in particular (which is of course still a problem this year), and part of it was I think my ability to explain it!  I've pulled out and updated all of my best(?) explanations and descriptions here. IF IT DOESN'T MAKE SENSE, PLEASE ASK QUESTIONS. If this is done wrong, or only halfway done, it will make for a LOT of pain downstream. ''Trust me.''
  
=Another analogy - 1950 vs. 2000 coordinates=
+
=1950 vs. 2000 Coordinates and Yet Another Analogy=
  
 
Review the differences between the 1950 and 2000 coordinate systems. '''REMINDER: THE STARS ARE NOT MOVING.''' (Several people last year kept saying things like "wow the stars move a lot" and if you say that, I WILL correct you!) '''''The coordinate system is moving, not the stars.''''' (Well, technically the stars may really be moving, but we don't have that information, and the size of any such motion will be on the order of a tenth of an arcsec or less; the change due to the change in coordinate systems is much greater.)  
 
Review the differences between the 1950 and 2000 coordinate systems. '''REMINDER: THE STARS ARE NOT MOVING.''' (Several people last year kept saying things like "wow the stars move a lot" and if you say that, I WILL correct you!) '''''The coordinate system is moving, not the stars.''''' (Well, technically the stars may really be moving, but we don't have that information, and the size of any such motion will be on the order of a tenth of an arcsec or less; the change due to the change in coordinate systems is much greater.)  
Line 80: Line 88:
 
=Venn Diagrams and Bookeeping=
 
=Venn Diagrams and Bookeeping=
  
One of the difficulties we will have during this project is keeping all the source lists straight. It happens every year, and I don't know how to make it easier, except for warning you that it will happen! Here is a Venn diagram explaining, roughly, the various source lists we will have, at minimum.  
+
One of the difficulties we will have during this project is keeping all the source lists straight. It happens every year, and I don't know how to make it easier, except for warning you that it will happen! Here is a Venn diagram explaining, roughly, the various source lists we will have before we are done, at minimum. This Venn diagram is meant to be a "big picture" sort of thing; this page on the source matching is meant to address just, well, the previously identified sources.
  
 
[[image:brcvenn1.png]]  
 
[[image:brcvenn1.png]]  
Line 87: Line 95:
 
*All "bright enough" sources seen in the WISE maps (a conceptual list only)
 
*All "bright enough" sources seen in the WISE maps (a conceptual list only)
 
*Sources in the WISE catalog of photometry (to which we will add photometry from 2MASS, Haleakala, and Spitzer in the places where we have that data)
 
*Sources in the WISE catalog of photometry (to which we will add photometry from 2MASS, Haleakala, and Spitzer in the places where we have that data)
*Sources in this general direction studied by anyone else, ever (the majority of those reported are also YSOs, but not all of them)
+
*Sources in this general direction studied by anyone else, ever (the majority of those reported are also YSOs, but not all of them) -- this is the list we are trying to assemble here.
  
Out of those sets, our scientific goals mean that we are striving to identify:
+
Out of those sets, our ultimate scientific goals mean that we are striving to identify:
 
*YSO candidates we select from IR excess
 
*YSO candidates we select from IR excess
 
*YSOs that others identify that do not appear to have an IR excess.
 
*YSOs that others identify that do not appear to have an IR excess.
Line 100: Line 108:
 
[[image:brcvenn2.png]]
 
[[image:brcvenn2.png]]
  
For this diagram, I tried to spatially represent the concepts behind what we're doing now, but I admit the circles are not as carefully constructed/laid out as the first one!
+
For this diagram, I tried to spatially represent the concepts behind what we're doing now (on this page), but I admit the circles are not as carefully constructed/laid out as the first one!
  
 
Each of the 5 papers studying things in the region of BRC27 (Ogura et al 2002, Gregorio Hetem et al. 2009, Chauhan et al 2009, Shevchenko et al 1999 and Wiramihardja et al 1986) looked in the direction of BRC27. Surely, then, they saw some of the same sources as each other, and as what we are seeing. For example, the Gregorio-Hetem survey covered a HUGE area, and we care only about a part of it. Ogura saw some of the same sources that Gregorio-Hetem did, but not all of them -- they were not covering the same areas, but even within the same area, they did not see the same sources, because one survey was X-ray driven, and one was Halpha driven. They '''*will*''' see different sources, not only because they're looking at different wavelengths, but also because each survey is not infinitely deep -- the sensitivity of the surveys is limited, and as such will not see every source in this direction. Same for each other pair of papers, and our survey.
 
Each of the 5 papers studying things in the region of BRC27 (Ogura et al 2002, Gregorio Hetem et al. 2009, Chauhan et al 2009, Shevchenko et al 1999 and Wiramihardja et al 1986) looked in the direction of BRC27. Surely, then, they saw some of the same sources as each other, and as what we are seeing. For example, the Gregorio-Hetem survey covered a HUGE area, and we care only about a part of it. Ogura saw some of the same sources that Gregorio-Hetem did, but not all of them -- they were not covering the same areas, but even within the same area, they did not see the same sources, because one survey was X-ray driven, and one was Halpha driven. They '''*will*''' see different sources, not only because they're looking at different wavelengths, but also because each survey is not infinitely deep -- the sensitivity of the surveys is limited, and as such will not see every source in this direction. Same for each other pair of papers, and our survey.
Line 131: Line 139:
 
If we had images, we could line them up by eye and identify the same objects in each frame. I don't necessarily mean "line them up in ds9" (which would be the ideal case). But also, you can identify the objects simply by comparison between images they publish and images to which you have access (IRAC, 2MASS, POSS). This is what we are going to have to retreat to, in the tough cases.
 
If we had images, we could line them up by eye and identify the same objects in each frame. I don't necessarily mean "line them up in ds9" (which would be the ideal case). But also, you can identify the objects simply by comparison between images they publish and images to which you have access (IRAC, 2MASS, POSS). This is what we are going to have to retreat to, in the tough cases.
  
Here are some notes on these 5 papers, in no particular order:
+
Here are some notes on the 5 BRC 27 papers, in no particular order:
 
*Gregorio-Hetem - finding charts for a few complex fields, which may or may not be in our region. J2000 coordinates tied to 2mass, though, so less worried about these. note that their discussion includes this very conceptual problem -- they are trying to tie X-ray sources to optical sources.
 
*Gregorio-Hetem - finding charts for a few complex fields, which may or may not be in our region. J2000 coordinates tied to 2mass, though, so less worried about these. note that their discussion includes this very conceptual problem -- they are trying to tie X-ray sources to optical sources.
 
*Wiramihardja - no finding charts; a few very coarse-scale ones, one with just YSOs. Not a lot of help here. Also just 1950 coordinates read off a photographic plate. Likely to be large and unsystematic errors.
 
*Wiramihardja - no finding charts; a few very coarse-scale ones, one with just YSOs. Not a lot of help here. Also just 1950 coordinates read off a photographic plate. Likely to be large and unsystematic errors.
Line 138: Line 146:
 
*Chauhan - no real finding charts provided; there are a few coarse-scale ones. J2000 coordinates tied to 2mass, though, so less worried about these.
 
*Chauhan - no real finding charts provided; there are a few coarse-scale ones. J2000 coordinates tied to 2mass, though, so less worried about these.
  
For each of these 5 papers, we need a machine-readable (read as "plain text file that the computer can parse into individual numbers rather than images of numbers") version of the relevant data tables. This was either:
+
=The mechanics of what we need to do=
 +
'''Links of interest:'''
 +
*http://irsa.ipac.caltech.edu/applications/Gator/ - Gator
 +
*http://irsa.ipac.caltech.edu/applications/FinderChart/ - Finder Chart
 +
 
 +
For each of these papers, we need a machine-readable (read as "plain text file that the computer can parse into individual numbers rather than images of numbers or gobbledegook from microsoft") version of the relevant data tables. This was either:
 
*obtained from the journal itself, in which case the data table is much longer than we need
 
*obtained from the journal itself, in which case the data table is much longer than we need
*obtained by typing in the coordinates of the objects in our fields from these older papers and try to get updated coordinates.
+
*obtained by typing in the coordinates of the objects in our fields from these older papers and then getting updated coordinates.
  
 
[http://www.youtube.com/watch?v=fR58i8zvMwQ Here] is a video I made for last year's team on getting updated coordinates. This approach SHOULD work in MOST cases but did not work in every case; more on this momentarily.  What we need to do is :
 
[http://www.youtube.com/watch?v=fR58i8zvMwQ Here] is a video I made for last year's team on getting updated coordinates. This approach SHOULD work in MOST cases but did not work in every case; more on this momentarily.  What we need to do is :
Line 156: Line 169:
 
*With that information, then go back to your PSC, and then use the 2MASS point source catalog to find the actual high-precision coordinates of that specific dot you have identified in the images as the match to the literature object.
 
*With that information, then go back to your PSC, and then use the 2MASS point source catalog to find the actual high-precision coordinates of that specific dot you have identified in the images as the match to the literature object.
  
Then we can take the updated high-precision lists of coordinates, merge them together, and come up with a new-and-improved list of all the previously identified objects in this region, one line per object with all the relevant information.
+
UPDATE 31 May 12: while the OUTPUT of this process has to be in plain ASCII text, you can work in Excel or Google Docs and spit it out as plain text as the last step. [http://www.youtube.com/watch?v=nCJ3ctOGvNk Here] is a video on getting plain text (including IPAC table format) files into Excel.
 +
 
 +
Here is a list of all the papers we have assembled as part of the lit review, as well as a list of their coordinate list status. First priority items (e.g. those for you to tackle first in May/June) are in '''bold''' in the last column:
 +
{| border="1"
 +
|'''paper'''
 +
|'''notes'''
 +
|'''coordinate list status'''
 +
|'''merging status and <font color="red">pending items</font>'''
 +
|-
 +
| Getman et al 2007
 +
| X-rays. BRC 38 only. coordinates likely just fine.
 +
| got all tables from journal. LMR has merged t1,t2,t3 together.
 +
|need to merge it to rest of catalogs.<BR> '''does it really have all of [http://web.ipac.caltech.edu/staff/rebull/working/cwayspapers/3-oktoskip/nisini01.pdf Nisini et al. (2001)] already in t3? - YES, ''' ''Getman Table 3 contains K, 2MASS and Spitzer data for all 20 ''Embedded Young objects'' from Nisini Table 2 <BR> Getman Tables 1 and 2 contain X-Ray and MIR data for Nisini stars 2, 3, 10, 11<BR>--Peggy Piper 20:56, 10 June 2012 (PDT)'' <br> <font color="green">NOW MERGED INTO BIG ARRAY-7/3/12 --LMR  NB: all sources detected are tabulated; the globule sources are the only ones clearly asserted as likely young, so i've just tagged those as YSOs.</font>
 +
|-
 +
| Beltran et al. 2009
 +
| NIR. 38 only.  coordinates likely just fine.
 +
| got all tables from journal. ([http://web.ipac.caltech.edu/staff/rebull/working/cwayspapers/1-essential/beltran09data/ link])
 +
| <strike>need to merge relevant bits of tables together</strike>. individual tables have been merged together. still need to merge that to rest of catalogs. <font color="green">NOW MERGED INTO BIG ARRAY-7/3/12 --LMR.  NB: all sources detected are tabulated! text talks a lot about how there are YSOs there, but THERE IS NO TABLE OF JUST YSOs. Sigh. Merged photometry in but cannot of course tag the ones they think are young.</font>
 +
|-
 +
|Choudhury et al. 2010
 +
| MIR. 38 only. coordinates likely just fine.
 +
| got all tables from journal. ([http://web.ipac.caltech.edu/staff/rebull/working/cwayspapers/1-essential/choud10_data/ link])
 +
| <strike>need to merge relevant bits of tables together</strike> tables from paper have been merged together. still need to merge that to rest of catalogs. <font color="green">NOW MERGED INTO BIG ARRAY-7/3/12 --LMR. NB: not entirely clear if all objects are being asserted as YSOs or not. Includes stuff in later table with a spectral type for an object that I can't find in Ogura (Ogura-BRC38-17). Merged photometry in but cannot of course tag the ones they think are young.</font>
 +
|-
 +
| Chauhan et al. 2009
 +
| NIR, MIR (IRAC only). 27 and 38. coordinates ... odd. look at carefully.
 +
| got all tables from journal. ([http://web.ipac.caltech.edu/staff/rebull/working/cwayspapers/1-essential/chauhan09data/ link])
 +
| need to merge their tables together (will take some care! - do this if the rest of the bold stuff here is done and you're chomping at the bit for more), then merge that to rest of catalogs. <BR>Chauhan table 6 has been merged with table 3.  Sources within .13 seconds for right ascension and 2 arcseconds for declination were considered matches (2 asec = ~.13 sec).  All 2MASS table 3 to ra/dec table 6 were much smaller than this.  Three 2MASS table 3 to ra/dec table 3 had slightly larger mismatch (0.15 and 0.17 sec mismatch).  Also checked 2MASS names against Ogura list and all are correct.  Origin/meaning of non matching? Megeath or Allen et al. 2004?  [[Media:ChauhanTableMerged1.xls]]  [[Media:ChauhanTableMerged1.txt]] <font color="green">NOW MERGED INTO BIG ARRAY-7/3/12 --LMR.  NB: just YSOs tabulated, though for 27 there are indeed sources that appear in the IRAC table that do not appear in the JHK table.</font>
 +
|-
 +
| Barentsen et al. 2011
 +
| iprime,rprime, halpha. BRC 34, 38.
 +
| got all tables from journal.
 +
| LMR has already merged it to rest of catalogs. <font color="green">NOW MERGED INTO BIG ARRAY-7/3/12 --LMR.  NB: just YSOs tabulated</font>
 +
|-
 +
| Nakano et al. 2012
 +
| iprime, halpha, some AKARI. BRC 34, 38.
 +
| got all tables from journal. ([http://web.ipac.caltech.edu/staff/rebull/working/cwayspapers/1-essential/nakano12data/ link])
 +
| LMR has already merged it to rest of catalogs. <font color="green">NOW MERGED INTO BIG ARRAY-7/3/12 --LMR.  NB: just YSOs tabulated</font>
 +
|-
 +
| Ogura et al. 2002
 +
| BRC 27, 34, 38.  turns out will probably need 26 too!! '''there are finding charts!!!'''
 +
| 34 is ~done. 27 only done in center (see rebull et al 2012 below). 26, 38 not touched. [http://web.ipac.caltech.edu/staff/rebull/working/cwayspapers/1-essential/group%20S/ogura02data/ link] for data files as downloaded from journal
 +
| '''COORD NEED **UPDATING**'''  34 is done (see below). 27 is done only in center (see below). 26, 38 are not touched. need to obtain list of objects as published, find match in 2mass, assemble list of matches betw this and 2mass. you probably will need the finding charts in the paper to sort out the matches.<BR>Ogura list updated 20120625 (sorry, ignore the date in the name)<BR>text [[File:CWAYSBRC273438Oguraids_20120614.txt]]<BR>excel[[File:CWAYSBRC273438Oguraids_20120614.xls]]<BR>coordinates entered and matched, Peggy's groups cross checked within group, with Rebull 2012,and with Jackie's group. waiting Lauren and Jackie's groups cross check.  There were no sources for BRC 26. <font color="green">NOW MERGED INTO BIG ARRAY-7/3/12 --LMR.  NB: just YSOs tabulated</font>
 +
|-
 +
| Gregorio-Hetem et al. 2009
 +
| BRC 27 only. X-rays. coordinates probably ok.
 +
| have whole tables from journal ([http://web.ipac.caltech.edu/staff/rebull/working/cwayspapers/2-scanfordata/gregoriodata/ link]). merged together. done in center (see below). over whole region, LMR beat against 2mass in region and took nearest within 2 arcsec. several still don't have matches. region we care about is small compared to region they worked on.
 +
| '''COORD NEED CHECKING AND POSSIBLY UPDATING.''' [[file:gregorio_20120514.txt]] here is my file of the sources which has the published RA and Dec written in two ways (both J2000) and the proposed 2mass match (done blindly by the computer). if it has an asterisk (*) in front of the name, it is in the region we care about. Is it correctly matched to the right 2mass source? if not, what should it be??<BR> Jackie and Bob source matching.
 +
[[File:SourceMatchingGregorio.xlsx‎]]
 +
[[File:SourceMatchingGregorio.txt‎]]
 +
<font color="green">NOW MERGED INTO BIG ARRAY-7/3/12 --LMR.  NB: just YSOs tabulated</font>
 +
|-
 +
| Shevchenko et al. 1999
 +
| BRC 27 only. antiquated coordinates.
 +
| have whole table from journal. done in center (see below). over whole region, LMR beat against 2mass in region and took nearest within 5(!!) arcsec. several still don't have matches. region we care about is small compared to region they worked on.
 +
| '''COORD NEED CHECKING AND POSSIBLY UPDATING.''' [[file:shevchenko_20120514.txt]] [[file:shevchenko_20120615.txt]] here is my file of the sources which has the published RA and Dec (B1950) and then a converted-to-J2000 version of the published coordinates, both written in two ways (degrees and hh:mm:ss, dd:mm:ss) and the proposed 2mass match (done blindly by the computer). if it has an asterisk (*) in front of the name, it is in the region we care about. Is it correctly matched to the right 2mass source? if not, what should it be??<BR>Waiting Debbie and Lauren source matching.<Br>[[File:Shevchenko_2012_06_27.xlsx]]<Br> The Shevchenko data had some very weird anomalies that bear further scrutiny by Luisa.  We concluded that maybe some numbers had been swapped on source #99, and there are several multiple sources that were listed as one source. <font color="green">NOW MERGED INTO BIG ARRAY-7/3/12 --LMR. Note also that there were several problems here, most likely because of version confusion, e.g., starting with 5/14 source list above, rather than 6/15 version. ''please be careful with catalog versions.'' [[File:Shevchenko_2012_06_27_lmr.xlsx]] is my updated version of your xls.  NB: just YSOs tabulated</font>
 +
|-
 +
| Wiramihardja et al. 1986
 +
| BRC 27 only. antiquated coordinates.
 +
| have nothing from journal (it's too old).
 +
|need to merge it to rest of catalogs.<BR> '''<strike>COORD NEED **UPDATING**</strike>''' done in center (see below). need to obtain list of objects as published, find match in 2mass, assemble list of matches betw this and 2mass. NB: region we care about is probably smaller than region they published. no need to find matches for objects outside of our region but may be easier to just blow through and do all of them?<BR> '''DONE''' coordinates entered, matched and data entered, Bob and Peggy's groups cross checked.<BR>text[[File:CWAYSBRC27Wiramihardjaidsdata_20120614.txt]] excel[[File:CWAYSBRC27Wiramihardjaidsdata_20120614.xls]]  <font color="green">NOW MERGED INTO BIG ARRAY-7/3/12 --LMR. NB: just YSOs tabulated</font>
 +
|-
 +
| Rebull et al. 2012
 +
| BRC 27, 34 only. coordinates and crossmatches to literature should be good. I hope!!
 +
| have tables, of course!
 +
| should hopefully be ok! [[file:rebull2012t1brcxids_20120514.txt]] -- file of all literature information, crossids. note that technically this is in latex format, so "&" divides columns and \nodata means, well, "no data".  some people provided errors, some didn't, so that is reflected in this table.  last object listed for BRC 27 is actually off the edge of the IRAC 4-band region, and no longer appears in the table. The cross-ids should actually be ok, though. [[file:rebull2012t2_20120518.txt]] -- file of new measurements discussed in Rebull et al. 2012.  Note that there are both limits and errors in this table.  <font color="green">NOW MERGED INTO BIG ARRAY-7/3/12 --LMR.  NB: just YSOs tabulated in the PAPER, but because I have access to the full catalogs we worked with, I've merged the full catalogs into our full catalogs.</font>
 +
|}
 +
 
 +
=Last (next) steps=
 +
 
 +
Once we have updated high-precision lists of coordinates from each paper, we can merge them together, and come up with a new-and-improved list of all the previously identified objects in this region in one place, one line per object with all the relevant information, in a plain text file.
 +
 
 +
Then we will compare this list to our WISE catalog. There will be some with IR excesses, and some without IR excesses. I expect that we will probably detect them all, but there might be some we do not detect.
 +
 
 +
Then we will also compare this list to where there is Spitzer data, and obtain Spitzer data for those sources where we can.
  
Then ultimately we will compare this list to our WISE+Spitzer catalog. There will be some with IR excesses, and some without IR excesses.
+
Then we will also compare this list to the optical data, where possible. There WILL be some we WILL NOT DETECT in the optical data.

Latest revision as of 21:11, 12 July 2012

Big Picture Introduction -- an analogy that might be too fanciful?

Oldmap1.png
Oldmap2.png

When westerners first discovered the Americas, they had largely set out with the goal of finding gold or other treasure. They were coming at the new continent from the perspective of someone in a boat, with minimal information about what the landforms really were, except for what they could see with their own eyes. Their maps look strange to those of us used to seeing images of these landforms from space, but we have a whole lot more information now than they did then.

The first thing that these early Western explorers were able to attempt to map was the coasts, because that's what they had the most information about...and the most immediate need to know. They needed to know where coral reefs were that might damage their ships, and where the big rivers emptied into the sea so that they could take on more fresh water. They also could learn about more land faster when boating up the rivers rather than walking.

As more and more boats explored the coasts, the maps got better, but they still seem distorted compared to the landforms we know today. In some of these early maps of the Americas, when Europe and Africa were included, even the African coast on the Mediterranean side doesn't look all that realistic, compared to what we know now.

As the westerners pushed further and further into the land (in the Americas or for that matter Africa) from the coasts, their knowledge deepened about what the continent actually looked like, aided by improvements in technology (such as more accurate ways of measuring longitude). Their knowledge of the land started in clumps around the rivers, again because that's what they needed that best enabled them to travel the furthest. But, their knowledge expanded as fast as they could expand. And their goals changed too -- certainly some were still looking for treasure (or freedom from persecution, religious or otherwise), but more in the earliest years were just trying to survive (here I'm thinking of Jamestown or Roanoke). They explored to find more food to eat (critters or plants).

The Native Americans, of course, had a perfectly good understanding of what their land looked like, but even so, most likely, one tribe only knew the land near them -- my guess is that the Powhatan tribe (in VA) had no idea whatsoever what the Sioux tribe's lands looked like, even if trade routes were such that items could move from the Dakotas to Virginia. But the Native Americans were observing the land in a different way, having lived there for a while and having their own methods of exploration. Once westerners realized that they could learn from the Native Americans (here I'm thinking of Lewis and Clark), their knowledge could expand even faster once they paid attention to what was already known.

There were, at nearly every stage of these early maps, regions that were sort of hazy and unexplored, e.g., "here be dragons". Someone might have a vague idea of what was there, but maybe only on the edges; no one (at least no one known to the map maker) had any detailed knowledge of what was there.

SO NOW.... here is a perhaps tortured analogy... We would like to go exploring in a particular region, making note of where the big landforms are, and we have a specific goal of finding edible animals and plants to support our efforts in further explorations. Some people have spent some time exploring parts of this region before. It will make our knowledge expand faster if we pay attention to what is already known before setting off on our own journey. Others have mapped different parts of the region using different methods of exploration before -- some on foot and some on horseback, and still others in boats. Some people just ran through this region identifying the big trees and big rocks. Some people wrote down what they learned in Algonquin (what the Powhatan spoke) and we need to translate it before it makes sense in the language we speak. Some people cared a lot about their tiny camp next to the river and they know that region really well, but beyond the borders of their camp, "here be dragons." There is some information about the area around the camp that we can obtain from other people and from exploring on our own, but we can also come back to this very well-known region and check what we think we know about the rest of the area by testing it on the well-known region. The people who know the region right next to the river really well also know that this animal or that plant is good food and won't make us sick. When we go exploring out further from the camp, if we find a critter or a plant that we think is the same as the stuff that the people next to the river know is ok to eat, we can bring it back to that camp to compare it and see if it is, in fact, the same or something new. We can also, among the animals and plants we find, put them in groups of apparently similar things -- these are all 4-footed furry critters, and those have feathers and wings.

Explicitly drawing lines between that analogy and reality:

analogy reality translation
We would like to go exploring in a particular region, making note of where the big landforms are, and we have a specific goal of finding edible animals and plants to support our efforts in further explorations. We have a goal of exploring a region (well, really 3) on the sky, specifically looking for young stars.
Some people have spent some time exploring parts of this region before. It will make our knowledge expand faster if we pay attention to what is already known before setting off on our own journey. We need to read and understand the literature.
Other people have mapped different parts of the region using different methods of exploration before -- some on foot and some on horseback, and still others in boats. Other people have used different wavelengths to explore this region before.
Some people just ran through this region identifying the big trees and big rocks. Some people just identified the bright young stars, or the things bright in the wavelengths they were using.
Some people wrote what they learned down in Algonquin (what the Powhatan spoke) and we need to translate it before it makes sense in the language we speak. Some people wrote down poorly constrained coordinates in epoch B1950 coordinates and we need to translate it to accurate J2000 coordinates.
Some people cared a lot about a tiny patch next to the river and know that region really well, but beyond the borders of their camp, "here be dragons." The NITARP team last year spent their year worrying about the 5'x5' patch with 4-band IRAC coverage, but did not care about anything else.
There is some information about the area around the camp that we can obtain from other people and from exploring on our own... We can comb the literature and use the 2MASS+WISE data to help guide us in exploring the region.
...but we can come back to this very well-known region and check what we think we know about the rest of the area by testing it on the well-known region. We can use WISE to identify things with YSO-like colors in the region. Did we rediscover the YSOs that other people found, particularly last year's team using Spitzer data? If not, why not? Do the objects with YSO-like colors look like point sources in 2MASS or do they look like galaxies?
The people who know the region right next to the river really well also know that this critter or that plant is good food and won't make us sick. When we go exploring out further from the camp, if we find a critter or a plant that we think is the same as the stuff that the people next to the river know is ok to eat, we can bring it back to that camp to compare it and see if it is, in fact, the same or something new. We have a region of space that is very well studied with Spitzer, and serendipitous Spitzer data nearby. We will use WISE over the whole region to find things that we think might be YSOs. We can check our hunch that some of the objects are in fact YSOs by comparing what we get to the Spitzer data where we can, and including that data in our analysis.
We can also, among the animals and plants we find, put them in groups of apparently similar things -- these are all 4-footed furry critters, and those have feathers and wings. Among the objects we find, we can put them in groups based on the shape of their SED, from 'really embedded' (class 0-I) down to 'not much of an IR excess' (class II-III).

We will not be able to get a comprehensive be-all-end-all understanding of the region (e.g., in the analogy, we will not go straight from Columbus or even Lewis and Clark to weather satellite views of the continent with a GPS in our car as we drive). We can, however, do the best that we can with the information we have, by learning from those who have gone before, learning as we go, and making intelligent guesses about what we don't know.

And, of course, we need to do this in all of the regions we care about.

More specific introduction to source matching from the literature

Several people have done prior studies in BRC 27, BRC 34, and BRC 38 before, but they have NOT found all the young stars! You worked hard to find all of these articles in the context of the proposal. Certainly investigator A working in BRC 27 in year X saw some of the same sources as investigator B working in that same region in year Y, as did investigator C in year Z. Now we actually have to do the work of figuring out which specific sources are which in all the papers - are the sources called out in paper 1 the same or different sources as paper 2?, etc., until all the papers are exhausted, and we have a complete catalog of all the previously studied sources in the region.

The thing that makes this complicated is that, even though everyone is reporting in RA and Dec, not everyone is using exactly the same system (some are 1950 coordinates and some are 2000 coordinate), and not everyone has the same coordinate accuracies (some are working off of photographic plates, and some are working off large-format CCDs). And, what does it mean to have "the same" coordinates -- is within an arcsecond ok? 5 arcseconds? an arcminute? This is where it gets tricky, and where you have to apply your brain! Spitzer, WISE, and 2MASS are all using exactly the same, high-accuracy coordinate system -- it's all tied to 2MASS's J2000 coordinates -- but even then the position of the same object will not be EXACTLY the same in each image, in each catalog, because there is a limit to the precision with which we can identify the coordinates. Where possible, we need to update the old coordinates by comparing what the old papers say to the 2MASS data. Then we need to fold in the objects with newer coordinates into our collection of sources.

Part of the challenge here is bookkeeping -- writing down coordinates correctly, keeping track of which sources are which, and getting the correct data matched to the correct source.

Last year, I thought this would be a relatively simple project that could be done before the summer visit. However, it turned out not to be the case. Part of this is, I think, the complexity of the region, BRC 27 in particular (which is of course still a problem this year), and part of it was I think my ability to explain it! I've pulled out and updated all of my best(?) explanations and descriptions here. IF IT DOESN'T MAKE SENSE, PLEASE ASK QUESTIONS. If this is done wrong, or only halfway done, it will make for a LOT of pain downstream. Trust me.

1950 vs. 2000 Coordinates and Yet Another Analogy

Review the differences between the 1950 and 2000 coordinate systems. REMINDER: THE STARS ARE NOT MOVING. (Several people last year kept saying things like "wow the stars move a lot" and if you say that, I WILL correct you!) The coordinate system is moving, not the stars. (Well, technically the stars may really be moving, but we don't have that information, and the size of any such motion will be on the order of a tenth of an arcsec or less; the change due to the change in coordinate systems is much greater.)

An analogy can provide an example. Through Google Maps, I can see that there is a Baskin Robbins near Ms. Linahan's school (from the NITARP 2011 class). I can give you the position of that Baskin Robbins in any of a number of ways:

  • 346 North Lake St, Mundelein, IL
  • 8 long blocks roughly west of the school
  • 4 minutes west of the school (if you drive)
  • 20 minutes west of the school (if you walk)
  • latitude 42.269711, longitude -88.004308

Or, I could be really pathological and/or vague and say:

  • the 3rd oldest Baskin Robbins in Mundelein (NB: i'm just making this up)
  • the 5th oldest ice cream store in Munelein (NB: i'm just making this up)
  • An ice cream store on Lake, south of Loch Lomond

Are the coordinates different in these systems? Yes, but they are pointing to the same physical location. Is the Baskin Robbins really moving? No.

That is what is going on in these BRCs. The most obvious difference is between B1950 coordinates and the J2000 coordinates -- these are different coordinate systems, and we are trying to translate one into the other, but we are also trying to understand the intrinsic errors of the earlier studies, and figure out which object they were 'really' talking about. In our example above, we want to translate each of the bullets in the above into "latitude 42.269711, longitude -88.004308". The added complication (in the BRCs and in the list above) is that some of the previous authors were not working in particularly precise coordinate systems. Given the list above, your steps in finding the 'true location' of this store would be slightly different for each bullet, and in the end, might include identifing all the ice cream stores on Lake, south of Loch Lomond, figuring out how old the ice cream stores are in Mundelein, figure out how far you could get in radius in a 4min drive from the school, figure out how far you could get in radius in a 20 min walk from school, etc. In this case, you know that all of these pointers are trying to indicate the same physical location, so you could then look at the commonalities in all of those results, and then, eventually, assert with confidence that all of them point to lat/long (42.269711, -88.004308). In the case of the BRCs, we don't know for sure that each object in each paper actually does have a match. Sometimes they don't. Sometimes there really are two objects close to each other, not just one.

So all of this is what we have to do with the BRCs. We have lots of lists of objects, all in slightly different systems and coming from different ways of searching for young stars. We have to apply our brain and figure out which objects each paper is talking about -- which ones are new objects and which ones are the same as objects previously identified.

Venn Diagrams and Bookeeping

One of the difficulties we will have during this project is keeping all the source lists straight. It happens every year, and I don't know how to make it easier, except for warning you that it will happen! Here is a Venn diagram explaining, roughly, the various source lists we will have before we are done, at minimum. This Venn diagram is meant to be a "big picture" sort of thing; this page on the source matching is meant to address just, well, the previously identified sources.

Brcvenn1.png

The source lists include:

  • All "bright enough" sources seen in the WISE maps (a conceptual list only)
  • Sources in the WISE catalog of photometry (to which we will add photometry from 2MASS, Haleakala, and Spitzer in the places where we have that data)
  • Sources in this general direction studied by anyone else, ever (the majority of those reported are also YSOs, but not all of them) -- this is the list we are trying to assemble here.

Out of those sets, our ultimate scientific goals mean that we are striving to identify:

  • YSO candidates we select from IR excess
  • YSOs that others identify that do not appear to have an IR excess.

The Venn diagram is even trying to correctly represent the relative sizes of the circles in that "all bright enough sources" ought to be darn close to "sources in the catalog" and that there will be some "sources in this general direction..." not in the regions we care about, and some of those sources that do not have IR excesses.

NOW we are going to work on the list of "Sources in this general direction studied by anyone else, ever" for BRC27.


Brcvenn2.png

For this diagram, I tried to spatially represent the concepts behind what we're doing now (on this page), but I admit the circles are not as carefully constructed/laid out as the first one!

Each of the 5 papers studying things in the region of BRC27 (Ogura et al 2002, Gregorio Hetem et al. 2009, Chauhan et al 2009, Shevchenko et al 1999 and Wiramihardja et al 1986) looked in the direction of BRC27. Surely, then, they saw some of the same sources as each other, and as what we are seeing. For example, the Gregorio-Hetem survey covered a HUGE area, and we care only about a part of it. Ogura saw some of the same sources that Gregorio-Hetem did, but not all of them -- they were not covering the same areas, but even within the same area, they did not see the same sources, because one survey was X-ray driven, and one was Halpha driven. They *will* see different sources, not only because they're looking at different wavelengths, but also because each survey is not infinitely deep -- the sensitivity of the surveys is limited, and as such will not see every source in this direction. Same for each other pair of papers, and our survey.

This is hardest for BRC 27 because it has the most previous work. This is easiest for BRC 34 because there have been so few papers done on this region.

The Goal

The goal here is to construct a list that is as clean as possible for each of the objects that these other folks studied, identifying which objects are truly the same between surveys, and identifying which of these objects are ones that those authors thought were actually young stars (as opposed to, e.g., background giants). We also want to carry along each of the relevant bits of information that these other authors provided -- the object is a lot easier to identify as clearly a young object or a contaminant if there is optical data, so if the other authors reported any optical measurements, we should keep track of those and tie them to the correct object in our analysis. We should also make note of any spectral types or other relevant information. The aim of this part of the project is thus:

  • Which objects from paper x are also seen in paper y?

and then, the next step we will take is

  • Which of these objects are seen in the WISE data?

The Challenges

This would be an easy task if:

  • everyone provided their original images, either as a figure or as a fits file
  • everyone worked in the same coordinate system, by which i mean not just "J2000" vs "B1950" but "J2000 tied to 2MASS" as opposed to "J2000 tied to the pulsars seen by NRAO" or "J2000 as calibrated as best I can based on the HST Guide Stars I happen to see in my image".
  • the objects were all greater than 5 arcseconds apart from each other on the sky, such that each source that is detected was cleanly and uniquely detected in each survey.
  • and, of course, if we were guaranteed a match between surveys.

Working backwards up that list...

We've already talked above about how we are not guaranteed a match between surveys, because stars are different brightnesses at different bands, and because the surveys have limited sensitivity.

There are plenty of sources that are very close together. Even among just the YSO candidates, some are very close to each other, closer than 5 arcseconds.

If we had fabulous coordinates for everything, we could let the computer match them all up and not worry about it. But we don't. And it's not just a matter of precessing the B1950 ones to J2000 ones either - there are inherent errors in those old coordinates which will not necessarily smoothly map into a clean match to other objects.

If we had images, we could line them up by eye and identify the same objects in each frame. I don't necessarily mean "line them up in ds9" (which would be the ideal case). But also, you can identify the objects simply by comparison between images they publish and images to which you have access (IRAC, 2MASS, POSS). This is what we are going to have to retreat to, in the tough cases.

Here are some notes on the 5 BRC 27 papers, in no particular order:

  • Gregorio-Hetem - finding charts for a few complex fields, which may or may not be in our region. J2000 coordinates tied to 2mass, though, so less worried about these. note that their discussion includes this very conceptual problem -- they are trying to tie X-ray sources to optical sources.
  • Wiramihardja - no finding charts; a few very coarse-scale ones, one with just YSOs. Not a lot of help here. Also just 1950 coordinates read off a photographic plate. Likely to be large and unsystematic errors.
  • Shevchenko - finding chart provided (fig 1). 1950 coordinates. big field (bigger than the region we care about).
  • Ogura - finding chart provided; J2000 coordinates but not necessarily tied to 2MASS.
  • Chauhan - no real finding charts provided; there are a few coarse-scale ones. J2000 coordinates tied to 2mass, though, so less worried about these.

The mechanics of what we need to do

Links of interest:

For each of these papers, we need a machine-readable (read as "plain text file that the computer can parse into individual numbers rather than images of numbers or gobbledegook from microsoft") version of the relevant data tables. This was either:

  • obtained from the journal itself, in which case the data table is much longer than we need
  • obtained by typing in the coordinates of the objects in our fields from these older papers and then getting updated coordinates.

Here is a video I made for last year's team on getting updated coordinates. This approach SHOULD work in MOST cases but did not work in every case; more on this momentarily. What we need to do is :

  • type the 1950 coordinates into the 2mass point source archive, making sure that it knows that it is 1950 coordinates; or type in the 2000 coordinates (and it will assume that they are 2000 coordinates).
  • look at what comes back, and take the closest *bright* object.

Then we will have 5 lists of UPDATED, HIGH QUALITY coordinates, one per paper, and we can let the computer run through the list, finding the matches between papers. We then can generate one file that purports to have one line per literature object, with all the relevant data on that line. The difficulty comes in that inevitably, a few sources during this process end up tied to the same object, or identified in other ways as duplicates or incorrect matches.

The approach above to get updated coordinates for targets works ON THE ASSUMPTION THAT THERE ARE NOT "TOO MANY" SOURCES NEARBY, AND THAT THE PRIOR SURVEYS, BEING SHALLOW, ARE MOST LIKELY TO MATCH TO THE BRIGHT 2MASS SOURCE. As I say, this should work in MOST cases but not ALL of them.

If the original authors are reporting more than one source within one paper, we should not consolidate them into one unless we are REALLY sure that the original authors were wrong. (This does happen; see my North American Nebula paper for examples.) Within each paper, this is what I would do:

  • Go do the 2MASS archive search (Go here: http://irsa.ipac.caltech.edu/applications/Gator/ -- pick 2MASS then on the next page pick 2mass point source catalog (PSC), then use that search page, making sure to specify which coordinate system in which you are working. Use the coordinates from the original paper to avoid transcription errors, as opposed to any subsequent notes or xls files. Be VERY careful to copy the coordinates exactly; that's the most common problem.)
  • Look at the sources returned. Is there just one within 5-10 arcsec? that's probably your match; go on to the next source. If there is more than one, is there just one BRIGHT one within 5-10 arcsec? Is there another source from that same paper within 5-10 arcsec? If the answer to either of those questions is yes, go on to the next step.
  • Go get the images in another window. Compare the images (if provided) from the original papers. We can use Finder Chart (http://irsa.ipac.caltech.edu/applications/FinderChart/) to get DSS and 2MASS images of the region where there is a confusing match. Compare this to the images provided in the papers. Identify which object is the one(s) in the paper.
  • With that information, then go back to your PSC, and then use the 2MASS point source catalog to find the actual high-precision coordinates of that specific dot you have identified in the images as the match to the literature object.

UPDATE 31 May 12: while the OUTPUT of this process has to be in plain ASCII text, you can work in Excel or Google Docs and spit it out as plain text as the last step. Here is a video on getting plain text (including IPAC table format) files into Excel.

Here is a list of all the papers we have assembled as part of the lit review, as well as a list of their coordinate list status. First priority items (e.g. those for you to tackle first in May/June) are in bold in the last column:

paper notes coordinate list status merging status and pending items
Getman et al 2007 X-rays. BRC 38 only. coordinates likely just fine. got all tables from journal. LMR has merged t1,t2,t3 together. need to merge it to rest of catalogs.
does it really have all of Nisini et al. (2001) already in t3? - YES, Getman Table 3 contains K, 2MASS and Spitzer data for all 20 Embedded Young objects from Nisini Table 2
Getman Tables 1 and 2 contain X-Ray and MIR data for Nisini stars 2, 3, 10, 11
--Peggy Piper 20:56, 10 June 2012 (PDT)

NOW MERGED INTO BIG ARRAY-7/3/12 --LMR NB: all sources detected are tabulated; the globule sources are the only ones clearly asserted as likely young, so i've just tagged those as YSOs.
Beltran et al. 2009 NIR. 38 only. coordinates likely just fine. got all tables from journal. (link) need to merge relevant bits of tables together. individual tables have been merged together. still need to merge that to rest of catalogs. NOW MERGED INTO BIG ARRAY-7/3/12 --LMR. NB: all sources detected are tabulated! text talks a lot about how there are YSOs there, but THERE IS NO TABLE OF JUST YSOs. Sigh. Merged photometry in but cannot of course tag the ones they think are young.
Choudhury et al. 2010 MIR. 38 only. coordinates likely just fine. got all tables from journal. (link) need to merge relevant bits of tables together tables from paper have been merged together. still need to merge that to rest of catalogs. NOW MERGED INTO BIG ARRAY-7/3/12 --LMR. NB: not entirely clear if all objects are being asserted as YSOs or not. Includes stuff in later table with a spectral type for an object that I can't find in Ogura (Ogura-BRC38-17). Merged photometry in but cannot of course tag the ones they think are young.
Chauhan et al. 2009 NIR, MIR (IRAC only). 27 and 38. coordinates ... odd. look at carefully. got all tables from journal. (link) need to merge their tables together (will take some care! - do this if the rest of the bold stuff here is done and you're chomping at the bit for more), then merge that to rest of catalogs.
Chauhan table 6 has been merged with table 3. Sources within .13 seconds for right ascension and 2 arcseconds for declination were considered matches (2 asec = ~.13 sec). All 2MASS table 3 to ra/dec table 6 were much smaller than this. Three 2MASS table 3 to ra/dec table 3 had slightly larger mismatch (0.15 and 0.17 sec mismatch). Also checked 2MASS names against Ogura list and all are correct. Origin/meaning of non matching? Megeath or Allen et al. 2004? Media:ChauhanTableMerged1.xls Media:ChauhanTableMerged1.txt NOW MERGED INTO BIG ARRAY-7/3/12 --LMR. NB: just YSOs tabulated, though for 27 there are indeed sources that appear in the IRAC table that do not appear in the JHK table.
Barentsen et al. 2011 iprime,rprime, halpha. BRC 34, 38. got all tables from journal. LMR has already merged it to rest of catalogs. NOW MERGED INTO BIG ARRAY-7/3/12 --LMR. NB: just YSOs tabulated
Nakano et al. 2012 iprime, halpha, some AKARI. BRC 34, 38. got all tables from journal. (link) LMR has already merged it to rest of catalogs. NOW MERGED INTO BIG ARRAY-7/3/12 --LMR. NB: just YSOs tabulated
Ogura et al. 2002 BRC 27, 34, 38. turns out will probably need 26 too!! there are finding charts!!! 34 is ~done. 27 only done in center (see rebull et al 2012 below). 26, 38 not touched. link for data files as downloaded from journal COORD NEED **UPDATING** 34 is done (see below). 27 is done only in center (see below). 26, 38 are not touched. need to obtain list of objects as published, find match in 2mass, assemble list of matches betw this and 2mass. you probably will need the finding charts in the paper to sort out the matches.
Ogura list updated 20120625 (sorry, ignore the date in the name)
text File:CWAYSBRC273438Oguraids 20120614.txt
excelFile:CWAYSBRC273438Oguraids 20120614.xls
coordinates entered and matched, Peggy's groups cross checked within group, with Rebull 2012,and with Jackie's group. waiting Lauren and Jackie's groups cross check. There were no sources for BRC 26. NOW MERGED INTO BIG ARRAY-7/3/12 --LMR. NB: just YSOs tabulated
Gregorio-Hetem et al. 2009 BRC 27 only. X-rays. coordinates probably ok. have whole tables from journal (link). merged together. done in center (see below). over whole region, LMR beat against 2mass in region and took nearest within 2 arcsec. several still don't have matches. region we care about is small compared to region they worked on. COORD NEED CHECKING AND POSSIBLY UPDATING. File:Gregorio 20120514.txt here is my file of the sources which has the published RA and Dec written in two ways (both J2000) and the proposed 2mass match (done blindly by the computer). if it has an asterisk (*) in front of the name, it is in the region we care about. Is it correctly matched to the right 2mass source? if not, what should it be??
Jackie and Bob source matching.

File:SourceMatchingGregorio.xlsx File:SourceMatchingGregorio.txt NOW MERGED INTO BIG ARRAY-7/3/12 --LMR. NB: just YSOs tabulated

Shevchenko et al. 1999 BRC 27 only. antiquated coordinates. have whole table from journal. done in center (see below). over whole region, LMR beat against 2mass in region and took nearest within 5(!!) arcsec. several still don't have matches. region we care about is small compared to region they worked on. COORD NEED CHECKING AND POSSIBLY UPDATING. File:Shevchenko 20120514.txt File:Shevchenko 20120615.txt here is my file of the sources which has the published RA and Dec (B1950) and then a converted-to-J2000 version of the published coordinates, both written in two ways (degrees and hh:mm:ss, dd:mm:ss) and the proposed 2mass match (done blindly by the computer). if it has an asterisk (*) in front of the name, it is in the region we care about. Is it correctly matched to the right 2mass source? if not, what should it be??
Waiting Debbie and Lauren source matching.
File:Shevchenko 2012 06 27.xlsx
The Shevchenko data had some very weird anomalies that bear further scrutiny by Luisa. We concluded that maybe some numbers had been swapped on source #99, and there are several multiple sources that were listed as one source. NOW MERGED INTO BIG ARRAY-7/3/12 --LMR. Note also that there were several problems here, most likely because of version confusion, e.g., starting with 5/14 source list above, rather than 6/15 version. please be careful with catalog versions. File:Shevchenko 2012 06 27 lmr.xlsx is my updated version of your xls. NB: just YSOs tabulated
Wiramihardja et al. 1986 BRC 27 only. antiquated coordinates. have nothing from journal (it's too old). need to merge it to rest of catalogs.
COORD NEED **UPDATING** done in center (see below). need to obtain list of objects as published, find match in 2mass, assemble list of matches betw this and 2mass. NB: region we care about is probably smaller than region they published. no need to find matches for objects outside of our region but may be easier to just blow through and do all of them?
DONE coordinates entered, matched and data entered, Bob and Peggy's groups cross checked.
textFile:CWAYSBRC27Wiramihardjaidsdata 20120614.txt excelFile:CWAYSBRC27Wiramihardjaidsdata 20120614.xls NOW MERGED INTO BIG ARRAY-7/3/12 --LMR. NB: just YSOs tabulated
Rebull et al. 2012 BRC 27, 34 only. coordinates and crossmatches to literature should be good. I hope!! have tables, of course! should hopefully be ok! File:Rebull2012t1brcxids 20120514.txt -- file of all literature information, crossids. note that technically this is in latex format, so "&" divides columns and \nodata means, well, "no data". some people provided errors, some didn't, so that is reflected in this table. last object listed for BRC 27 is actually off the edge of the IRAC 4-band region, and no longer appears in the table. The cross-ids should actually be ok, though. File:Rebull2012t2 20120518.txt -- file of new measurements discussed in Rebull et al. 2012. Note that there are both limits and errors in this table. NOW MERGED INTO BIG ARRAY-7/3/12 --LMR. NB: just YSOs tabulated in the PAPER, but because I have access to the full catalogs we worked with, I've merged the full catalogs into our full catalogs.

Last (next) steps

Once we have updated high-precision lists of coordinates from each paper, we can merge them together, and come up with a new-and-improved list of all the previously identified objects in this region in one place, one line per object with all the relevant information, in a plain text file.

Then we will compare this list to our WISE catalog. There will be some with IR excesses, and some without IR excesses. I expect that we will probably detect them all, but there might be some we do not detect.

Then we will also compare this list to where there is Spitzer data, and obtain Spitzer data for those sources where we can.

Then we will also compare this list to the optical data, where possible. There WILL be some we WILL NOT DETECT in the optical data.