View source for Weeding down a big target list

Here is a case study on how to weed down a big list of potential targets using semi-automatic searches.
This is exactly how I weeded down this list of Lynds Dark Nebula.

=1. get master LDN list.=

this link on wiki: http://adsabs.harvard.edu/abs/1996yCat.7007....0L
go to Online Data: http://vizier.cfa.harvard.edu/viz-bin/VizieR?-source=VII/7A
ask it to give us plain text ("| separated values"), all objects, ldn number, ra and dec
(1950 and 2000), glon and glat (galactic lat and long), area, and
opacity class, plus any other columns you want.  

take plain text and import into excel, such that columns import
cleanly.  i had to rename the file to something.txt and then ask
excel to import it, with "|" delimiting the columns.
you have then i think 1794 objects.

=2. get rid of the lower opacity clouds.=

i think it shows up already sorted by ldn number.
sort by opacity class, reverse numerical order.
move everything other than opacity class 5 or 6 down to the
bottom of the table.  keep the opacity class 5 or 6 near the top.
i entered some blank lines which i colored bright red to remind
myself easily where the border was between opacity classes.
this gets rid of 1247 of the objects.

=3. get rid of the ones i know are covered by spitzer.=

i happen to know there are two big surveys using Spitzer, called
GLIMPSE and MIPSGAL. both are surveying the galactic plane, the
former using IRAC and the latter using MIPS.  The GLIMPSE survey
is older than MIPSGAL. their website is here:
http://www.astro.wisc.edu/sirtf/
here, you can learn that they surveyed the inner 65 degrees (of
longitude) of the galaxy, plus or minus 1 degree of latitude.

sort the lynds target list by ldns number.  this ends up being
rougly galactic long sorted.

what i did next was ask xls to do conditional formatting, and
(a) color the cells in the galactic longitude column to be light
blue if the value was less than 65 or greater than 295 (and less
than 360). and (b)  color the cells in the galactic longitude
column to be light blue if the value was greater than -1 and
less than 1.  anything that is then blue in both the glat and
glon cells is an object that appears in mipsgal and glimpse.
this gets rid of just 28 of the objects; i was expecting it to
get rid of many more.
i entered some blank lines which i colored bright red to remind
myself easily where the border was between glimpse and
non-glimpse coverage.

i have now 518 objects left in the pool of potential targets.
sort this by ldn number.
extract the ldn number to its own spreadsheet and use xls to
create a text file of just ldn numbers for these 518 objects.
in order to do this, i make a separate worksheet, copy the ldn
numbers to column b, in column a, make the first cell "LDN" and
then "fill down" to fill out the rest of the column with all
"LDN".  i make column A right-adjusted, column B left-adjusted,
and save this worksheet to a text file.

=4. get number of references from simbad.=

go to simbad
http://simbad.u-strasbg.fr/simbad/sim-fid
tell it ("output options") that you want plain text output.
query by list (second down on that page). pick the plain text list of targets
file you just created.
it will take several sections to come back.  save the file of results as a
plain text file.

import this results text file back into xls, as a new worksheet.
copy the column with the number of bibliographic references into
a new column in the master spreadsheet. if it doesn't match the
number of rows you had from before, you did something wrong, and
you should go back and check things.

=5. get hits in spitzer archive.=

take the input file you just made for simbad, reformat it so that
each line is surrounded by quotes, e.g. "ldn	4   "
rather than just ldn   4 without the quotes
and break it into pieces of 100 targets per file (you'll have 6 files).
The top of each of these files should look like this:

 COORD_SYSTEM: Equatorial # Equatorial (default), Galactic, or Ecliptic
 EQUINOX: J2000 # B1950, J2000 (default), or blank for Galactic
 NAME-RESOLVER: Simbad # NED or Simbad (default)
 RADIUS: 5 # RADIUS is in unit arcmin
 #Name

start leopard.
query by fixed target list, load the first of your files you just
created.
this will take a while for each leopard search file. when it is
done searching, it will ask if you want to load the AORs for each
program it found. just tell it "cancel" for now.
leopard automatically generates a file in the same directory as
your target list, with the same root filename but with a
time/date stamp appended.
because of a leopard bug, you need to restart leopard after each
target search.

this header tells leopard to search for things within 5 arcmin.
it's going to tell us everything (IRS, IRAC, MIPS) that it
thinks overlaps. it's not perfect, but it will get us close
enough for weeding purposes.

make a new column for these results in the master xls file.
look at the summary files, and if there is an IRAC/MIPS hit,
enter that information in the xls file.


=6. get information from lee and myers paper.=
http://adsabs.harvard.edu/abs/1999ApJS..123..233L
this is the one looking at optically-selected cores to see if
they thought there was a YSO inside.
add this to a column in the spreadsheet.


=7. sort by spitzer availability.=  
start with ones without any spitzer data (as opposed to ones with just irac or just mips).
this leaves 270 obj.


=8. make a sorted cut=
sort those by lee&myers, then number refs (decr numbers), then
area (incr number).
just 9 yeses from lee&myers.
14 without lee&myers but with >10 pubs (worth going to look at
those pubs to see if those pubs think there is something there).

-> 23 objects to go look up the pubs, look up the POSS image, see
how big/interesting they look.
LDN number:
31
129
219
425
518
543
769
778
981
1036
1041
1082
1100
1125
1139
1143
1235
1257
1340
1407
1598
1685
1744

=final product at this point=

[[media:lmrfulllynds.xls]]


=next step=
still need to add notes here re: pub checking and image checking for those objects