A few more things to do:
- Edit cur_cmf/com/aliases_for_cmfgen.sh to show where cur_cmf and atomic are locted.
- Put the following in your .tcshrc:
- source /Volumes/strax/cmfgen/cur_cmf/com/aliases_for_cmfgen.sh
- setenv PGPLOT_DEV /xserve
- setenv PGPLOT_FONT /users/massey/pgplotm1/pgplot/pgplot/grfont.dat setenv NCPUS 4 limit stacksize unlimited
- When you go to plot a spectrum make sure that you have the /pgplotm1/pgplot/pgplot/pgxwin_server running.
Miscellanous
- Dealing with lo-gateway:
- To ssh or sftp TO lo-gateway use sftp -oHostKeyAlgorithms=+ssh-rsa lo-gateway
- To ssh, sftp, scp, or rsync FROM lo-gateway, use -oHostKeyAlorithms= ecdsa-sha2-nistp256
- Yes, you CAN put this in your .ssh/config file. My config file on lo-gateway now says: Hosts * and then on an indented line: HostKeyAlgorithms ecdsa-sha2-nistp256
- quarantined tag "@" in an ls -l ? Do a xattr -d com.apple.quarantine filename. If that doesn't work, try a "-c" instead of a -d
- A very useful trick to see what's being loaded in python
- DYLD_PRINT_LIBRARIES=1 pyraf [say]
- With XCode 10 the "include" folder is gone so gcc doesn't work. You need to do an "open /Library/Developer/CommandLineTools/Packages/macOS_SDK_headers_for_macOS_10.14.pkg"
- awk '{print substr($1,1,13)}' filename > somethingelse Will print the first 13 characters of the first field.
- awk '{print $4 " " $46 " " $47 }' test.tab > RBC.txt will print the 4th, 46th, and 47th word with spaces in between them!
- Set up public/private key to eliminate need for password:
- On the local machine do a ssh-keygen. Hit a CR for the passphrase
- Copy the key to the remote machine: ssh-copy-id -i /users/massey/.ssh/id_rsa observer@dct-obs1 [enter password]
- Test by doing an ssh observer@dct-obs1. Hopefully you won't have to enter the password every again.
- To set up a cron job for a remote backup:
- Go to the disk you want to copy to, say /volumes/Gandalf
- Create a file "rsync_script" that looks like this:
- #!/bin/sh
- rsync -avz observer@dct-obs1:/lmi/2017* /volumes/Gandalf/.
- Now create the cron job:
- crontab -e
- Insert: 30 13 * * * /Volumes/Gandalf/rsync_script
- The stuff at the beginning is Min (0-59), Hour (0-23), Day of month (1-31), Month (1-12), Day of Week (0-7).
- Should be good to go! If it doesn't work, try a "set nonomatch"
- rsync -avP obs1@llama.lco.cl:/VOLUMES/DATA_BAADE/MAGE/ut221001_02 .
- cp -av preserves links
- Terminal line profiler: /usr/sbin/system_profiler SPHardwareDataType
- Which version of XCODE: /Developer/usr/bin/xcodebuild -version
- open -a "Dropbox.app"
- In compiling John Thorstensen old skycalc.c code under Mojave, it complained it couldn't find stdio.h. I already had command-line-tools installed, but I needed to do a open /Library/Developer/CommandLineTools/Packages/macOS_SDK_headers_for_macOS_10.14.pkg. Once I did that, a gcc skycalc.c -o skycalc -Im worked with a few warning messages.
FIRE
Rob Simcoe (MIT) provides some excellent reduction software for his amazing Magellan instrument FIRE. I did find some of the installation and usage instructions a bit confusing. The following instructions were constructed with some help from Jane Rigby.
- To access the manual one needs an account on the Wiki. Follow the instructions on how to get an account, and then email Rob as instructed. One delay was due to the fact that he can't actually add you until you've tried unsuccessfully to log in once.
- Download the relevant software: This is revised Jan 1, 2021
- mkdir idlstuff
- cd idlstuff
- svn co https://svn.sdss.org/public/repo/sdss/idlutils/trunk/ idlutils
- svn co https://svn.sdss.org/public/repo/eboss/idlspec2d/ idlspec2d
- cd idlspec2d ; mv trunk/* . ; cd ..
- git clone https://github.com/profxj/xidl.git
- svn export svn://pele.mit.edu//FIRE FIREHOSE
- svn export svn://pele.mit.edu/firehose_test Test
- Define numerous environmental variables:
- setenv IDL_LOCAL /users/massey/idlstuff
- setenv IDL_DIR /Applications/exelis/idl
- setenv IDLUTILS_DIR $IDL_LOCAL/idlutils
- setenv XIDL_DIR $IDL_LOCAL/xidl
- setenv IDLSPEC2D_DIR $IDL_LOCAL/idlspec2d
- setenv FIRE_DIR $IDL_LOCAL/FIREHOSE/
- setenv IDL_PATH +/users/massey/idlstuff/FIREHOSE:+/users/massey/idlstuff/idlspec2d:+/users/massey/idlstuff/idlutils/goddard/pro:+/users/massey/idlstuff/idlutils/pro:+/users/massey/idlstuff/xidl:+/Applications/exelis/idl/lib:
- set path=($path /users/massey/idlstuff/idlutils/bin)
- Compile the software using "evilmake":
- cd $IDLUTILS_DIR
- evilmake clean
- evilmake all
- cd $IDLSPEC2D_DIR
- evilmake clean
- evilmake all
- cd $XIDL_DIR
- evilmake clean
- evilmake all
- Okay, there's a new problem with Quartz 2.7.11. You need to do the following or else you'll get a scary error message: Error: attempt to add non-widget child "dsm" to parent "idl" which supports only widgets
- sudo mv /opt/X11/lib/libXt.6.dylib{,.bak}
- sudo cp /opt/X11/lib{/flat_namespace,}/libXt.6.dylib
- Here's another fun fact: if some of your files have a * after them, do a ls -l. If you see a "@" it means they have an "attribute." This really screws up the pipeline. Get rid of those by doing a attr -c *.fits
- It also hates files that are +x. Do a chmod a-x *.fits
- Take the software for a test drive:
- cd $IDL_LOCAL/Test/redux
- NOTE: When you're doing this with your own data, you need to have created both a Raw and a Redux subdirectory. Be sure you're in the Redux subdirectory when you start up idl.
- idl
- firehose
- Setup tab
- Raw Directory: set to default and check that it shows the right path.
- Redux Director: set to default and check that it should the righ path.
- Observing catalog: Browse your way over to it.
- Trace tab
- Select one or multiple QF flats and hit "trace orders." (NOTE: I always get an error if I select multiple flats. It's supposed to work. Seems to be fixed Jan 2021.) A plot similar to this will be displayed:
- The xatv window will pop up showing the orders outlined:
- Select one or multiple QF flats and hit "trace orders." (NOTE: I always get an error if I select multiple flats. It's supposed to work. Seems to be fixed Jan 2021.) A plot similar to this will be displayed:
- Flats tab
- Select all of the QFs for the flat field images (1-10 for the test data; image 11 is incorrectly labeled). If you are combining flats from more than one night, check to see if there are shifts. You might need to put them in a separate directory so you can choose them all, and revise the setup temporarily.
- Select the sky flats for the illum flats (15+16 for the test data). NOTE: Make sure your sky flats have EXPTYPE set to "TrcFlat"
- Select an arc for the slit tilt. I know the manual says to just use a long exposure, but when I did this I had nothing but trouble. The arcs worked wonderfully.
- Select the orders mask you just created for the Orders Mask.
- Hit Make Flat Field. Go out for coffee.
- Viewer pops up, flat field w orders traced out, Hit Done.
- Some illumination function plots pop up.
- Viewer pops up with flatfield. Hit Done.
- Structure tab
- Flats tab should be populated thanks to the previous step.
- Generate structure. Give it a few seconds.
- Edit (and Save) Structure. Make things are right. If not, fix.
- For the test data: the title of fire_0011.fits is "QF" but it's actually an arc. Even more confusing is that it is an arc that crashes the program in the next step. Change its type to "unknown."
- Hit Done.
- NOTE: I you need to redo some subsequent steps you can do a "Load Structure"
- Extract tab
- Highlight the object(s) you want.
- You can do this the easy way or the hard way....
- Easy way: In Preferences, keep the Object/Aperture Finding automated and hit "Run Pipeline." Don't use the boxcar; use the full blown.
- Hard way: In Preferences, set the Object/Aperture to Interactive/User Fits.The following is parapharased from Jane Rigby:
Read the manual carefully to learn how to best extract your targets. Generally there are three cases: (a) Very strong continuum source, which allows you to (in principle) use a boxcar extraction. This runs fast, but produces inferior sky subtraction. (b) Fairly strong continuum source. This is great, as the GUI will do most of the work for you. All you have to do, when the viewer pops up, is find the continuum ridge line in any order, mouse over it, hit space bar, hit O (capital oh), and then hit Done. The GUI will find the trace. (c) Objects with strong emission lines but weak to non-existent continuum. This is the hardest case, but there is a specific options who allows one to fit the spatial profile from a bright emission line, and then use optimal extraction using that profile, adopting the trace from of the telluric star. [Phil: to me, this sounds similar to IRAF's aperture extraction using another star as the reference. I'm surprised that it doesn't just adopt the profile from the standard as well.] Here's how to deal with "c."
- Identify a bright emission line when the view pops up.
- Change MouseMode to Zoom, and click the line to zoom in on it.
- The panner box at upper right works like ds9’s, if that’s more comfortable.
- Put the cursor just below the emission line and hit "B." Then put the cursor right above the emission line and hit "T."
- A gui comes up and fits a bspline:
- Note that there's a help menu at top of gui.
- You can change the fitting function via the gui.
- Delete points w "D". Add data pts w left click (to set zero at left).
- "R" plots residuals, "V" plots values.
- "u" increases order of fit, "d” decreases.
- Hit Done when satisfied. Then it does the sky sub w/o fitting the profile. There is no gui feedback yet on this part.
- Telluric Correction Tab
- The telluric correction relies upon "Xtellcor" and associated routines from the Spextool package. A complete version of the manual (written by Michael Cushing, William Vacca, and John Rayner) can be found here; see Chapter 6. I found that the instructions at the APO TripleSpecTool were a bit easier to follow; see Section 10.2. In truth, the on-line help window is quite good too.
Still, it took me about a day to figure out how the damn cursor works. You know how Doug Tody eventually implemented cursor readback in IRAF about 30 years ago? In the subsequent decades many of us got used to the idea that if the instructions tell you to hit an "n," you put the cursor where you want and hit an "n" and the software reads the x and y position as well as the "n" and acts accordingly. This is not the philosophy of Spextool, apparently. Instead, what the manual actually means is that you hit an "n" and that after that you left-click the mouse at the position you want, and that you are simply in "n" mode until you hit another key, or make a mistake. It takes a bit of getting used to, but so did the transition from thumbwheels on Tektronix 4010. Sigh.
So, here's what I did.
- Expand the folder(s) for the "targets to calibrate tellurics." Click on the first one in the tree.
- Correct Tellurics. A little "fire_telluric" window should pop up allowing you to select the telluric standard obervations relevant to this exposure. Select them and then click "continue." I don't know what it would do if there were two different stars.
- A very large plot will pop up (in "x_splot") showing you all of the orders. Presumably this is so you can identify the fact that Order 26 around 1.00 micron contains the strong Pa Delta line (n=7 to 3) at 1.005 microns. Got it. Hit "q."
- The fire_xtellcor window will pop up identifying the telluric observations for this target. It should be populated with the file names of the telluric (standard) star observations, the magnitudes of the telluric standard, and the name of the object spectrum. If the magnitudes aren't there, look them up and enter them.
- The main thing that is to come up with a proper convolution kernal between the A0V star telluric observation you made and some standard Vega model. In box 2 make sure the radio button for "Deconvolution" is selected, and that Order 26 is selected, as this is what contains the Pa Delta line.
- BEFORE YOU DO ANYTHING ELSE, HIT THE "Help" button on the bottom of the gui because this will become grayed out once you actually need it. (Doesn't anyone screen these calls?)
- Hit "Construct Kernel."
- You will be confronted wih what is apparently a simplified version of the Xconkern Spextools window. You'll see the spectrum of the telluric standard in white exending from about 0.972 to 1.034 microns with the big whopping Pa Delta line pretty much in the middle.
- Click on "n" to inform the software that you are planning to mark the continuum regions in order to fit the continuum. It's a good question how close to the line you should mark. The example shown in the APO guide is silly; don't do that! I did okay by marking 0.973 to about 0.996 on the left side and 1.016 to 1.034 on the right and using an 11th order. Recently I've been using "n" to define two regions on the left and two on the right. The normalization matters a lot!
- Now you have to click "Normalized spectrum" and select the wavelength region that contains "just" the line. You might find it useful to use the zoom feature: hit "z" and then click twice with the left-mouse to outline the regions you want. Or you can try "x". w will unzoom.
- You must now "select" using the s key: hit s, and then mark the left and
the right side of the spectral line using left-clicks on the mouse.
- Now hit "Construct kernel." The holy grail here (according to the APO guide) is maximum deviations less than 1.5% ad an RM deviation less than 0.7%.
Using the sample data I could not come close to achieving this. I did manage to achieve errors that were about twice that. If you don't get anything you like, go back and redo the normalization. Hit "accept."
- Okay, we're back to thefire_xtellcor. Hit Scale lines. You'll be happier with just hitting "accept." If the spectrum is missing, you neglected to add the B and V magnitudes if they were missing. Mess around with "e" and dragging points with all of the orders until you're happy. Hit "accept." My experience is that this is too obscure to mess wit
- Decide what units you want (such as ers/sec/cm^2/A) and hit "construct telluric spectra"
- Determine Shift. So, this turns out to matter: Select Aperture 01 and Order 28. Do an "s" and then click on the left and then the right of that region with a whole bunch of telluric stuff. Hit find auto. Don't be surprised to get a value like -1.50. Hit accept, and then Apply to All orders.
- Write File.
- Repeat for the other 3 program (or 7) program images. Each time select all 4 of the telluric observations. The nice thing is that a much smaller window will pop up. You can just hit "Load Spectra", "Apply to All orders," and "Correct spectrum."
- Do a Combine. Make sure it was the correct number (4 or 8, usually).
- The telluric correction relies upon "Xtellcor" and associated routines from the Spextool package. A complete version of the manual (written by Michael Cushing, William Vacca, and John Rayner) can be found here; see Chapter 6. I found that the instructions at the APO TripleSpecTool were a bit easier to follow; see Section 10.2. In truth, the on-line help window is quite good too.
- Setup tab
Running Larry's astrometry code
The Buie/Wasserman IDL package produces excellent results on LMI images, but the naming conventions are idiocentric.- Use apphot's "center" or psf fitting to produce a file of accurate x and y positions.
- Hack on m31makeobj.f So it produces:
- fort.9 which contains an imcopy of the image to a date-based image name.
- a "Mask*.obj" file that contains some magic numbers and a list of 0-indexed x and ys.
- Run m31makeobj
- cl < fort.9
- copy lmi.key one level above your working directory
- copy astrom.inf into your working directory
- copy dofind.pro to your working directory and edit:
- Update the root name
- Update the directory name to your working directory ending with a "/"
- idl
- dofind,1,5 if you have rootname.001, .002, .003. If it doesn't work you probably have screwed up the directory name.
- mkdir SRC and mv *.src SRC
- edit doast.pro to include the right root name and directory name.
- doast,1,5
- Results are in *.ast. Other fun files:
- fitcoeff.dat
- Refstars/image
- centers.dat
GMOS Software
Designing a slitlet mask
- Download a DDS image. Example m33.fits[0]
- Mess with ds9 and region files deciding that where it makes sense to put masks, and approximately angle. Keep in mind parallactic angles.
- Make list of objects and potential alignment stars (MosA.in) including priorities.
- When you're close, make a smaller version of the dss image, i.e., dssE.fits[0].
- Bring up the gemini IRAF. Two ways currently:
- Ureka
- ur_setup
- cl
- anacoda [installation instructions below]
- bash
- source .bash_profile
- conda activate iraf27a
- cd iraf
- cl
- Ureka
- gmskcreate
- Note: the naming convention is stupidly important. "GMI" images are what gets used by GMMPS.
- indata="MosA19.in"
- gprgid="GN-2020B-Q-237"
- instrument="gmos-n"
- rafield=1.55138888889 [can enter as HH:MM:SS.SSS]
- decfield=30.527777778 [can enter as +DD:MM:SS.SS]
- pa=340
- fl_getim=yes
- inimage="dssA.fits[0]"
- outimage="GN2020BQ237-01"
- fl_getxy=yes
- outcoords="GN2020BQ237-01.coo" (not used by anything)
- outtab="GMIGN2018BQ228-01_OT.fits"
- iraunits="hours"
- fraunits="hours"
- slitszx=1
- slitszy=5
- verbose=yes
- gmmps: Make sure you have the latest version (in 2020 I used 1.5.2). I could not compile the code, so I used the binary versions.
- gmmps
- File-->open-->GMIGN2020B237-01.fits
- GMMPS-->Load Object Table-->GMIGN2020BQ237-01_OT.fits
- Do any editing. For instance, I got rid of 501 and 599 and 598 was supposed to be better.
- Configure mask
- Pick grating, no filter, and reasonable CWL
- No to pack spectra
- min sep 4 pixels
- wiggle 15%
- Make mask (or whatever). See if everyone got on.
- Now go through and try to make the slits longer by changing 5-->10. Update slit. After each change do a new Make mask.
- Note: GMMPS isn't good at counting. The number of 0/1/2/3
- GMMPS Load ODF. Check that 4686 is clear. Make sure that everything looks groovy.
- Rename the out GMIGN2020BQ237-01_OTODF1.fits files to GN2020BQ237-01_ODF.fits without the GMI at the start, or th OT in the middle or the 1 at the end.
- Don't try to load a .gz file. You can, but as soon as you go to load an OT or ODF very bad things happen.