Mac Software Hints

1. Download and install Homebrew by:
   • bash
   • ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
   • exit
   • Do a 'brew doctor' and try to make it happy. Brew hates having macports, fink, or any librariespresent in /usr/local/bin that it doesn't already know about. This may be very childish of it, but you can hide your libraries and macports and fink by sticking them in temporary files that you can then put back when you've got astrometry.net working. Keep running 'brew doctor' and doing what it requests until you finall get a 'ready to brew' message.
2. brew install astrometry-net
3. Follow the instructions when it gets stuck on something.
4. One problem with version 72 is a conflict with Pyfits; need to do a: sudo pip install fitsio after first downloading and installing pip (see https://pip.readthedocs.io/en/stable/installing/ You'll have to do this as sudo).
5. Install the indicies.
   • Download the 2MASS indicies, the TYCHO indicies and the USNO-B1.0indicies and stick them somewhere where you have a spare 65Gb of space.
   • Edit the /usr/local/Cellar/astrometry-net/etc/astrometry.cfg file to reflect where you've put the indicies, i.e., something like this:

     # In which directories should we search for indices?
     add_path /Volumes/pepe/astromindicies/2MASS-indicies
     add_path /Volumes/pepe/astromindicies/TYCHO-indicies
     add_path /Volumes/pepe/astromindicies/USNO-indicies 
     

6. Add /usr/local/astrometry/bin to your path
7. Try it on a test image:
   • solve-field --overwrite --scale-units arcsecperpix --scale-low 0.44 --scale-high 0.46 --ra 23.33 --dec 29.27 --radius 0.5 121028-987.fits
   • solve-field --overwrite --scale-units arcsecperpix --scale-low 0.43 --scale-high 0.44 --ra 84.0 --dec -69.2 --radius 0.5 n2044test.fits
   • solve-field --overwrite --scale-units arcsecperpix --scale-low 0.23 --scale-high 0.25 --ra 179.8 --dec 12.3 --radius 0.1 lmi.0079.fits
   • solve-field --overwrite --scale-units arcsecperpix --scale-low 0.43 --scale-high 0.44 --ra 92.08 --dec -71.96 --radius 0.3 --tweak-order 1 --no-plots --use-sextractor -N LMC000iCIIt1.fits LMC000iCII.fits
   • Can add:
     • --use-sextractor
     • --no-tweak or --tweak-order 1
     • --no-remove-lines (very useful when it won't solve). Should probably be the default.

Sextractor

1. brew install brewsci/science/sextrator

CMFGEN

1. You will seriously need to buy the PGF95 compiler.
   • Hints in getting the damn license server to work:
     • You **may** be able to just get by doing a "lmreread"
     • In getting the permanet license, first do a "lmutil lmhostid" to see which MAC address the damn thing is picking up.
     • Starting up the license daemon:
       • sudo bash
       • cd /opt/bin/osx86-64/2018/bin
       • sudo cp com.pgroup.flexlm.plist /Library/LaunchDaemons
       • sudo chown root:wheel /Library/LaunchDaemons/com.pgroup.flexlm.plist
       • sudo chmod 0644 /Library/LaunchDaemons/com.pgroup.flexlm.plist
       • sudo launchctl load /Library/LaunchDaemons/com.pgroup.flexlm.plist.
       • When that fails, just do a /opt/pgi/osx86-64/18.3/bin/lmgrd
       • Recently I've been trying this and getting a message about not being able to open a port. Turns out that here was some conflict with the idl83 lmgrd. Do a ps -ax | grep lmgrd. Now...if you just kill it, it will restart. So, I did something more draconian and did a mv of the executible in the IDL directory to lmgrdold. I then rebooted. At that point the 2018 PGI version came to life, and it read my license file fine and I could compile again. Lord knows what's going to happen when I go to run IDL though...
     • Strongly recommend downloading latest xcode and trying to open it (so that you install the other stuff) before doing much else.
     • For the pgfortran compiler to work, you need to do a sudo ln -s /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/lib/crt1.o /usr/lib/crt1.o
     • For Yosemite you will also need a
       • sudo ln -s /opt/X11 /usr/X11R6
       • sudo ln -s /opt/X11 /usr/X11
     • For El Capitan/Sierra, one needs to find an older version of stdint.h and put it in /usr/include. Here is a copy This is needed for compiling pgplot.
   • In the .tcshrc put
     • setenv PGI /opt/pgi
     • setenv LM_LICENSE_FILE $PGI/license.dat
     • source /Volumes/strax/cmfgen/cur_cmf/com/aliases_for_cmfgen.sh
     • set path = ($PGI/osx8-64/15.3/bin/ $path)
     • setenv PGPLOT_DEV /xserve
     • setenv PGPLOT_DIR /Volumes/strax/pgplotorig/pgplot/pgplot (pretty sure this line does NOTHING)
     • setenv PGPLOT_FONT /Volumes/strax/pgplotorig/pgplot/pgplot/grfont.dat
     • setenv NCPUS 4
     • limit stacksize unlimited
2. One needs the "special" Mac version of pgplot. Here it is. cd to pgplot/pgplot (not pgplotsrc!), "fix" the directory names at the top in Makefile, and then do a "make clean" and a "make all". It will bomb with an error message when it tries to create the cpgdemo stuff (Undefined symbols in MAIN) but you will find to your surprise that libpgplot.a has been created! Joy!
3. Download the CMFGEN source files from the above link. Create a subdirectory cur_cmf_date and stick the stuff there. Create a symbolic link ln -s cur_cmf_date cur_cmf.
4. Download the latest and great atomic data as well from the above link.
5. Edit cmfgen/cur_cmf/Makefile_definitions so it looks something like this.
6. If you update the version of the compiler, make sure you change the library location in the Makefile_definitions.
7. NO LONGER NEEDED: Modify the Makefile definitions in "subs" and "plane" to invoke FZ rather than whatever flag it users. This gets around a bug with 14.10 that affects subs/fg_j_cmf_v9.f and plane/fg_j_cmf_v12.f.
8. edit cur_cmf/com/aliases_for_cmfgen.sh to contain the right paths
9. make clean
10. make all
11. So, you think you're all set, don't you? Well, you're not. You're going to have so start some damn server: /Volumes/strax/PGPLOTOrig/pgplot11.0/pgplot/pgxwin_server &
12. While how to use CMFGEN is way beyond the scope of this little note, Kathryn's CMFGENGuide is invaluable.

Michael's photometry routines (Thanks to NickM)

• Install homebrew:
  • bash
  • ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
  • brew doctor
• Install anaconda3:
  • https://www.anaconda.com/distribution/
  • conda update conda
  • conda config --add channels http://ssb.stsci.edu/astroconda
  • conda install iraf-all pyraf-all stsci
  • add to .bash_profile: source activate base
  • Add to .condarc file in home directory: changeps1: False
  • conda install -c astropy pyephem astroquery
  • conda install -c conda-forge uncertainties lmfit pysftp scikit-image cfitsio
  • conda install -c anaconda future pillow
• Update pip:
  • pip install --upgrade pip
• Install sextractor and scamp:
  • brew install brewsci/science/sextractor
  • brew install brewsci/science/scamp
• Install extra software:
  • brew install wget
  • brew install imagemagick
  • brew install GraphicsMagick
  • brew install inetutils
• git clone https://github.com/mommermi/photometrypipeline
• Add to .bashrc:
  • export PHOTPIPEDIR=/Users/massey/photometrypipeline
  • export PATH=/Users/massey/photometrypipeline:$PATH

Miscellanous

• Put a "setenv COPYFILE_DISABLE 1" in your .tcshrc if you want to avoid all those annoying little ._* files when you tar something.
• A very useful trick to see what's being loaded in python
  • DYLD_PRINT_LIBRARIES=1 pyraf [say]
• With XCode 10 the "include" folder is gone so gcc doesn't work. You need to do an "open /Library/Developer/CommandLineTools/Packages/macOS_SDK_headers_for_macOS_10.14.pkg"
• awk '{print substr($1,1,13)}' filename > somethingelse Will print the first 13 characters of the first field.
• awk '{print $4 " " $46 " " $47 }' test.tab > RBC.txt will print the 4th, 46th, and 47th word with spaces in between them!
• Set up public/private key to eliminate need for password:
  • On the local machine do a ssh-keygen. Hit a CR for the passphrase
  • Copy the key to the remote machine: ssh-copy-id -i ~/.ssh/id_rsa.pub observer@dct-obs1 [enter password]
  • Test by doing an ssh observer@dct-obs1. Hopefully you won't have to enter the password every again.
• To set up a cron job for a remote backup:
  • Go to the disk you want to copy to, say /volumes/Gandalf
  • Create a file "rsync_script" that looks like this:
    • #!/bin/sh
    • rsync -avz observer@dct-obs1:/lmi/2017* /volumes/Gandalf/.
  • Now create the cron job:
    • crontab -e
    • Insert: 30 13 * * * /Volumes/Gandalf/rsync_script
    • The stuff at the beginning is Min (0-59), Hour (0-23), Day of month (1-31), Month (1-12), Day of Week (0-7).
  • Should be good to go! If it doesn't work, try a "set nonomatch"
• rsync -avP obs1@llama.lco.cl:/VOLUMES/DATA_BAADE/MAGE/ut221001_02 .
• cp -av preserves links
• Terminal line profiler: /usr/sbin/system_profiler SPHardwareDataType
• Which version of XCODE: /Developer/usr/bin/xcodebuild -version
• open -a "Dropbox.app"

FIRE

Rob Simcoe (MIT) provides some excellent reduction software for his amazing Magellan instrument FIRE. I did find some of the installation and usage instructions a bit confusing. The following instructions were constructed with some help from Jane Rigby.

• To access the manual one needs an account on the Wiki. Follow the instructions on how to get an account, and then email Rob as instructed. One delay was due to the fact that he can't actually add you until you've tried unsuccessfully to log in once.
• Download the relevant software: This is revised Jan 1, 2021
  • mkdir idlstuff
  • cd idlstuff
  • svn co https://svn.sdss.org/public/repo/sdss/idlutils/trunk/ idlutils
  • svn co https://svn.sdss.org/public/repo/eboss/idlspec2d/ idlspec2d
  • cd idlspec2d ; mv trunk/* . ; cd ..
  • git clone https://github.com/profxj/xidl.git
  • svn export svn://pele.mit.edu//FIRE FIREHOSE
  • svn export svn://pele.mit.edu/firehose_test Test
• Define numerous environmental variables:
  • setenv IDL_LOCAL /users/massey/idlstuff
  • setenv IDL_DIR /Applications/exelis/idl
  • setenv IDLUTILS_DIR $IDL_LOCAL/idlutils
  • setenv XIDL_DIR $IDL_LOCAL/xidl
  • setenv IDLSPEC2D_DIR $IDL_LOCAL/idlspec2d
  • setenv FIRE_DIR $IDL_LOCAL/FIREHOSE/
  • setenv IDL_PATH +/users/massey/idlstuff/FIREHOSE:+/users/massey/idlstuff/idlspec2d:+/users/massey/idlstuff/idlutils/goddard/pro:+/users/massey/idlstuff/idlutils/pro:+/users/massey/idlstuff/xidl:+/Applications/exelis/idl/lib:
  • set path=($path /users/massey/idlstuff/idlutils/bin)
• Compile the software using "evilmake":
  • cd $IDLUTILS_DIR
  • evilmake clean
  • evilmake all
  • cd $IDLSPEC2D_DIR
  • evilmake clean
  • evilmake all
  • cd $XIDL_DIR
  • evilmake clean
  • evilmake all
• Okay, there's a new problem with Quartz 2.7.11. You need to do the following or else you'll get a scary error message: Error: attempt to add non-widget child "dsm" to parent "idl" which supports only widgets
  • sudo mv /opt/X11/lib/libXt.6.dylib{,.bak}
  • sudo cp /opt/X11/lib{/flat_namespace,}/libXt.6.dylib
• Here's another fun fact: if some of your files have a * after them, do a ls -l. If you see a "@" it means they have an "attribute." This really screws up the pipeline. Get rid of those by doing a attr -c *.fits
• It also hates files that are +x. Do a chmod a-x *.fits
• Take the software for a test drive:
• cd $IDL_LOCAL/Test/redux
• NOTE: When you're doing this with your own data, you need to have created both a Raw and a Redux subdirectory. Be sure you're in the Redux subdirectory when you start up idl.
• idl
• firehose
  • Setup tab
    • Raw Directory: set to default and check that it shows the right path.
    • Redux Director: set to default and check that it should the righ path.
    • Observing catalog: Browse your way over to it.
  • Trace tab
    • Select one or multiple QF flats and hit "trace orders." (NOTE: I always get an error if I select multiple flats. It's supposed to work. Seems to be fixed Jan 2021.) A plot similar to this will be displayed:

      

    • The xatv window will pop up showing the orders outlined:

      

  • Flats tab
    • Select all of the QFs for the flat field images (1-10 for the test data; image 11 is incorrectly labeled). If you are combining flats from more than one night, check to see if there are shifts. You might need to put them in a separate directory so you can choose them all, and revise the setup temporarily.
    • Select the sky flats for the illum flats (15+16 for the test data). NOTE: Make sure your sky flats have EXPTYPE set to "TrcFlat"
    • Select an arc for the slit tilt. I know the manual says to just use a long exposure, but when I did this I had nothing but trouble. The arcs worked wonderfully.
    • Select the orders mask you just created for the Orders Mask.
    • Hit Make Flat Field. Go out for coffee.
    • Viewer pops up, flat field w orders traced out, Hit Done.
    • Some illumination function plots pop up.
    • Viewer pops up with flatfield. Hit Done.
  • Structure tab
    • Flats tab should be populated thanks to the previous step.
    • Generate structure. Give it a few seconds.
    • Edit (and Save) Structure. Make things are right. If not, fix.
    • For the test data: the title of fire_0011.fits is "QF" but it's actually an arc. Even more confusing is that it is an arc that crashes the program in the next step. Change its type to "unknown."
    • Hit Done.
    • NOTE: I you need to redo some subsequent steps you can do a "Load Structure"
  • Extract tab
    • Highlight the object(s) you want.
    • You can do this the easy way or the hard way....
      • Easy way: In Preferences, keep the Object/Aperture Finding automated and hit "Run Pipeline." Don't use the boxcar; use the full blown.
      • Hard way: In Preferences, set the Object/Aperture to Interactive/User Fits.The following is parapharased from Jane Rigby:

        Read the manual carefully to learn how to best extract your targets. Generally there are three cases: (a) Very strong continuum source, which allows you to (in principle) use a boxcar extraction. This runs fast, but produces inferior sky subtraction. (b) Fairly strong continuum source. This is great, as the GUI will do most of the work for you. All you have to do, when the viewer pops up, is find the continuum ridge line in any order, mouse over it, hit space bar, hit O (capital oh), and then hit Done. The GUI will find the trace. (c) Objects with strong emission lines but weak to non-existent continuum. This is the hardest case, but there is a specific options who allows one to fit the spatial profile from a bright emission line, and then use optimal extraction using that profile, adopting the trace from of the telluric star. [Phil: to me, this sounds similar to IRAF's aperture extraction using another star as the reference. I'm surprised that it doesn't just adopt the profile from the standard as well.] Here's how to deal with "c."

        • Identify a bright emission line when the view pops up.
        • Change MouseMode to Zoom, and click the line to zoom in on it.
        • The panner box at upper right works like ds9’s, if that’s more comfortable.
          • Put the cursor just below the emission line and hit "B." Then put the cursor right above the emission line and hit "T."
          • A gui comes up and fits a bspline:
            • Note that there's a help menu at top of gui.
            • You can change the fitting function via the gui.
            • Delete points w "D". Add data pts w left click (to set zero at left).
            • "R" plots residuals, "V" plots values.
            • "u" increases order of fit, "d” decreases.
            • Hit Done when satisfied. Then it does the sky sub w/o fitting the profile. There is no gui feedback yet on this part.

              

  • Telluric Correction Tab
    • The telluric correction relies upon "Xtellcor" and associated routines from the Spextool package. A complete version of the manual (written by Michael Cushing, William Vacca, and John Rayner) can be found here; see Chapter 6. I found that the instructions at the APO TripleSpecTool were a bit easier to follow; see Section 10.2. In truth, the on-line help window is quite good too.

      Still, it took me about a day to figure out how the damn cursor works. You know how Doug Tody eventually implemented cursor readback in IRAF about 30 years ago? In the subsequent decades many of us got used to the idea that if the instructions tell you to hit an "n," you put the cursor where you want and hit an "n" and the software reads the x and y position as well as the "n" and acts accordingly. This is not the philosophy of Spextool, apparently. Instead, what the manual actually means is that you hit an "n" and that after that you left-click the mouse at the position you want, and that you are simply in "n" mode until you hit another key, or make a mistake. It takes a bit of getting used to, but so did the transition from thumbwheels on Tektronix 4010. Sigh.

      So, here's what I did.

      • Expand the folder(s) for the "targets to calibrate tellurics." Click on the first one in the tree.
      • Correct Tellurics. A little "fire_telluric" window should pop up allowing you to select the telluric standard obervations relevant to this exposure. Select them and then click "continue." I don't know what it would do if there were two different stars.
      • A very large plot will pop up (in "x_splot") showing you all of the orders. Presumably this is so you can identify the fact that Order 26 around 1.00 micron contains the strong Pa Delta line (n=7 to 3) at 1.005 microns. Got it. Hit "q."
      • The fire_xtellcor window will pop up identifying the telluric observations for this target. It should be populated with the file names of the telluric (standard) star observations, the magnitudes of the telluric standard, and the name of the object spectrum. If the magnitudes aren't there, look them up and enter them.
      • The main thing that is to come up with a proper convolution kernal between the A0V star telluric observation you made and some standard Vega model. In box 2 make sure the radio button for "Deconvolution" is selected, and that Order 26 is selected, as this is what contains the Pa Delta line.
      • BEFORE YOU DO ANYTHING ELSE, HIT THE "Help" button on the bottom of the gui because this will become grayed out once you actually need it. (Doesn't anyone screen these calls?)
      • Hit "Construct Kernel."
      • You will be confronted wih what is apparently a simplified version of the Xconkern Spextools window. You'll see the spectrum of the telluric standard in white exending from about 0.972 to 1.034 microns with the big whopping Pa Delta line pretty much in the middle.
      • Click on "n" to inform the software that you are planning to mark the continuum regions in order to fit the continuum. It's a good question how close to the line you should mark. The example shown in the APO guide is silly; don't do that! I did okay by marking 0.973 to about 0.996 on the left side and 1.016 to 1.034 on the right and using an 11th order. Recently I've been using "n" to define two regions on the left and two on the right. The normalization matters a lot!

        

      • Now you have to click "Normalized spectrum" and select the wavelength region that contains "just" the line. You might find it useful to use the zoom feature: hit "z" and then click twice with the left-mouse to outline the regions you want. Or you can try "x". w will unzoom.
      • You must now "select" using the s key: hit s, and then mark the left and the right side of the spectral line using left-clicks on the mouse.

        

      • Now hit "Construct kernel." The holy grail here (according to the APO guide) is maximum deviations less than 1.5% ad an RM deviation less than 0.7%. Using the sample data I could not come close to achieving this. I did manage to achieve errors that were about twice that. If you don't get anything you like, go back and redo the normalization. Hit "accept."

        

      • Okay, we're back to thefire_xtellcor. Hit Scale lines. You'll be happier with just hitting "accept." If the spectrum is missing, you neglected to add the B and V magnitudes if they were missing. Mess around with "e" and dragging points with all of the orders until you're happy. Hit "accept." My experience is that this is too obscure to mess wit
      • Decide what units you want (such as ers/sec/cm^2/A) and hit "construct telluric spectra"
      • Determine Shift. So, this turns out to matter: Select Aperture 01 and Order 28. Do an "s" and then click on the left and then the right of that region with a whole bunch of telluric stuff. Hit find auto. Don't be surprised to get a value like -1.50. Hit accept, and then Apply to All orders.
      • Write File.
      • Repeat for the other 3 program (or 7) program images. Each time select all 4 of the telluric observations. The nice thing is that a much smaller window will pop up. You can just hit "Load Spectra", "Apply to All orders," and "Correct spectrum."
      • Do a Combine. Make sure it was the correct number (4 or 8, usually).

Running Larry's astrometry code

• Use apphot's "center" or psf fitting to produce a file of accurate x and y positions.
• Hack on m31makeobj.f So it produces:
  • fort.9 which contains an imcopy of the image to a date-based image name.
  • a "Mask*.obj" file that contains some magic numbers and a list of 0-indexed x and ys.
• Run m31makeobj
• cl < fort.9
• copy lmi.key one level above your working directory
• copy astrom.inf into your working directory
• copy dofind.pro to your working directory and edit:
  • Update the root name
  • Update the directory name to your working directory ending with a "/"
  • idl
  • dofind,1,5 if you have rootname.001, .002, .003. If it doesn't work you probably have screwed up the directory name.
  • mkdir SRC and mv *.src SRC
  • edit doast.pro to include the right root name and directory name.
  • doast,1,5
  • Results are in *.ast. Other fun files:
    • fitcoeff.dat
    • Refstars/image
    • centers.dat

GMOS Software

Designing a slitlet mask

• Download a DDS image. Example m33.fits[0]
• Mess with ds9 and region files deciding that where it makes sense to put masks, and approximately angle. Keep in mind parallactic angles.
• Make list of objects and potential alignment stars (MosA.in) including priorities.
• When you're close, make a smaller version of the dss image, i.e., dssE.fits[0].
• Bring up the gemini IRAF. Two ways currently:
  1. Ureka
     • ur_setup
     • cl
  2. anacoda [installation instructions below]
     • bash
     • source .bash_profile
     • conda activate iraf27a
     • cd iraf
     • cl
• gmskcreate
• Note: the naming convention is stupidly important. "GMI" images are what gets used by GMMPS.
  • indata="MosA19.in"
  • gprgid="GN-2020B-Q-237"
  • instrument="gmos-n"
  • rafield=1.55138888889 [can enter as HH:MM:SS.SSS]
  • decfield=30.527777778 [can enter as +DD:MM:SS.SS]
  • pa=340
  • fl_getim=yes
  • inimage="dssA.fits[0]"
  • outimage="GN2020BQ237-01"
  • fl_getxy=yes
  • outcoords="GN2020BQ237-01.coo" (not used by anything)
  • outtab="GMIGN2018BQ228-01_OT.fits"
  • iraunits="hours"
  • fraunits="hours"
  • slitszx=1
  • slitszy=5
  • verbose=yes
• gmmps: Make sure you have the latest version (in 2020 I used 1.5.2). I could not compile the code, so I used the binary versions.
• gmmps
  • File-->open-->GMIGN2020B237-01.fits
  • GMMPS-->Load Object Table-->GMIGN2020BQ237-01_OT.fits
  • Do any editing. For instance, I got rid of 501 and 599 and 598 was supposed to be better.
  • Configure mask
  • Pick grating, no filter, and reasonable CWL
  • No to pack spectra
  • min sep 4 pixels
  • wiggle 15%
  • Make mask (or whatever). See if everyone got on.
  • Now go through and try to make the slits longer by changing 5-->10. Update slit. After each change do a new Make mask.
  • Note: GMMPS isn't good at counting. The number of 0/1/2/3
  • GMMPS Load ODF. Check that 4686 is clear. Make sure that everything looks groovy.
  • Rename the out GMIGN2020BQ237-01_OTODF1.fits files to GN2020BQ237-01_ODF.fits without the GMI at the start, or th OT in the middle or the 1 at the end.
  • Don't try to load a .gz file. You can, but as soon as you go to load an OT or ODF very bad things happen.