
# CRUSH: SOFIA/HAWC+

Author: Attila Kovacs <attila[AT]sigmyne.com>

Last updated: 23 Sep 2018


----------------------------------------------------------------------------

#### Table of Contents


1. __Introduction__

2. __Installation__
     - 2.1. Optional automatic pipeline configuration (skyhawc)

3. __Quickstart guide__
     - 3.1. Passing CRUSH options via the DRP
     - 3.2. HAWC+ specific options

4. __Technical details for expert users__
     - 4.1. HAWC+ specific pixel divisions
     - 4.2. Glossary of HAWC+ specific options
     - 4.3. HAWC+ specific log quantities

-----------------------------------------------------------------------------




## 1. Introduction

This document contains information specific to using CRUSH-2 with 
SOFIA/HAWC+. It should be used in conjuction with the more generic README
for SOFIA instruments (`README.sofia`), and with the master README for using 
CRUSH in general.

It is recommended that you also familiarize yourself with both the SOFIA
specific README and the main CRUSH-2 README (especially Section _1. Getting 
Started_), which covers installation and basic usage.




## 2. Installation

The basic installation procedure for CRUSH is covered by both the SOFIA
specific README (`README.sofia`) and the main CRUSH README. Here only the 
installation specific to HAWC+ is discussed.
   

### 2.1. Optional automatic pipeline configuration (skyhawc)

 Automatic pipeline reductions (such as in flight) should never hang if 
 possible.

 Unfortunately, there are two uncommon conditions that can cause CRUSH (Java, 
 really) to hang indefinitely. First, while trying to render a PNG output if 
 the X11 display connection had dropped unexpectedly. And second, while trying 
 to check for updates if the network had disconnected unexpectedly. 
   
 Both of these scenarios can be preemted by specifying appropriate runtime 
 options for running CRUSH reductions with limited scope.

 Simply (create and) edit a file: 

    ~/.crush2/startup/crush/autopipeline.conf

 (This will specify settings only for the `crush` executable, and will not
 affect other tools, like `imagetool` or `show`). Into this file add the 
 following lines:

    CRUSH_NO_UPDATE_CHECK="1"
    EXTRAOPTS="$EXTRAOPTS -Djava.awt.headless=true"

 The first will disable update checking (which can hang if the network 
 connection drops), while the second will run CRUSH in headless mode (assuming 
 no X11 rendering functionality), in addition to any startup options defined 
 for all programs, thereby avoiding hangs if and when PNG outputs are rendered. 

 Normal users, outside of the automatic pipeline, should probably not set these
 options. Update notification is a useful feature, letting users know if or 
 when improved CRUSH releases become available. And, while CRUSH does not 
 currently require on-screen rendering, it is possible that future releases may
 add graphical functionalities that users may want to access (but would not be 
 available in headless mode!). 


  


## 3. Quickstart guide

For SOFIA/HAWC+ CRUSH will be launched exclusively from the DRP pipeline. See 
the DRP documentation for details.

However, if you wish to run CRUSH manually, you may easily do so from the 
command-line with `hawc+` as your instrument. E.g.:

    ./crush hawc+ [...]

See the SOFIA specific README for using CRUSH with a SOFIA instrument, or 
consult the main README for more details on using CRUSH in general. This 
document discusses only the use of CRUSH specific to HAWC+.  


### 3.1. Passing CRUSH options via the DRP

 The DRP interface to CRUSH allows manually specifying reduction options, which
 are passed to CRUSH verbetum as a single line. Each option in the line begins 
 with a dash `-`, and should not contain white spaces, unless these are 
 enclosed in quotes. E.g. below is a valid option line, with three options, one
 of which contains a white-space characters:

    -faint -name="My HAWC+ image.fits" -projection=TAN

 Below is a brief guide to a few useful options, grouped by functionality.


### 3.2 HAWC+ specific options

    -subarray=<list>    Restrict imaging to specific subarrays only. The 
                        argument is a comma separated list of subarray IDs
                        (R0, R1, T0, T1). E.g.:

                          > crush hawc+ -subarray=R0,T0 ...
   


## 4. Technical details for expert users

From this point on, the documentation is of more technical nature, intended
for expert users only.


### 4.1. HAWC+ specific pixel divisions

 For a general overview of channel divisions, please consult Section 3.
 (Correlated Signals) in the main README document.


    polarrays   Pixels of the the R and T polarization arrays, respectively.
		
    subarrays   Pixels of each of the 4 HAWC+ subarrays (40x32). Each subarray
                may contain correlated signals to themselves, e.g. do to
                correlated thermal fluctuations on their wafers.

    bias        Grouping of pixels by TES bias line. Each subarray has 20
                bias lines, each applied to two consecutive detectors rows.

    mux         A grouping of pixels based on their SQUID multiplexing scheme
                Each 4x8 pixel quadrant of the array is read out through the
                same SQUID amplifier. Therefore, it is not suprizing that 
                correlated signals are present on these quadrants. The
                decorrelating on `mux` groups is default in HAWC+ reductions

    rows        The HAWC+ multiplexing scheme is implemented in the time-domain
                Thus, the first channels of each SQUID are read out at the same
                time, followed by the second channel in each group etc. Thus,
                if there is any pickup of high-frequency signals in the
                multiplexing scheme, one could expect some correlated signals
                to be present on these multiplexer address lines. There
                is little evidence for these, but the reduction of very faint
                compact sources may benefit from the additional decorrelation
                on these groups.
                For HAWC+, these SQUID address lines are effectively the
                detectors rows, hence the naming.

    series      The grouping of pixels based on the series array through which
                they are read out.



### 4.2. Glossary of HAWC+ specific options

    band=X              Set the observing band to A, B, C, D, or E. Normally,
                        the band setting is automatic through the SPECTEL1
                        fits keyword. However, setting the band manually may
                        be useful for polling band specific settings. E.g.

                          > crush hawc+ -band=A -poll=beam

                        The above line will report the Gaussian FWHM beam size
                        (in arcsec) used when reducing band A (53um) data.
   
    biaslines           @Alias: 'correlated.bias'
                        @Expert
                        Decorrelate TES bias lines. All correlated modality 
                        suboptions apply.
                        @See: 'correlated.<?>'


    filter=<wavelength> @Expert
                        Sets the filter band to one of:

                          53um, 62um, 89um, 155um, or 216um

                        Normally, the filter is set automatically to the 
                        correct value based on the 'SPECTEL1' SOFIA header key.
                        For this reason, it is not recommended, nor trivial
                        for the user to override this automatic behaviour.
                        Rather, the setting of this option is useful for
                        setting conditional settings, which depend on the
                        HAWC+ waveband.

    fixjumps            @Expert
                        Enables ficing timestream blocks around flux jumps
                        in such a way that these will not cause problems.
                        @See: 'jumpdata'

    gyrocorrect         @Telescope: SOFIA
                        @Since: 2.41
                        @Advanced
                        Correct for gyro drifts based on guide-star relock
                        data stored in the scan headers. This isn't normally
                        needed when the gyros funtion properly. But,
                        occasionally, they drift a fair bit, and this option
                        can activate the correction scheme on demand.

    jumpdata=<path>     @Expert
                        @Since: 2.40
                        Specify the file containing the residual pixel jump
                        counts when flux jumps occur. The input file follows
                        the format by Darren with 3 columns: (1) The character 
                        'C' specifying that the value is per column, (2)
                        the column index (starting from 0), and (3) the
                        residual flux jump in readout counts.
                        @See: 'fixjumps'

    los                 @Alias: 'correlated.los'
                        @Since: 2.41
                        @Expert
                        Remove correlations with the second-derivative to the
                        telescope line-of-sight (LOS) angle. It's a good proxy
                        for removing pitch-type acceleration response from the
                        detector timestreams.
                        @See: 'correlated.<?>'

    mux                 @Alias: 'correlated.mux'
                        @Expert
                        Decorrelate on SQUID muxes. All correlated modality
                        suboptions apply.
                        @See: 'correlated.<?>'

    offset.R1=dx,dy
    offset.R2=dx,dy
    offset.T1=dx,dy
    offset.T2=dx,dy     @Expert
                        Specify the subarray offset (as dx,dy pixels in
                        the focal plane) from their nominal positions, for
                        both the first and second subarrays of the R and T
                        polarization arrays. For example, specifying zero
                        offsets for all (default) describes the focal plane
                        where R and T are perfectly aligned relative to
                        one another, and the two subarrays of each form a 
                        single seamless polarization array.

    peakflux            @Since: 2.41
                        Switch to peak-flux calibration instead of the default
                        aperture flux calibration. Recommended for point 
                        sources only.

    pixelsize=<a>[,<b>] @Expert
                        Specify the size of the pixels for calculating pixel 
                        positions based on a regular grid. 
                        The argument can be either a lateral size (in arcsec) 
                        for square pixels, or two comma separated sizes for 
                        rectangular pixels.
                        @See: 'rotation', 'pcenter'

    pols                @Alias: 'correlated.polarrays'
                        @Expert
                        Decorrelate each polarization array separately. All 
                        correlated modality sub-options apply.
                        @See: 'correlated.<?>'

    roll                @Alias: 'correlated.roll'
                        @Since: 2.41
                        @Expert
                        Remove correlations with the second-derivative of the
                        aircraft roll angle (roll-type accelerations).
                        @See: 'correlated.<?>'

    rotation=<deg>      @Expert
                        Specify the array rotation (in degrees).
                        @See: 'rotation.T', 'pixelsize'

    rotation.T=<deg>    @Expert
                        Specify the rotation of the T polarization subarray
                        relative to the R subarray.

    rows                @Alias: 'correlated.rows'
                        @Expert
                        Decorrelate along the 'row' direction of the arrays,
                        the same as SQUID address lines. All correlated 
                        modality sub-options apply.
                        @See: 'correlated.<?>'

    rtoc                @Expert
                        @Since: 2.33
                        Instruct crush to reference maps to Real-Time Object 
                        Coordinates (RTOC) for sidereal and non-sidereal
                        sources alike. Normally, sidereal object coordinates
                        are determined via the header keywords OBSRA/OBSDEC or
                        OBJRA/OBJDEC. However, these were not always filled
                        correctly during the 2016 October flights, so this
                        option provides a workaround for those scans.

    series              @Alias: 'correlated.series'
                        @Expert
                        Decorrelate on the series arrays. It seems that the
                        series array (at 4K) is sensitive to some level of
                        thermal fluctuation, and enabling decorrelation on
                        these can improve imaging quality.

    subarray=<list>     @Expert
                        Restrict the reduction only to the subarrays listed.
                        The argument is a comma-separated list of subarray 
                        codes: R1, R2, T1, T2 or R (= R1,R2) or T (= T1,T2). 
                     
                          > crush hawc+ -subarray=R1,T1 [...]

    write.flatfield[=path]      @Expert
                                Write a DRP flatfield FITS file, to be used by
                        the chop-nod pipeline. The optional argument can 
                        specify the FITS file's name. Otherwise, a default
                        name is used, containing the scan's ID, and placed in
                        the the directory specified by 'outpath'.
                        The file format is specified by Marc Berthoud.
                        @See: 'outpath'


### 4.3. HAWC+ specific log quantities

 Here you will find only the log quantities specific to HAWC+. See also 
 `README.sofia` for all SOFIA specific log quantities, and the main CRUSH
 `README` for an even more generic list of available log quantities.


    gyro.max            (arcsec) Maximum gyro drift during scan.
    gyro.rms            (arcsec) RMS gyro drift during scan.
    
    scan.Ax             (arcsec) X amplitude
    scan.Ay             (arcsec) Y amplitude
    scan.cross          Is cross scanning?
    scan.dY             (arcsec) Raster scan step size
    scan.frel           Lissajous Y/X frequency ratio
    scan.iters          Number of iterations
    scan.nsubs          Number of subscans
    scan.PA		(deg) Current scan position angle
    scan.pattern        Scanning pattern
    scan.phi0           (deg) Lissajous Y/X relative phase.
    scan.strips         Number of raster strips
    scan.T              (s) Scan duration
    scan.t0             (s) Lissajous time shift.
    scan.trk            Is tracking enabled?
    scan.X              (arcsec) Raster scan length

------------------------------------------------------------------------------
Copyright (C)2018 -- Attila Kovacs <attila[AT]sigmyne.com>


