


1. Pipeline Options
=====================


  b. Basic pipeline configuration

	beam=X			Set the instrument beam to X arcseconds.

	blacklist=clear		Clear the list of banished options.
	
	estimator=<type>	'median' or 'maximum-likelihood' estimators to
				use in deriving signal models. 'median' 
				estimators are less sensitive to the presence 
				of bright sources in the data, therefore it is
				the default for the initial gain iterations 
				(see 'gainrounds') and when 'bright' is 
				specified (see 'bright.cfg').
				
				When medians are used, the corresponding models
				are reported on the console output in []'s...
				(see the Console Output section).

	rounds=N		Iterate N times.

	gains=<method>		Specify that gains should be solved where
				appropriate. (individual models may override 
				this when they have the 'nogains' oprion set).
				Additionally an estimation <method> can be
				specified ('median' or 'maximum-likelihood'). 
				
	weighting=<method>	Enable pixel weighting by measured noise. 
				The optional <method> can specify which way
				noise estimates are derived. The following
				methods are available:

				  rms	         Standard rms calculation.
				  robust         Use robust estimates for the
					         standard deviation.
				  differential   Estimate noise based on pairs
					         of data separated by some
						 interval in time.

	time-weighting[=dT]	In addition to pixel weighting, time-weights
				can also be calculated to allow for 
				non-stationary noise. Without an argument
				all exposures are indepently weighted. 
				Otherwise, weight are derived for blocks of
				exposures spanning dT seconds. The value
				'auto' matches the time-constant to that of
				'drifts'.	

	scanweighting=  	'robust' or 'maximum-likelihood'.
				If specified, each scan gets an assigned weight
				with which it contributes to the composite map.
				This weight is measured directly from the noise
				properties of the produced map. Note, that this
				option conflicts the the 'extended' option, and
				is ignored when both are specified.

	iteration.N=[options]	Set the ; separated list of options when
				starting iteration N. For use in command-line
				mode, the '=' sign in the usual 'key=value' 
				pairs can be replaced by ':'. E.g.:
				-iteration:3=cables:true;despike:true
				This avoids a shell cry.

	iteration.last=...	Similar to the above, but setting the options
				for the last iteration, independently from
				how many iterations were specified.
	
	iteration.last-N=...	Similar to the options above but sets options
				for the N-before-last iteration.

	iteration.XXX=clear     Clear all setting for iteration XXX.

	iterations=[options]	Set [options] for all iterations explicitly
				defined thus far. E.g.
				  iterations=forget:despike
				will remove despiking from the iterations
				where it was defined...

	iterations=clear	Clear all iteration-based settings. 

	relative-pixel-noise=	Specify what range of pixel noises are 
		min,max		admissible, relative to the median pixel
				noise. Pixels that fall outside of the
				specified range (min, max) will be flagged.

	nefd-range [min],max	Specify a range of acceptable NEFDs in 
				Jy sqrt(s) units. All scans with NEFD outside
				this range will not contribute to the composite
				map.
				If the optional 'min' value is not given, then
				it is assumed as 0. ('-' may also be used in
				place of ',' for separating min and max 
				values.) 

        stability=X		Specify the instrument's 1/f stability time
				scale in seconds. This value is used for 
				optimizing reduction parameters when (e.g. the
				filtering time scale for the 'drifts' option)
				when these are not explicitly specified.

	ordering=a,b,c	Specify the order of pipeline elements as a comma
			separated list of keys.
	


 
   c. Model enabling/disabling

      Each model in the pipeline can be disabled and enabled at will.
      A more detailed discussion of what models there are and what these do
      is discussed later. Here's just a very brief mention.      

	<model> [options]	Enables the model <model> in the pipeline
				with the optional [options].

	forget <model>		Disables the model.





2. Advanced Source Configuration
================================


	blank=X		Skip data from modeling over points that have a source
			flux exceeding the signal-to-noise level X. This may
			be useful in reducing the filtering effect around 
			bright peaks. See also -clip below.

	clip=X		In early generations of the source map, force map
			pixels with flux below signal-to-noise level X to zero.
			This may help getting lesser baselines, and filtering 
			artefacts around the brighter peaks.	

	noiseclip=X	Flag (clip) map pixel with a noise level that is more
			than X times higher than the deepest covered parts
			of the map.

	exposureclip=X  Flag (clip) map pixels whose relative time coverage
			is less than the specified value X. 

	sourceweights	If specified, appropriate weigths will be calculated
			separately for mapping. Otherwise the usual rms weights
			are used. The difference is that proper source weights
			account for just how much sources of 'sourcesize' are
			filtered as a result of the various decorrelating and
			noise whitening steps.

	scanmapdespike=X  Despike scan maps at the significance level X. 
			  Clearly you want to set X to be higher than the most 
			significant source in your map. Therefore it is only 
			really useful in 'deep' mode, where 5-sigma despiking
			is default (see 'deep.cfg').

	jansky=X	Specify the calibration factor from uV to Jy.


	scanmap-redundancy=N  Specify the minimum redundancy (N samples) that
			      each scan-map pixel ought to have in order to be
			considered valid. Pixels with redundancies smaller than
			this critical value will be flagged an not used in
			the composite source mapmaking.

	sources=<file>	Add sources as specified by a crush-style mask <file>.
			This is useful for runnign reduction on simulated data
			that can help understanding the systematics of the
			reduction. One thing to look out for is that the
			peaks here have to be specified in the instrument's 
			native units (i.e. volts), rather than Jy. This way
			any conversion related confusion is bypassed since the
			signal levels are unambigiously specified.

	sourcemodel=<file>  Specify a in initial source model tgo use in the
			    reduction. The option is useful to feed back the
			result from a previous reduction into another reduction
			round. This may be useful when reducing large datasets
			where all data cannot be reduced together. Instead the
			data can be split in manageable sized chunks, which are
			reduced separately. The results can be coadded with the
			'coadd' utility to create a composite map. This may
			be further manipulated (e.g. s/n clipping, smoothing,
			filtering etc.) with 'imagetool' before feeding back
			into another round of reduction.


3. Advanced Scan Configuration
==============================

	vclip=min[,max]	Clip data where the field scan velocity is outside
			the specified range (in arcsec/sec). The successfull
		 	disentangling of the source structures from the various
			noise terms relies on these being separated in 
			frequency space. With the typical 1/f type limiting 
			noise, this is harder when the scan speed is low s.t. 
			the source signals occupy the low frequencies. 
			Therefore, requiring a minimum scanning speed is a 
			good idea...
			On the other side, too high scanning speeds will smear
			out sources, if the movement between samples is larger
			than ~1/3 beam.


	aclip=X		Clip data when the telescope acceleration is above
			X arcsec/s^2. Heavy accelerations can put mechanical
			energy into the detector system, thereby generating
			false signals. Clipping data when there is danger of 
			this happening is a good idea. (see also the 'accel' 
			option for possible modeling of these signals)

	downsample=X	Downsample the data by a factor of N. At times the
			raw data is sampled at unnecessarily high frequencies.
			By donwsampling, you can ease the memory requirement
			and speed up the reduction.

	frames=<from>-<to>  Read only frames <from>-<to> from the data. Maybe
			useful for quick peeks at the data without processing
			the full scan.

	shift=X		Shift positional information by X seconds. This is 
			meant to be used only when there is a timing problem, 
			whereby the data frames are misaligned from the 
			telescope position data.
	
	scramble	Make a map with inverted scanning offsets. Under the
			typical scanning patterns, this will not produce a
			coherent source. Therefore it is a good method for
			checking on the noise properties of deep maps.

	taufactor=X	One can specify a factor by which skydip-equivalent 
			tau values ought to be multiplied to obtain good 
			calibration. A gain dependece on loading can be 
			effectively modeled in this way. E.g. for LABOCA
			taufactor=1.3 provides approximately flat calibration
			in all weather conditions and at all elevations.

	scanmap-redundancy=N   Specify the minimum number of data points that
			       have to contribute to each pixel in a scan's
			       map in order for that pixel to be valid.

	flag=ch1,ch2,...chn    Specify a list of backend channels that ought
	flag=ch1-ch2,ch3...    to be flagged for the successively read scans.

	pixeldata=<filename>   Load pixel defaults from <filename>. The format
			       is identical to the output of 'pixeldata=write'
			       option.

	rcp=<filename>	Use the non-default RCP file from <filename>.	

	uniform		Specify to use uniform pixel gains initially
			instead of the values read from the appropriate
			pixel data file (see 'pixeldata' key)...



4. Pipeline elements (Models) and their options
===============================================

	source		Solve for the source model.
			The optional argument [opt] can be:

				nosync		Do not sync the derived source
						increment to the data.
						Use only if you're sure you
						want this...
				mem		Maximum-Entropy correction. 

			MEM mapping can be used to suppress low-level noise
			but may cause some filtering on faint structures
			also. Nonetheless, it is an attractive method to
			produce very clean looking maps when for sources with
			S/N >> 1.

	offsets		Remove the residual DC offsets from the bolometer 
			signals (ignored when 'drifts' below is also 
			specified.)

	drifts=X	Filter low frequencies below the characteristic 
			timescale of X seconds. An effective way of dealing 
			with 1/f noise.
	
	hipass=X	Highpass filter the timestream over X second 
			timescales. This is an alternative way to 'drifts' to 
			dealing with 1/f noise. However, 'hipass' should always
			be used in conjunction with 'offsets'...

	decorrelate	Remove the correlated noise term accross the entire
			array. This is an effective way of dealing with most
			of the sky noise, grounf pickup and/or temperature 
			fluctuations
			
	gains		Solve for pixel gains based on their response to the
			correlated noise (above).

	weighting	Derive pixel weights based on the rms of the unmodelled
			timestream signals.

	despike=<method>,X[,...]  Use despiking method <method> at a S/N level 
	despike2=[...]            X (with possible further options). The
	despike3=[...]	          following methods are available:

				   neighbours  Despike only by comparing 
					       neighbouring data.

				   absolute    Flag data that falls above the
					       specified level.

				   gradual     Like 'absolute' but proceeds 
					       more cautiously.

				   features    Look for spikes wider than just
					       a sigle frame. The 'featurewidth'
					       option (below) controls up to
					       what timescale are spikes sought.

				  All methods can flag pixels and frames if 
				  these have too many spikes. The critical 
				  number of spikes pee channel is set by the 
				  larger of 'spikefraction' times the number of
				  frames in the scan, or 'spikecount'. 
				  The critical number of spikes per frame are
				  determined by the 'framespikes' option.
				  See these options below. 
	
	spikefraction=X		Tolerate (w/o pixel flagging) spikes up to 
				fraction X of the scan frames in each channel.

	spikecount=N		Tolerate (w/o pixel flagging) up to N spikes up
				in each pixel.

	framespikes=N		Tolerate up to N spikes per frame;

	spikewidth=X		When despiking using the 'features' method,
				spikes up to X second in width will be sought.

	sky			Model spacial sky noise as gradients accross
				the array.

	whiten			Use noise whitening algorithm. Careful, it is
				not yet well tested, and may affect the 
				calibration!!!
	
	neighbours		Decorrelate pixel neighbours on array. Note,
				this is effectively an extended structure 
				filter which gets more aggressive with 
				iterations. Therefore use only with care, if
				you must, and for 1 or few iterations only!!!





4.5. Further output options
===========================

	pixeldata=write	Write the pixel data file (gains, weights, flags). The
			putput will be 'pixel-<scanno>.dat'. You can use these
			files to update instrumental defaults in the instrument
			subdirectory. E.g., you can overwrite 

				laboca/laboca-pixel.dat

			to change what default pixel settings to use.


	covars		Write covariance data. The full pixel-to-pixel 
			covariance data is written, as well as versions of it 
			with pixels reordered in groups (boxes, cables, boards,
			squidgroups or squids) with the dead pixels discarded, 
			for convenient study of the correlated signals that 
			may be present.


	intermediatemaps  Write the maps made during the reduction into 
			  'intermediate.fits' (inside the minicrush directory).
			This option thus allows to keep an eye on the evolution
			of maps iteration-to-iteration. Each iteration will
			overwrite this 'temporary' file, and it will be erased
			at the end of the reduction.

	scanmaps	When specified, a map will be written for each scan
			(every time it is solved), under the name 
			'scan-<scannumber>.fits' in the usual output path.
			Best to use as:
			     
			     iteration:last scanmaps
 			
			to avoid the unnecessary writing of scan maps for every
			iteration (unless you really want that to be the case).

	spectrum [name][,][size]  Writes channel spectra (of residuals) in an 
				  ascii table. Optional arguments include the
			'name' of the window function to use (default is 
			'Hamming', and the window size (in powers of 2!, 
			default is 512). 





	uniform		Assume uniform sky-noise gains initially. Gains will
			be solved for during the reduction, unless the 'gains'
			key is also explicitly unset.

	wiring=<filename>	This option is commonly used to specify a
				file containing the wiring information of
			the detectors, which can be used to establish the
			typical pixel groupings of instruments. There is no
			standard format for the wiring file (it may be
			different for each instrument), and not all instrument
			may use such information. However, those that do have
			this type of data do use the 'wiring' key to point
			to the location of the information.

	febe		Defines the frontend-backend combination to use. E.g.
			for LABOCA, this would be set to 'LABOCA-ABBA'.


   	config=<filename>	Load the configuration file filename. 
				The file is looked for in the locations in the
				following order:

					1. ./
					
					2. ./<instrument>/

					3. ~/.crush2/

					4. ~/.crush2/<instrument>/

				Whenever a matching file is found its contents
				are parsed. Because of the ordering, it is 
				convenient to create overriding configurations.
				Thus instrument specific settings can be used 
				to override default settings, and user specific
				settings placed in '~/.crush2' can override
				shipped defaults. Whenever a configuration is
				parsed, there is a note of it on the console
				output so that one always knows which files 
				were read and in what order.
				E.g. when using
				
				   > crush laboca -faint 12066

				the following configuration files will be
				loaded in order (provided they exist):

				   default.cfg
				   laboca/default.cfg
				   ~/.crush2/default.cfg
				   ~/.crush2/laboca/default.cfg
				  
				   faint.cfg
				   laboca/faint.cfg
				   ~/.crush2/faint.cfg
				   ~/.crush2/laboca/faint.cfg
				   
				Each successively loaded file may override
				the options set before it.

	forget=<option>...	Forget the priorly set values for <option>
				as if it were never defined. E.g. 
		     		
					forget=outpath

				will unset the 'outpath' option.
				You can specify more than one options as a
				comma-separated list. E.g.
	
					forget=outpath,project
				
				With unset both the 'outpath' and 'project' 
				options.	

	recall=<option>		Undoes 'forget', and reinstates the <option>
				to its old value.	

	remove=<option>		Similar to 'forget', but removes the entire
				branch. Thus '-remove=despike' unsets:

					despike
					despike.level
					despike.method
					despike.flagfraction
					...

	replace=<option>	Undoes the 'remove' option, reinstating the
				<option> tree to its prior state.

	blacklist=<option>...	Similar to 'forget', except it will not
				set options even if they are then specified
				at a later time. This is useful for altogether
				removing settings from the configuration.

	whitelist=<option>...	Remove <option> from the blacklist, allowing
				it to be set again if desired. Whitelisting
				an option will not reinstate it to its prior
				value. After whitelisting, you must explicitly
				set it again, or 'recall' or 'replace' it
				to its prior state.

	poll			Whenever unsure what options are set at any
	poll=<option>		given stage, you can poll the settings.
				Without an additional argument it will list
				all options to the standard output. When
				an argument is specified it will list
				all configuration settings that start with
				the specified string. E.g.

					> crush [...] -poll=iter
	
				will list all iteration based options that 
				are set including all the [...] options set
				prior to '-poll' in the command line.


