Basics:

	OK Overhaul dependence accounting...
	OK What's the matter with robust decorrelating
	OK What's wrong with nefd?
	OK what's wrong with decorrelating cables -> Empty gainflagging range!
	OK Fix He3 correction
	OK time ordering options?
	  alternatively, set forgets/recalls/blacklists/whitelists
	  at the beginning of setOptions...
	OK setOptions to proceed in time order
	OK blacklist/whitelist (whitelist problems)...
	! implement poll
	OK insertion order -> easy: abolish insert
	OK performance: 166s vs 90s (almost 2x overhead...)
		-> arraylists at most a few seconds....
		-> speeded up decorrelating/gains(!) -> ~115s
		   (of which gains are ~30s)
		-> always iterate frames first (faster caching) -> 83s
			OK offsets			(4%)
			OK standard rms weighting	(9%)
			OK !source weights?		(13%)
			OK flagSpikyChannels...		(6%)
			OK absolute despiking		(11%)
			OK (differencial weights)
			OK (despike neighbours)
			===============================================
			Total				(43%!)
		-> Source on predefined channel group (no flag check)
		   (remove from all sky channels...)	(8+4%)
		-> minimize double -> float conversions	(23%)...
		   (sourcemap + removal...?)
		===============================================
							(88%!)
		-> getTimeStream() + whiten (mostly fft)(10+7%)
		OK sourceBoxing to work on extreme pixels only...

	  Latest results: 73s vs 84s (ml no whiten). -> 13% improved!!!
	  Ultimate1: (noHe3, ml, wrms, dA, nowhiten)
		54s vs 80s -> 33% improvement (50% faster)
	  Ultimate2: (+10 rounds, source last only)...
		40s vs 103s -> 62% improvement (158% faster!!!)

	  Ultimate3: (-deep blacklist=whiten,filter -weighting=rms)
		>34.2 vs 103.4	-> (3.02x)
	  Deep:
		>45.6 vs 122.8	-> (2.69x)
	  Default:	
		>47.6 vs 115.9	-> (2.43x)

	OK What's wrong with source weighting?
	   (not sure, but simplified it altogether...)
	   (actually, used to measure high-freq noise only)...
	OK WorkPath
	OK Why initial cable gains screw up???
	   (because laboca/pixel.dat contains garbage cable gains...)
	OK Move Subscan private arrays inside decorrelate/gains/offsets
	OK Change backendIndex to 1...n (used in lookups)
	OK consistent mode indexing: 0...n-1 or 1...n?
	OK CorrelatedSignal to have its own temporary storage
	   (WeightedPoint[channels*resolution])
	OK wiring groups to include blind channels...
	OK (for LABOCA...)
	* temperature correction as pipeline element (LABOCA)?
	! jansky units for SkyMap -- use instrument value -- or better:
	  make scanmaps using the value?
	OK Whitening messes up -> timestream was mostly 0
	OK Why are there NaN weights before whitening (there weren't)
	* filtering: what about medians/field rotation...
	  CorrelatedMode.calcFiltering(Vector<Scan> scans)?
	  This way instruments can calculate their own filtering?
	OK Check flagging ranges...
	* ordering.optimize
	.. Add latest flags/rcps...
	OK Are sky gradients completely kosher?...
	* dynamic model resolution (perhaps later using addSignal())
	OK despike only live channels?
	OK correlated smooth instead of blocks? -- nay!
	* Filtering accounting similar to parms?
	* SkyMap.gaussConvolve... (SkyMap.smooth)
		* Convolve only at positions 1/5th of the convolving FWHM.
		  (even if flagged)
		* Interpolate inbetween
	OK smarter blanking...
	OK what's wrong with source boxing on extremes?
	* autocrop image at the end of reduction (if not fixed size)...
	* Acceleration mode/modality...	
	OK forget/recall/blacklist/whitelist to process lists.
	* instruments,projections,modalities,groups,divisions etc list.
	! Test multithreaded.
	! Copyright...
	OK Weight normalized to seconds of integration time...
		OK instrument.integrationTime
		OK read/write weights converting from/to standard weights
		OK sharc2 samplingInterval -> nativeSamplingInterval
	OK Why does 1 exposure always get trimmed?
	OK Why is whitening not working? -> doubled windowSize to bridge gap...
	OK Frugal Slimming/trimming... 
	* Update Spherical package to xcrush....
	OK Spherical coordinates native and default offsets...
		* addOffset(Vector2D), addNativeOffset(Vector2D);
		* offsetFrom(), nativeOffsetFrom()
	OK PA sign was wrong -- ooops...
	OK downsample = auto (based on mean scanning speed and beam size)
	* jansky definition (jansky = x.xx [V]) 
	OK add smooth=beam (halfbeam, minimal) options...
	OK subscan to have truly independent instruments for weights/gains...
	* APEX read generic scans spec...
	* APEX auto compressed detection...
	OK Uranus 11564 default S/N: 659 vs 763 (236 vs 251 channels)
	  ...778 vs 762 with robust weights.
	  ...852 vs 762 with ml estimates.
	  ...900 vs 762 with rounds 6+.
	  ...925 vs 762 with ML estimator AND weights
	  ...975 vs 762 with ML + differential weights
	  ...942 vs 762 + w/o sky-gradients
	  ...936 vs 762 w/o MEM
	  ...1060 vs 762 w minv=18 (was 36)
	  ...963 vs 762 w time-weights
	OK differential weights with crossing time sep. (10x)
	OK Deep no extfilter 14.2 vs 13.0 mJy/beam
	OK Rename sourcemap -> source
	* 'corners'?
	OK covar[=group1,group2...]
	? write.pixeldata, write.covar[.group]
	* calculate beam efficiencies...
	OK whitening overstates source filtering? Fixes to whitening...
	OK Adjustment to average filtering calculation... 
	OK Revise "extended" conditionals in source
	OK Revise "sourcesize" in source (except drifts).
	OK C1: Bug with ArraySubscan.getSourceSize(double)
	OK alternative group filtering calc...	
	OK automatic vclip (from sampling and stability)
	OK Problem with He3 correction? (pixel.dat had 0 weight for blinds.)
	OK Reinstate final robust weighting if !extended || deep
	* MEM changes...
	OK Without pixeldata laboca misbehaves... (Default weight was 0.0)
	OK problem when using pixel.dat. 
	  (IntegrationTime defined only in subscan)...
	OK 'group.<name> ch1,ch2...chn' key
	OK 'division.<name> group1,group2,...groupn' key
		+ division.<name>.gainField
		+ division.<name>.gainFlag
	OK blind {....} for specifying blind channels
	OK pixel lists to include ranges...
	* list groups & modalities in instrument.validate()...
	* Make sure CorrelatedMode calculates signal on subset of channels
	  but remove from all and calculates gains for all...
	? merged decorrelation/gains?	
	* CorrelatedMode always automatically include its own gain flag..
	* Instrument.getModelingFlags()?
		* apply to modalities...
	* Instrument.getSourceFlags()?
		* apply to census, flagWeights, nefd, whitening mean... 
		* check for all Channel.isUnflagged() if applicable... 
	OK include laboca resistors in cables decorrelation...
	OK whiten level directly from whiten option...
	OK Source synched even to SKIP_SOURCE frames
	OK Frames flagged as invalid are discarded by Subscan.validate()
	OK Fixed flagging (contains vs containsKey).
	OK 'date' key working the same as 'mjd' but more human readable...
	* date.[2008-06-15:2008-08-02]
	* getCrossingTime for position switched data (where aveOnSourceTime)
	* Does detwisting make sense?
		non: 76.6 (11.6)
		yes: 74.6 (11.1) --> 3-5% percent improvement...
	OK AtlasGal C1 inf problem... filtering corrections were derived for
	  actual source size -- should have been for point sources...
	OK What's wrong with new saboca scan coordinates? (Wrong RCP?)
	* positive ranges not working with ':'...
	* wanderer photometry
	OK covariance grouping (-covar[s])
	.. output products
	OK polling
	* add configuration to FITS header
	* 'obslog' and Scan.getObsLogEntry()
	OK Scan.writeProducts()
	* LABOCA pin order cables...
	* aliases:
		- Define common aliases for more backward compatibility
		Generic: covar=write.covar?
		LABOCA: boxes,cables,amps,twisting
		SHARC2: rows,mux,tau(225GHz)=tau.225GHz
	OK Full covariance: issues with backend indexing...
	* fit -- to do source fitting at the end...
	* show -- to display image at the end...
	* beammap -- after reduction, make a map for each bolo, and fit
	* -poll=conditionals,blacklisted,forgotten,aliases,all	
	OK Fixed decorrelation filtering calculation...
	OK Fixed measuring scanning v on drift scans.
	OK much improved on filtering calculation... 
	OK filtering now always reflects its up-to-date value.
	OK added 'smooth.assume-only' key.
	OK fixed blind pixels (e.g. resistors) being weight flagged.
	   weight flagging is now applied to sky-weights not source weights...
	OK repackaged.
	OK whitelist also recalls...
	! Add correlated signals (chopper, azymuth, elevation, 
	  airmass[?], acceleration[x,y,total])
	! Add modes for signals... (Perhaps SignalMode to create 
	  CorrelatedSignal and Mode simultaneously)
	OK Framework for generic scan spec reading.
	OK Update APEXArrayScan for generic scan spec (name or number).
	! Jackkniving problem
	! Range parse problems...
	! Error if whitening without weighting (need 'weighting' option)?
	OK APEXscans with -altaz -> PA had wrong sign...
	OK EQ <--> HO conversions -- verified with LABOCA 
		OK Something fishy with precession (epochs need to be right :-)
	OK Simplified EQ <--> HO conversions...
	OK 'compressed' no longer required. Automatically detected...
	OK Skydips are automatically skipped.
	OK Not blanking for last 2 iterations, s.t. noise can be better 
	   modeled.
	? MEM before or after smooth?
	OK more stringent whitening produces better results...
	! whitening,clipping,blanking with many iterations...
	! processChopper before measuring velocities
	  (calculates aveOnSourceTime)
	! actual on source time is combo of chopper time and scanning...
	OK If downsampling is specific, then do it before velocity flagging...
	! [rot>x.xx] condition...
	OK extended filter no longer default in deep. Whitening makes it
	   somewhat unnecessary...
	OK Simplified parallactic angle calculation!
	OK Laboca -altaz problems... -- Misunderstanding native coord system.
	OK Why altaz maps are centered on EQ coordinates?
	! jansky definition...
	! scan HDUs ("Scan 10234", "Scan #1" if serial is undefined...)
	! scandata, models, instrumentdata (for FITS storage)
	! fit, show, point (point.cfg) for pointing
	OK dataunit
	OK instrument.gain
	* Optimal filter
		whiten -> filter
		filter.type = whiten|optimal
		filter.whiten.level
		filter.optimal.lambda	
	* sourcename
	OK epoch.setImmutable(boolean)
	OK privatize precess(year)
	* Instrument registry 
	* Instruments main() to 'install' 'remove'	 	
	OK 10^6 factor in Jy conversion (and NEFD)...
	* Problem with : or = (or spaces) in conditional.
	.. Move Channel.parseHeader and toString up to root (for default)
		* Update all pixel data files to new format...
	OK Move calibration and Tau tables up to root...
	? GenericAPEXArray <- an apex array that can be described
		(cabin A, pixels n, rotation x.xx, etc.)
	OK Unnecessary error message when scan range is specified...
	OK Fixed offset/native-offset sign problem...
	OK added iteration.[mid]
	OK Added [!key] type conditions
	* update [extended] configurations
	* add [multiscan] condition
	! Chopped data processing...
	! Write configuration into FITS header
	.. Added two-way dynamic conditional...


* xcrush: CoordinatePair, Range, EquatorialCoordinates, GeodeticCoordinates
	... etc. Check and merge as necessary...
	

!!! ML estimators are much more reliable...

ASZCA/SABOCA migration...
	OK sourcemap -> source
	OK add pixeldata
	OK bootstrap pixel.dat files
	OK add verbosity to APEX scans
	OK filtering always calculated for point sources...

SHARC-2 migration

	-------------------------------------------------------------
	[Scan 8302]
	=============================================================
	Default:
		45.5s	vs	90.7	---> 	(2.0x)
	Deep:
		50.9s	vs	127.8	--->	(2.5x)
	-------------------------------------------------------------

	OK add divisions and modalities.
	* ask Darren for wiring...
	OK implement 13 Hz filter (see if filtering can be part of Subscan)...
	  filter(double[] response). but not used....
	OK direct-tau/gains etc.
	OK move instrument settings from scan to Sharc2?...
	OK update configs...
	  -> add serial keys for:
		OK rotator angles (make angles read)...
		OK pixel scales (make pixel scales read)...

	* Change loading model to include coupling and offset...
	  -> update configs...
	  -> Use only original nonlinearity file...
	XX implement 'edge'
	OK Check for missing implementation... 	
	XX make backend/row/mux/block/amp indexing start at 1...
	OK add:
		OK rot0 values
		OK pixel scales
		OK nonlinearity data
	OK data/offset signs...
	OK performance 3.5x faster in deep!!!
	OK What goes wrong when pixeldata is used?
	* measure frame transmissions
	XX bootstrap nonlinearity...
	OK center pointing
	OK Check 9713 reduction (rotation)
	OK Need gradients? not really -- perhaps in deep
	OK What's wrong with Pallas? -- serial option not loaded due to Range...
	OK Vesta S/N (smooth=4.0) (extra round for new version!)
		* 106 vs 77			(38% more sensitive)
		* 102 vs 77 w/o gradients	32.9s vs 41.2s
		* 106 vs 77 90% v accomodation	26.0s vs 41.2s (58% faster)
		* 109 vs 77 vclip (rounds 5)	22.8s vs 41.2s (80% faster)
		* 114 vs 77 w/o MEM		(48% more sensitive)
		* 121 vs 77 changes to whiten
		* 147 vs 77 extended fullres
	OK Deep
		* no extfilter ( vs 45.4 mJy 47.9 mJy)

	OK Pointing problems (pointingCenterOffset was wrongly moved...)
	OK Pointing problems II > EQ centering OK, horizontal off by pcoffset?
	  	I. rotation updated only the horizontal offsets and not
		   the eq coords.
		II. setReference mispelled, did not override...
		III. error in frame rotation formula... 
	OK What's wrong with load calculation?
	* fasttau, or dynamictau or tau.dynamic or tau.variable
	OK profile scan reading...
		OK? faster maitau?
		OK faster robust offsets? (after downsample)
	OK 1s resolution is OK, 3 not so much...
	! Setting 225GHz tau is failing...
	XX default to sharc2 (for compatibility)
	OK Process chopper signal (read, add to offsets and center...)
	* Auto chopper detection (throw and frequency...)
	  (adjust position smoothing...)
	OK What's wrong with Jan 2009 scans?
	   ROTZERO is stored in the FITS files as of late...
	* Chopoffset mode...
	OK mux degrees of freedom restricted, by solving entire row at once...
	OK why outpath does not work...
	OK "-faint -extended" == "-extended -faint"
	* chopped signals
	OK SHARC-2 backend indexing changed to run 1--384
	OK fazo, fzao keys units and signs...
	OK Fixed direct tau calculattion, updated gain data...
	* Use of excess load...
	OK Incorporated elevation dependent response using Darren's files...
	OK slowed down rows/mux decorrelation for better fidelity images...
	OK NaN problems (with map nefd) in more iterations...
	OK MJD calculated from DATE-OBS if not otherwise defined...
	OK Separate instrument rotation (rotating pixel positions)
	  from frame rotation...
	OK Fixed pointing center offset bug...
	OK Setting wave options on startup in unncessary (set at scan read)...
	OK Fixed flagging by invalid RA/DEC and AZ/EL

GISMO:
	OK horizontalOffset troubles...	


ConfigurationManager:
	OK forget/recall
	OK lists for forget,recall,blacklist,whitelist
	OK wildcards processing for branches...
	OK Conditionals...

Modalities:
	* Acceleration modality
	OK Gradient modality
	OK FieldGradientMode...

configs:
	- Merge options into new hierarchy...
	OK mjd dependence (rcps etc.)
	* Change README (rewrite scripting)
	* Document changes...

more options:
	* identify where more configurability can be useful...

MiniCRUSH:
	OK loads of work...
	
Timeconstants...
	OK not a big deal...

Copyright...


Test...


CRUSH-2:

  *	Repackage into 'xcrush/minicrush'
	(need to add imports all over the place...)
	All instrument specific source files into 'minicrush.instruments.?'	
		crush.apex
		crush.laboca
		crush.sharc2	

  OK	Add basic SHARC-2 capability...
	OK (how to deal with rotator/nasmyth mount...)

  *	Deprecate ConfidenceCalculator. Use CumulativeNormalDistribution 
	instead...
	
  * 	Add GISMO functionality...

  *	Merge SkyMap functionality...

  *	Migrate CRUSH tools...

  * 	Make gain/weight estimation mergeable...

Further down...

  *	Read ESO archive data...

  *	abstract SourceModel -> (SkyMap, Photometry, Spectrum, ChannelMaps)

  *	Add old CRUSH functionality:
		* writing scan data (incl. models, histograms)
		* automatic pipeline reordering
		* automatic timeconstants
		* dynamic gains?
		* filtering w oriented modes vs non-oriented modes...
		
  *	Deresonate (Decouple) capability...

  *	Spectral and mixed instruments (CHAMP+)

  *	Herschel support.

  * 	Further instrument support (Bolocam, Z-Spec, SCUBA-2...)

  *	Interferometers (VLA, SMA, IRAM, CARMA, ALMA)...

