Map-making
Harlequin includes a routine that applies the destriping algorithm to time-ordered data, producing a maximum-likelihood map of the intensity and polarization signal.
The destriping algorithm used in the code is described in the paper Destiping CMB temperature and polarization maps, Kurki-Suonio et al., A/A 506, 1511-1539 (2009), and the source code closely follows the terminology introduced in that paper.
The destriping algorithm is effective in removing 1/f noise originated within the detectors of an instrument, provided that the noise is uncorrelated among detectors. It requires the user to specify a baseline, i.e., the maximum time span in which the noise can be assumed to be uncorrelated (i.e., white).
Since the destriper is effective only when much data is available, it is often the case that the input data to be fed to the algorithm is larger than the available memory on a machine. In this case, the destriper can take advantage of a distributed-memory environment and of the MPI libraries, if they have been loaded before Harlequin, like in the following example:
import MPI
import Harlequin # Ok, use MPI whenever possible
# ...
You can check if MPI is being used with the function use_mpi
.
Destriping is based on a datatype, DestripingData
, and on the function destripe!
. To use the destriper, you create an object of type DestripingData
and then call destripe!
on it. The TOD must be split in a set of observations, using the datatype Observation
; we will see how to use them in the section Splitting a TOD into observations.
Splitting a TOD into observations
Harlequin.Observation
— Type.mutable struct Observation{T <: Real}
An observation is a sequence of time-ordered data (TOD) that has been acquired by some detector. It implements the following fields:
time
: an array ofN
elements, representing the time of the sample. Being defined as anAbstractArray
, a range (e.g.,0.0:0.1:3600.0
) can be used to save memorypixidx
: an array ofN
elements, each being a pixel index in a map. In no way the Healpix pixelization scheme is enforced; in fact, the map must not even be spherical.psi_angle
: an array ofN
elements, each containing the orientation of the polarization angle with respect to some fixed reference system (e.g., the Ecliptic plane). The angles must be in radians.tod
: an array ofN
elements, containing the actual data measured by the instrument and already calibrated to some physical units (e.g., K_CMB)sigma_squared
: an array ofN
elements, containing the squared RMS of each sample. To save memory, you should usually use aRunLengthArray
here.flagged
: a Boolean array ofN
elements, telling whether the sample should be discarded (true
) or not (false
) during the map-making process.name
: a string representing the detector. This is used only for debugging purposes.
To create an observation, the constructor takes keyword arguments instead of parameters; in this way, the code should be more readable. Also, sensible defaults will be provided for the missing fields:
# We do not specify "sigma_squared" nor "flagged", so they will be
# initialized to a vector of ones and a vector of `false`
obs = Observation{Float64}(
time = 0.0:0.1:3600.0,
pixidx = 1:3601,
psi_angle = zeros(3601),
tod = randn(3601),
)
High-level functions
Harlequin.DestripingData
— Type.mutable struct DestripingData{T <: Real, O <: Healpix.Order}
Structure containing the result of a destriping operation. It contains the following fields:
skymap
: aPolarizedMap
containing the maximum-likelihood maphitmap
: a map containing the weights of each pixel (this is not the hit count, as it is normalized over the value of $σ^2$ for each sample in the TOD)nobs_matrix
: an array ofNobsMatrixelement
objects, which can be used to determine which pixels were the most troublesome in the reconstruction of tehe I/Q/U componentsbaselines
: the computed baselines. These must be kept as a list ofRunLengthArray
objectsstopping_factors
: sequence of stopping factors, one for each iteration of the CG algorithmthreshold
: upper limit on the tolerable error for the Conjugated Gradient method (optional, default is1e-9
). Seecalc_stopping_factor
.max_iterations
: maximum number of allowed iterations for the Conjugated Gradient algorithm (optional, default is1000
).use_preconditioner
: a Boolean flag telling whether a preconditioner should be used or not (optional, default istrue
). Usually you want this to betrue
.
The structure provides only one constructor, which accepts the following parameters:
nside
: resolution ofskymap
andhitmap
obs_list
: vector ofObservation
; it is used to determine which pixels in the map are going to be observedruns
: list of vectors (one for each element inobs_list
), containing the number of samples to be included in each baseline for each observation
The constructor accepts the following keywords as well:
threshold
(default:1e-9
)max_iterations
(default: 1000)use_preconditioner
(default:true
)
The function destripe!
use DestripingData
to return the result of its computation.
Harlequin.destripe!
— Function.destripe!(obs_list, destriping_data::DestripingData{T,O}; comm, unseen, callback) where {T <: Real, O <: Healpix.Order}
Apply the destriping algorithm to the TOD in obs_list
. The result is returned in destriping_data
, which is of type DestripingData
and contains the baselines, I/Q/U maps, weight map, etc.
The parameters must satisfy the following constraints:
obs_list
must be an array ofN
Observation
objects;destriping_data
must be aDestripingData
object. It does not need to have been initialized usingreset_maps!
.
The optional keyword have the following meaning and default value:
comm
is a MPI communicator object, ornothing
is MPI is not used.unseen
is the value to be used for pixels in the map that have not been observed. The default ismissing
; other sensible values arenothing
andNaN
.callback
is eithernothing
(the default) or a function that is called before the CG algorithm starts, and then once every iteration. It can be used to monitor the progress of the destriper, or to save intermediate results in a database or a file. The function receivesdestriping_data
as its parameter
The function uses Julia's Logging
module. Therefore, you can enable the output of diagnostic messages as usual; consider the following example:
global_logger(ConsoleLogger(stderr, Debug))
# This command is going to produce a lot of output…
destripe!(...)
Harlequin.use_mpi
— Function.use_mpi()
Return true
if Harlequin is taking advantage of MPI, false
if not.
Low-level functions
The following functions are listed here for reference. You are not expected to use them, unless you want to debug the destriper or dig into its internals.
Harlequin.update_nobs!
— Function.update_nobs!(nobs::NobsMatrixElement{T}; threshold = 1e-7)
This function makes sure that all the elements in nobs
are synchronized. It should be called whenever the field nobs.m
(matrix M_p in Eq. 10 of KurkiSuonio2009) has changed.
Harlequin.update_nobs_matrix!
— Function.update_nobs_matrix!(nobs_matrix::Vector{NobsMatrixElement{T}}, psi_angle, sigma_squared, pixidx, flagged)
Apply Eq. (10) of KurkiSuonio2009 iteratively on the samples of a TOD to update matrices $M_p$ in nobs_matrix
. The meaning of the parameters is the following:
nobs_matrix
is the structure that gets updated by this functionpsi_angle
is an array ofN
elements, containing the polarization angles (in radians)sigma_squared
is an array ofN
elements, each containing the value of σ^2 for the samples in the TODpixidx
is an array ofN
elements, containing the pixel index observed by the TODflagged
is a Boolean array ofN
elements;true
means that the sample in the TOD should not be used to produce the map. This can be used to produce jackknives and to neglect moving objects in the TOD
Harlequin.compute_nobs_matrix!
— Function.compute_nobs_matrix!(nobs_matrix::Vector{NobsMatrixElement}, observations::Vector{Observation{T}})
Initialize all the elements in nobs_matrix
so that they are the matrices M_p in Eq. (10) of KurkiSuonio2009. The TOD is taken from the list of observations in the parameter observations
.
Harlequin.update_binned_map!
— Function.update_binned_map!(vec, skymap::PolarizedMap{T, O}, hitmap::Healpix.Map{T, O}, obs::Observation{T}; comm=nothing, unseen=NaN) where {T <: Real, O <: Healpix.Order}
update_binned_map!(skymap::PolarizedMap{T, O}, hitmap::Healpix.Map{T, O}, obs::Observation{T}; comm=nothing, unseen=NaN) where {T <: Real, O <: Healpix.Order}
update_binned_map!(skymap::PolarizedMap{T, O}, hitmap::Healpix.Map{T, O}, obs_list::Vector{Observation{T}}; comm=nothing, unseen=NaN) where {T <: Real, O <: Healpix.Order}
Apply Eqs. (18), (19), and (20) to compute the values of I, Q, and U from a TOD, and save the result in skymap
and hitmap
.
The three versions differ for the source of the TOD calibrated data:
- if the
vec
parameter is present (first form), it will be used instead ofobs.tod
(second form) - instead of
obs
(a singleObservation
object), you can pass an arrayobs_list
(third form). All the elements in this array will be used to updateskymap
andhitmap
.
Harlequin.reset_maps!
— Function.reset_maps!(d::DestripingData{T, O}) where {T <: Real, O <: Healpix.Order}
Set the skymap and the hitmap in d
to zero. Nothing is done on the other fields.
Harlequin.compute_z_and_subgroup!
— Function.compute_z_and_subgroup!(dest_baselines, vec, baseline_lengths, destriping_data::DestripingData{T, O}, obs::Observation{T}; comm=nothing, unseen=NaN) where {T, O <: Healpix.Order}
compute_z_and_subgroup!(dest_baselines, baseline_lengths, destriping_data::DestripingData{T, O}, obs::Observation{T}; comm=nothing, unseen=NaN) where {T, O <: Healpix.Order}
Compute the application of matrix $A$ in Eq. (25) in KurkiSuonio2009. The result is a set of baselines, and it is saved in dest_baselines
(an array of elements whose type is T
). The two forms differ only in the presence of the vec
parameter: if it is not present, the value of obs.tod
will be used.
This function only operates on single observations. If you want to apply it to an array of observations, use compute_z_and_group!
.
Harlequin.compute_z_and_group!
— Function.compute_z_and_group!(dest_baselines, vectors, baseline_lengths, destriping_data::DestripingData{T, O}, obs_list::Vector{Observation{T}}; comm=nothing, unseen=NaN) where {T, O <: Healpix.Order}
compute_z_and_group!(dest_baselines, baseline_lengths, destriping_data::DestripingData{T, O}, obs_list::Vector{Observation{T}}; comm=nothing, unseen=NaN) where {T, O <: Healpix.Order}
Compute the application of matrix $A$ in Eq. (25) in KurkiSuonio2009. The result is a set of baselines, and it is saved in dest_baselines
(an array of elements whose type is T
). The two forms differ only in the presence of the vec
parameter: if it is not present, the value of obs.tod
will be used.
Harlequin.compute_residuals!
— Function.compute_residuals!(residuals, baseline_lengths, destriping_data, obs_list; comm=nothing, unseen=missing)
This function is used to compute the value of $A x - b$ at the beginning of the Conjugated-Gradient algorithm implemented in destripe!
.
Harlequin.array_dot
— Function.array_dot(x, y; comm = nothing)
Compute the dot product between x
and y
, assuming that both are arrays of arrays.
Harlequin.calc_stopping_factor
— Function.calc_stopping_factor(r; comm=nothing)
Calculate the stopping factor required by the Conjugated Gradient method to determine if the iteration must be stopped or not. The parameter r
is the vector of the residuals.
Harlequin.apply_offset_to_baselines!
— Function.apply_offset_to_baselines!(baselines)
Add a constant offset to all the baselines such that their sum is zero.
Harlequin.calculate_cleaned_map!
— Function.calculate_cleaned_map!(obs_list, destriping_data; comm=nothing, unseen=missing)
Provided a set of baselines in baselines
, this function cleans the TOD in obs_list
and computes the I/Q/U maps, which are saved in destriping_data
.