Map-making

Map-making

Harlequin includes a routine that applies the destriping algorithm to time-ordered data, producing a maximum-likelihood map of the intensity and polarization signal.

The destriping algorithm used in the code is described in the paper Destiping CMB temperature and polarization maps, Kurki-Suonio et al., A/A 506, 1511-1539 (2009), and the source code closely follows the terminology introduced in that paper.

The destriping algorithm is effective in removing 1/f noise originated within the detectors of an instrument, provided that the noise is uncorrelated among detectors. It requires the user to specify a baseline, i.e., the maximum time span in which the noise can be assumed to be uncorrelated (i.e., white).

Since the destriper is effective only when much data is available, it is often the case that the input data to be fed to the algorithm is larger than the available memory on a machine. In this case, the destriper can take advantage of a distributed-memory environment and of the MPI libraries, if they have been loaded before Harlequin, like in the following example:

import MPI
import Harlequin  # Ok, use MPI whenever possible

# ...

You can check if MPI is being used with the function use_mpi.

Destriping is based on a datatype, DestripingData, and on the function destripe!. To use the destriper, you create an object of type DestripingData and then call destripe! on it. The TOD must be split in a set of observations, using the datatype Observation; we will see how to use them in the section Splitting a TOD into observations.

Splitting a TOD into observations

mutable struct Observation{T <: Real}

An observation is a sequence of time-ordered data (TOD) that has been acquired by some detector. It implements the following fields:

  • time: an array of N elements, representing the time of the sample. Being defined as an AbstractArray, a range (e.g., 0.0:0.1:3600.0) can be used to save memory
  • pixidx: an array of N elements, each being a pixel index in a map. In no way the Healpix pixelization scheme is enforced; in fact, the map must not even be spherical.
  • psi_angle: an array of N elements, each containing the orientation of the polarization angle with respect to some fixed reference system (e.g., the Ecliptic plane). The angles must be in radians.
  • tod: an array of N elements, containing the actual data measured by the instrument and already calibrated to some physical units (e.g., K_CMB)
  • sigma_squared: an array of N elements, containing the squared RMS of each sample. To save memory, you should usually use a RunLengthArray here.
  • flagged: a Boolean array of N elements, telling whether the sample should be discarded (true) or not (false) during the map-making process.
  • name: a string representing the detector. This is used only for debugging purposes.

To create an observation, the constructor takes keyword arguments instead of parameters; in this way, the code should be more readable. Also, sensible defaults will be provided for the missing fields:

# We do not specify "sigma_squared" nor "flagged", so they will be
# initialized to a vector of ones and a vector of `false`
obs = Observation{Float64}(
    time = 0.0:0.1:3600.0,
    pixidx = 1:3601,
    psi_angle = zeros(3601),
    tod = randn(3601),
)
source

High-level functions

mutable struct DestripingData{T <: Real, O <: Healpix.Order}

Structure containing the result of a destriping operation. It contains the following fields:

  • skymap: a PolarizedMap containing the maximum-likelihood map
  • hitmap: a map containing the weights of each pixel (this is not the hit count, as it is normalized over the value of $σ^2$ for each sample in the TOD)
  • nobs_matrix: an array of NobsMatrixelement objects, which can be used to determine which pixels were the most troublesome in the reconstruction of tehe I/Q/U components
  • baselines: the computed baselines. These must be kept as a list of RunLengthArray objects
  • stopping_factors: sequence of stopping factors, one for each iteration of the CG algorithm
  • threshold: upper limit on the tolerable error for the Conjugated Gradient method (optional, default is 1e-9). See calc_stopping_factor.
  • max_iterations: maximum number of allowed iterations for the Conjugated Gradient algorithm (optional, default is 1000).
  • use_preconditioner: a Boolean flag telling whether a preconditioner should be used or not (optional, default is true). Usually you want this to be true.

The structure provides only one constructor, which accepts the following parameters:

  • nside: resolution of skymap and hitmap

  • obs_list: vector of Observation; it is used to determine which pixels in the map are going to be observed

  • runs: list of vectors (one for each element in obs_list), containing the number of samples to be included in each baseline for each observation

The constructor accepts the following keywords as well:

  • threshold (default: 1e-9)

  • max_iterations (default: 1000)

  • use_preconditioner (default: true)

The function destripe! use DestripingData to return the result of its computation.

source
Harlequin.destripe!Function.
destripe!(obs_list, destriping_data::DestripingData{T,O}; comm, unseen, callback) where {T <: Real, O <: Healpix.Order}

Apply the destriping algorithm to the TOD in obs_list. The result is returned in destriping_data, which is of type DestripingData and contains the baselines, I/Q/U maps, weight map, etc.

The parameters must satisfy the following constraints:

The optional keyword have the following meaning and default value:

  • comm is a MPI communicator object, or nothing is MPI is not used.
  • unseen is the value to be used for pixels in the map that have not been observed. The default is missing; other sensible values are nothing and NaN.
  • callback is either nothing (the default) or a function that is called before the CG algorithm starts, and then once every iteration. It can be used to monitor the progress of the destriper, or to save intermediate results in a database or a file. The function receives destriping_data as its parameter

The function uses Julia's Logging module. Therefore, you can enable the output of diagnostic messages as usual; consider the following example:

global_logger(ConsoleLogger(stderr, Debug))

# This command is going to produce a lot of output…
destripe!(...)
source
Harlequin.use_mpiFunction.
use_mpi()

Return true if Harlequin is taking advantage of MPI, false if not.

source

Low-level functions

The following functions are listed here for reference. You are not expected to use them, unless you want to debug the destriper or dig into its internals.

update_nobs!(nobs::NobsMatrixElement{T}; threshold = 1e-7)

This function makes sure that all the elements in nobs are synchronized. It should be called whenever the field nobs.m (matrix M_p in Eq. 10 of KurkiSuonio2009) has changed.

source
update_nobs_matrix!(nobs_matrix::Vector{NobsMatrixElement{T}}, psi_angle, sigma_squared, pixidx, flagged)

Apply Eq. (10) of KurkiSuonio2009 iteratively on the samples of a TOD to update matrices $M_p$ in nobs_matrix. The meaning of the parameters is the following:

  • nobs_matrix is the structure that gets updated by this function
  • psi_angle is an array of N elements, containing the polarization angles (in radians)
  • sigma_squared is an array of N elements, each containing the value of σ^2 for the samples in the TOD
  • pixidx is an array of N elements, containing the pixel index observed by the TOD
  • flagged is a Boolean array of N elements; true means that the sample in the TOD should not be used to produce the map. This can be used to produce jackknives and to neglect moving objects in the TOD
source
compute_nobs_matrix!(nobs_matrix::Vector{NobsMatrixElement}, observations::Vector{Observation{T}})

Initialize all the elements in nobs_matrix so that they are the matrices M_p in Eq. (10) of KurkiSuonio2009. The TOD is taken from the list of observations in the parameter observations.

source
update_binned_map!(vec, skymap::PolarizedMap{T, O}, hitmap::Healpix.Map{T, O}, obs::Observation{T}; comm=nothing, unseen=NaN) where {T <: Real, O <: Healpix.Order}
update_binned_map!(skymap::PolarizedMap{T, O}, hitmap::Healpix.Map{T, O}, obs::Observation{T}; comm=nothing, unseen=NaN) where {T <: Real, O <: Healpix.Order}
update_binned_map!(skymap::PolarizedMap{T, O}, hitmap::Healpix.Map{T, O}, obs_list::Vector{Observation{T}}; comm=nothing, unseen=NaN) where {T <: Real, O <: Healpix.Order}

Apply Eqs. (18), (19), and (20) to compute the values of I, Q, and U from a TOD, and save the result in skymap and hitmap.

The three versions differ for the source of the TOD calibrated data:

  • if the vec parameter is present (first form), it will be used instead of obs.tod (second form)
  • instead of obs (a single Observation object), you can pass an array obs_list (third form). All the elements in this array will be used to update skymap and hitmap.
source
Harlequin.reset_maps!Function.
reset_maps!(d::DestripingData{T, O}) where {T <: Real, O <: Healpix.Order}

Set the skymap and the hitmap in d to zero. Nothing is done on the other fields.

source
compute_z_and_subgroup!(dest_baselines, vec, baseline_lengths, destriping_data::DestripingData{T, O}, obs::Observation{T}; comm=nothing, unseen=NaN) where {T, O <: Healpix.Order}
compute_z_and_subgroup!(dest_baselines, baseline_lengths, destriping_data::DestripingData{T, O}, obs::Observation{T}; comm=nothing, unseen=NaN) where {T, O <: Healpix.Order}

Compute the application of matrix $A$ in Eq. (25) in KurkiSuonio2009. The result is a set of baselines, and it is saved in dest_baselines (an array of elements whose type is T). The two forms differ only in the presence of the vec parameter: if it is not present, the value of obs.tod will be used.

This function only operates on single observations. If you want to apply it to an array of observations, use compute_z_and_group!.

source
compute_z_and_group!(dest_baselines, vectors, baseline_lengths, destriping_data::DestripingData{T, O}, obs_list::Vector{Observation{T}}; comm=nothing, unseen=NaN) where {T, O <: Healpix.Order}
compute_z_and_group!(dest_baselines, baseline_lengths, destriping_data::DestripingData{T, O}, obs_list::Vector{Observation{T}}; comm=nothing, unseen=NaN) where {T, O <: Healpix.Order}

Compute the application of matrix $A$ in Eq. (25) in KurkiSuonio2009. The result is a set of baselines, and it is saved in dest_baselines (an array of elements whose type is T). The two forms differ only in the presence of the vec parameter: if it is not present, the value of obs.tod will be used.

source
compute_residuals!(residuals, baseline_lengths, destriping_data, obs_list; comm=nothing, unseen=missing)

This function is used to compute the value of $A x - b$ at the beginning of the Conjugated-Gradient algorithm implemented in destripe!.

source
Harlequin.array_dotFunction.
array_dot(x, y; comm = nothing)

Compute the dot product between x and y, assuming that both are arrays of arrays.

source
calc_stopping_factor(r; comm=nothing)

Calculate the stopping factor required by the Conjugated Gradient method to determine if the iteration must be stopped or not. The parameter r is the vector of the residuals.

source
apply_offset_to_baselines!(baselines)

Add a constant offset to all the baselines such that their sum is zero.

source
calculate_cleaned_map!(obs_list, destriping_data; comm=nothing, unseen=missing)

Provided a set of baselines in baselines, this function cleans the TOD in obs_list and computes the I/Q/U maps, which are saved in destriping_data.

source