VLASS-SE-CUBE imaging workflow¶

This documentation describes the VLASS Coarse Cube Imaging Pipeline (CCIP) workflow.

In general, the CCIP workflow uses similar imaging heuristics as the VLASS-SE-CONT workflow, with two major differences:

The CCIP workflow skips several selfcal/masking-related stages in VLASS-SE-CONT, and replaces them with a single equivalent hifv_restorepims() stage that utilizes the products from VLASS-SE-CONT (e.g. reimaging_esource.tgz and tier1/tier2 masks). hifv_restorepims() restores the flag, corrected, and weight columns in a Per-Image Measurement Set (PIMS) to the same state of the final imaging-ready MS in the VLASS-SE-CONT production run.
The imaging operation will go through individual specified SPW groups and generate/analyze/export full-Stokes images, using slightly less expensive algorithms (mosaic gridder, nterm=1). However, the per-iteration imaging masks and sequence of each spw group are the same as that used in the final imaging stage (vlass_stage=3) of VLASS-SE-CONT.

Task Changes¶

hifv_restorepims¶

The task will perform the following operations:

Fill the model column in a PIMS using the selfcal model image from VLASS-SE-CONT
Restore flags from ${FSID}_split_split.ms.flagversions/statwt_1
Re-run statwt using the same settings from the VLASS-SE-CONT production run
Reapply selfcal table

The task will try to extract ancillary files (e.g. selfcal table/model_image and flag backup table) required for the above operations from reimagig_resources.tgz generated by VLASS-SE-CONT. It will also extract/verify mask files needed for the VLASS-SE-CUBE imaging sequence. If any required resource is missing, Pipeline will throw an exception and exit from the workflow. In that case, the weblog of hifv_restorepims will still render and the exception message from the weblog can help identify the missing file(s).

hif_editimlist (PIPE-1401)¶

hif_editimlist will be executed twice in the VLASS Coarse Cube workflow. At Stage 2, it will use SEIP_parameter.list from VLASS-SE-CONT to re-create the single-epoch continuum imaging heuristics/context for hif_restorepims. At Stage 6, it will use CCIP_parameter.list to create the cube imaging target list used by hif_makeimages (one target per spectral plane). This is done with editmode='replace', so the target list from stage2 will be reset.

For the “cube” mode of hif_editimlist, i.e. imaging_mode=‘VLASS-SE-CUBE’ in the task input at Stage 6:

The spw parameter is expected to be a list of spw so that spws or spwgroups can be used for making cubes e.g.,
- spw=['2','3','4','5',...,'17']
- spw=['2,3,4','5,6,7'] Plese note that a channel-wise selection (e.g. spw=['2:0~15','2:16~31']) and a range-syntax selection (e.g. spw=['1~4','5~7'] are not allowed.
the reffreq parameter for tclean will be explicitly set to the mean frequency of each spw/spwgroup.
If the spw-selected data is flagged above a percentage threshold (currently, hardcoded to 100%, i.e., completely flagged ), then that spw/spwgroup will not be added to the imaging target list, and a warning will be issued on the weblog.
The weblog is improved to show imagename/mask/spw/frereq of each imaging target in separate rows.

hif_makeimages (PIPE-1401)¶

hif_makeimage runs under the “vlass-cube” mode (imaging_mode=‘VLASS-SE-CUBE’) in the Coarse Cube worflow.

For a Pipeline parallel run, imaging of individual targets (i.e. different spws or spw groups) will be executed using the Pipeline tier-0 parallel scheme, which is built upon the CASA parallel framework. The imaging operations of targets will be put into a FIFO queue constructed from the available MPIserver (num_server=num_proc-1) and commanded by the MPIclient. `hif_makeimage will wait for the completion of all imaging operations and then move forward to render weblog.

Besides the tier-0 parallelization, the imaging heuristics of VLASS-SE-CUBE is similar to that of VLASS-SE-CONT at its last imaging stage (vlass_stage=3), except for the following major differences:

CASA/tclean is run in the nterms=1 mode, and a Stokes IQUV cube is generated in a single tclean call.
tclean input mask (iter1/iter2 from QL/combined mask of SE), are copied as *.iter1/2.cleanmask for each target.
Only *.cleanmask/.image/.residual are preserved for iter1/iter2 to reduce storage I/O and disk space usage.
If imaging fails on one target, the weblog will still present the successful targets and show the failed one(s) in the top error message banner.
The weblog uses the VLASS-cube-specific template, which has a grid layout with four Stokes planes in columns and different imaging targets in rows.
The tclean summary plot shows the peak residual model flux of each Stokes planes.
A workaround for CAS-13401 is implemented: the beam information is missing in residual/restored image from CASA/tclean when Stokes=‘IQUV’.

hif_makermsimages¶

hif_makecutoutimages¶

hifv_analyzestokescubes (PIPE-1356)¶

This task performs analysis on cutout images of the full Stokes imaging products and generates two groups of plots:

Fractional Stokes U vs. Q at different frequencies
- This analysis is done at two locations: the peak of the Stokes I map, and the peak of the linearly polarized intensity map.
- I, Q, and U are measured from a 3x3 pixel box average centered at the peak pixel.
- The peak search is restricted in the region defined by a joint mask from the tclean iter2 (from the SE tier2-combined) mask and iter3 mask (pbmask=0.4 from the current heuristics).
Median of .rms image from individual Stokes planes, as a function of frequency.
- A scaled/normalized VLA SEFD is overlaid for comparisons.

hifv_exportvlassdata (PIPE-1434)¶

hifv_exportvlassdata has been updated to include the following new features for the Coarse Cube data.

The full-Stokes cube (IQUV) per spectral window is split into the IQU and V images before being exported into FITS files in products/.
The position corrections applied to the FITS image header is now frequency-dependent: each plane will get a slightly different position correction (see notes in PIPE-1434 or Eq.9 of VLASS memo 14.
Besides the full resolution Stokes cubes, hifv_exportvlassdata() provides another set of FITS “pbcor” images which were smoothed to the smallest common beam in the dataset and were regridded to have the same WCS (with the suffix .com). However, these images are not included in the manifest and will not be archived.
- the common beam is the smallest possible beam to which all the beams in the image set can be convolved to, determined by ia.commombeam() via a pipeline utils wrapper function (pipeline.infrastructure.utils.imaging.predict_kernel).
- the “smooth” operation will only proceed if the prediction convolution kernel has an FWHM major larger than 0.2 times the diagonal length of a pixel.
- The regridded images has a reference pixel centered at the nomimal phasecenter.
- the “regrid” operation will only proceed if the position correction introduces a crval shift larger than 0.1 times the diagonal length of a pixel.
- Depending on whether an image meets the smooth/regrid operation threshold as stated above, one image from the common beam/WCS sets might still show a slightly different deviation in CRVAL and BMAJ/BMIN (less than 0.1 and 0.2 of the pixel diagonal length).
The SEIP_parameter.list (for stage 2) and CCIP_parameter.list (for stage 6) files are exported to the products/ directory and included in pipeline_manifest.xml.