Full APIs (autosummary)

pipeline

Pipeline software package.

Pipeline Tasks (autosummary)

pipeline.h.cli	Generic Tasks
pipeline.hif.cli	Interferometry Generic Tasks
pipeline.hifa.cli	Interferometry ALMA Tasks
pipeline.hifv.cli	Interferometry VLA Tasks
pipeline.hsd.cli	Single Dish ALMA Tasks
pipeline.hsdn.cli	Single Dish Nobeyama Tasks

Pipeline domain/context (autosummary)

pipeline.domain
pipeline.infrastructure.launcher	Pipeline initialization and context management.

Pipeline domain/infrastructure modules (automodapi)

pipeline.domain

Classes

Antenna(antenna_id, name, station, position, ...)	A logical representation of an antenna.
AntennaArray(name, position, antennas)	A logical representation of the antenna array.
DataDescription(dd_id, spw, pol_id)	A logical representation of an entry in the DATA_DESCRIPTION table.
DataTable
DataType(*values)	Enumeration of data types used in the pipeline.
Field(field_id, name, source_id, time, direction)	A logical representation of a field in a MeasurementSet.
FluxMeasurement(spw_id, I[, Q, U, V, spix, ...])	FluxMeasurement is a logical representation of a flux measurement.
MeasurementSet(name[, session])	A logical representation of a MeasurementSet (MS).
ObservingRun()	A logical representation of an observing run.
Polarization(pol_id, num_corr, corr_type, ...)	A logical representation of a polarization.
Scan([id, antennas, intents, fields, ...])	A logical representation of a single scan.
Source(source_id, name, direction, ...)	A logical representation of an astronomical source.
SpectralWindow(spw_id, name, spw_type, ...)	A logical representation of a spectral window (spw).
SpectralWindowWithChannelSelection(subject, ...)	Wrapper for SpectralWindow that includes channel selection in the ID.
State(state_id, obs_mode)	A logical representation of rows in the STATE table.

pipeline.infrastructure.launcher

Pipeline initialization and context management.

This module provides classes for initializing the pipeline and managing pipeline state, including loading and saving context from/to disk.

Classes:

Context: Container for all pipeline state during execution. Pipeline: Entry point for initializing and managing the pipeline.

Classes

Context([name])	A class holds all pipeline state for a given execution.
ContextCreatedEvent([context_name, output_dir])	Emitted when a pipeline Context is created.
ContextResumedEvent([context_name, output_dir])	Emitted when a pipeline Context is resumed.
Pipeline([context, loglevel, ...])	Entry point for initializing the pipeline.

Variables

LOG	Instances of the Logger class represent a single logging channel.
MAX_CASA_REVISION
MIN_CASA_REVISION	Built-in mutable sequence.
TYPE_CHECKING	bool(x) -> bool
annotations
current_task_name

pipeline.infrastructure.project

Classes

LoggingModificationListener()	ModificationListener that logs INFO messages when the target is modified.
ModificationListener()	Interface for listener classes that want to be notified when an object property changes.
ModificationPublisher()	Base class that publishes an event to registered listeners when public properties of an instance of this class are set or deleted.
PerformanceParameters([...])
ProjectStructure([ous_entity_type, ...])
ProjectSummary([proposal_code, ...])

pipeline.infrastructure.callibrary

Functions

ant_add(td1, td2, join)
ant_sub(td1, td2, join)
applycaltocallib(filename[, append, field, ...])
consolidate_calibrations(all_my_calapps)	Consolidate a list of (CalTo, [CalFrom..]) 2-tuples into a smaller set of equivalent applications by consolidating their data selection arguments.
contiguous_sequences(l)	Generate contiguous sequences of numbers from a list.
copy_calapplication(calapp[, origin])	Copy a CalApplication, overwriting any CalTo or CalFrom values with the given override values.
create_data_reducer(join)	Return a function that creates a new TimestampedData object containing the result of executing the given operation on two TimestampedData objects.
create_interval_tree(a)	Create an IntervalTree containing a set of Intervals.
create_interval_tree_for_ms(ms)	Create a new IntervalTree fitted to the dimensions of a measurement set.
create_interval_tree_nd(intervals, value_fn)	Create a multidimensional IntervalTree.
data_selection_contains(proposed, calto_args)	Return True if one data selection is contained within another.
defrag_interval_tree(tree)	Condense an IntervalTree by consolidating fragmented entries with the same value into contiguous Intervals.
expand_calstate(calstate)	Convert an IntervalCalState into the equivalent consolidated list of (CalToArgs, [CalFrom..]) 2-tuples.
expand_calstate_to_calapps(calstate)	Convert an IntervalCalState into a list of (CalTo, [CalFrom..]) tuples.
expand_interval(interval, calto_args, calto_fn)	Convert an Interval into the equivalent list of (CalTo, [CalFrom..]) 2-tuples.
expand_intervaltree(tree, convert_fns, calto_fn)	Convert an IntervalTree into the equivalent list of (CalTo, [CalFrom..]) 2-tuples.
field_add(td1, td2, join)
field_sub(td1, td2, join)
fix_cycle0_data_selection(context, calstate)
get_calstate_shape(ms)	Get an IntervalTree shaped to the dimensions of the given measurement set.
get_calto_from_inputs(inputs)	Get a CalTo data selection object based on the state of an Inputs object
get_id_to_field_fn(ms_to_id_to_field)	Return a function that can convert field IDs to a field name.
get_id_to_intent_fn(id_to_intent)	Return a function that can convert intent IDs to a string intent.
get_intent_id_map(ms)	Get the mapping of intent ID to string intent for a measurement set.
get_matching_calstate(context, calstate, ...)	Return an IntervalCalState contain calibrations in the input IntervalCalState that match the predicate function.
get_min_max(l[, keyfunc])
intent_add(td1, td2, join)
intent_sub(td1, td2, join)
interval_to_set(interval)	Get all the indices covered by an Interval.
merge_contiguous_intervals(tree)	Merge contiguous Intervals with the same value into one Interval.
merge_intervaltrees(on_intersect)	Return a function that merges two IntervalTrees, executing a function on the intersecting Interval ranges in the resulting merged IntervalTree.
merge_lists([join_fn])	Return a function that merge two lists by calling the input operation on the two input arguments.
ranges(lst)
safe_join(vals[, separator])
sequence_to_casa_range(seq)
sequence_to_range(l)
set_calstate_marker(calstate, marker)	Return a copy of a calstate, modified so that TimeStampedData objects in the final leaf node are annotated with the given marker object.
spw_add(td1, td2, join)
spw_sub(td1, td2, join)
trim(tree, ranges)	Return an IntervalTree trimmed to the specified ranges.
trim_nd(tree, selection)	Return an IntervalTree with each dimension trimmed to the specified set of ranges.
trim_to_valid_data_selection(calstate[, vis])	Trim an IntervalCalState to the shape of valid (present) data selections.
unit(x)

Classes

CalAppOrigin(task, inputs)
CalApplication(calto, calfrom[, origin])	CalApplication maps calibration tables and their application arguments to a target data selection, encapsulated as CalFrom and CalTo objects respectively.
CalFrom([gaintable, gainfield, interp, ...])	CalFrom represents a calibration table and the CASA arguments that should be used when applying that calibration table.
CalLibrary
CalState
CalTo([vis, field, spw, antenna, intent])	CalTo represents a target data selection to which a calibration can be applied.
CalToArgs(vis, spw, field, intent, antenna)
CalToIdAdapter(context, calto)	CalToIdAdapter is an adapter class for CalTo that return some of its attributes as lists of IDs/names, instead of as the CASA-style string argument.
CalToIntervalAdapter(context, calto)
IntervalCalLibrary(context)	IntervalCalLibrary is the root object for the pipeline calibration state.
IntervalCalState()	IntervalCalState is a data structure used to map calibrations for all data registered with the pipeline.
TimestampedData(time, data[, marker])

Variables

CYCLE_0_END_DATE	datetime(year, month, day[, hour[, minute[, second[, microsecond[,tzinfo]]]]])
LOG	Instances of the Logger class represent a single logging channel.
TYPE_CHECKING	bool(x) -> bool
annotations
tsd_accessor	Return a callable object that fetches the given attribute(s) from its operand.

pipeline.infrastructure.imagelibrary

Classes

ImageItem(imagename, sourcename, spwlist, ...)
ImageLibrary()
ImageMetadata()

Variables

LOG	Instances of the Logger class represent a single logging channel.

pipeline.infrastructure.vdp

vdp is a pipeline framework module that contains classes to make writing task Inputs easier.

• InputsContainer lets task implementations operate within the scope of a single measurement set, even if the pipeline run contains multiple data sets.

• VisDependentProperty is a reworking of pipeline properties to reduce the amount of boilerplate code required to implement an Inputs class.

Implementation details:

See the documentation on the classes, particularly VisDependentProperty, for detailed information on how the framework operates.

Examples

There are three common scenarios that use VisDependentProperty. The following examples show each scenario for an Inputs property belonging to an Inputs class that extends vdp.StandardInputs.

1. To provide a default value that can be overridden on a per-MS basis. Use the optional 'default' argument to VisDependentProperty, eg:

   myarg = VisDependentProperty(default='some value')

2. For more sophisticated default values, e.g., a default value that is a function of other data or properties, use the @VisDependentProperty decorator. A class property decorated with @VisDependentProperty should return the default value for that property. The function will execute in the scope of a single measurement set, i.e., at the time it is called, vis is set to exactly one value. The function will be called to provide a default value for any measurement set that does not have a user override value.

   @VisDependentProperty def myarg():

   # do some processing then return the calculated value return 'I am a custom property for measurement set %s' % self.vis

3. Convert or validate user input before accept it as an Inputs argument. Use the @VisDependentProperty.convert decorator, possibly alongside the getter decorator as above.

   @VisDependentProperty def myarg():

   # this will return 100 - but only if the user has not supplied # an override value! return 100

   @VisDependentProperty.convert def myarg(user_input):

   # convert then return the user input which is provided as an # argument to the convert decorator. The converted value will be # returned for all subsequent 'gets'. return int(user_input)

Classes

InputsContainer(task_cls, context, *args, ...)	InputsContainer is the top-level container object for all task Inputs.
ModeInputs(context[, mode])	ModeInputs is a facade for Inputs of a common task type, allowing the user to switch between task implementations by changing the 'mode' parameter.
StandardInputs()
VisDependentProperty([fdefault, fconvert, ...])	VisDependentProperty is a Python data descriptor that standardises the behaviour of pipeline Inputs properties and lets them create default values more easily.

pipeline.h*.tasks modules (autmodapi)

pipeline.h.tasks

Classes

Applycal(inputs)
ExportData(inputs)	ExportData is the base class for exporting data to the products subdirectory.
ImportData(inputs)
MsSplit(inputs)
RestoreData(inputs)	RestoreData is the base class for restoring flagged and calibrated data produced during a previous pipeline run and archived on disk.
Tsyscal(inputs)
Tsysflag(inputs)

Class Inheritance Diagram

pipeline.hif.tasks

Classes

Analyzealpha(inputs)
Antpos(inputs)
Bandpass
BandpassMode(inputs)
ChannelBandpass(inputs)
CheckProductSize(inputs)
Correctedampflag(inputs)
Editimlist(inputs)
FindCont(inputs)
Fluxscale(inputs)
GSplineGaincal(inputs)
GTypeGaincal(inputs)
Gaincal
GaincalMode(inputs)
IFApplycal(inputs)
KTypeGaincal(inputs)
Lowgainflag(inputs)
MakeImList(inputs)
MakeImages(inputs)
Makecutoutimages(inputs)
Makermsimages(inputs)
Mstransform(inputs)
PhcorBandpass(inputs)
Polarization(inputs)
PolcalWorker(inputs)
Rawflagchans(inputs)
RefAnt(inputs)
Selfcal(inputs)
SerialIFApplycal(inputs)
SerialRefAnt(inputs)
SetModels(inputs)
Setjy(inputs)
Tclean(inputs)
Transformimagedata(inputs)
UVcontSub(inputs)	UVcontSub class for parallelization.

Class Inheritance Diagram

pipeline.hifa.tasks

Classes

ALMAAntpos(inputs)
ALMAExportData(inputs)
ALMAImportData(inputs)	ALMAImportData class for parallelization.
ALMAPhcorBandpass(inputs)
ALMARestoreData(inputs)
Bandpassflag(inputs)
BpSolint(inputs)
DiffGaincal(inputs)
FlagDeterALMA(inputs)	FlagDeterALMA class for parallelization.
FlagTargetsALMA(inputs)
FluxcalFlag(inputs)
GaincalSnr(inputs)
GcorFluxscale(inputs)
Gfluxscaleflag(inputs)
ImagePreCheck(inputs)
LockRefAnt(inputs)
Polcal(inputs)
Polcalflag(inputs)
Renorm(inputs)
SessionALMAPhcorBandpass(inputs)
SessionGcorFluxscale(inputs)
SessionRefAnt(inputs)
SpwPhaseup(inputs)
Targetflag(inputs)	ALMA Targetflag class for parallelization.
TimeGaincal(inputs)
TsysFlagContamination(inputs)
Tsysflag(inputs)
UnlockRefAnt(inputs)
Wvrgcal(inputs)
Wvrgcalflag(inputs)

Class Inheritance Diagram

pipeline.hifv.tasks

Classes

Analyzestokescubes(inputs)
Applycals(inputs)
Checkflag(inputs)
Circfeedpolcal(inputs)
Exportvlassdata(inputs)
Finalcals(inputs)	Class for the finalcals pipeline task.
Fixpointing(inputs)
FlagDeterVLA(inputs)
Flagcal(inputs)
Fluxboot(inputs)
GainCurves(inputs)
Hanning(inputs)	Class for the hifv_hanning pipeline smoothing task.
Opcal(inputs)
Pbcor(inputs)
PlotSummary(inputs)
Priorcals(inputs)	Class for the Priorcals pipeline task.
Restorepims(inputs)
Rqcal(inputs)
Selfcal(inputs)
Solint(inputs)	Class for the solint pipeline task.
Statwt(inputs)
Swpowcal(inputs)
Syspower(inputs)
Targetflag(inputs)
TecMaps(inputs)
VLAExportData(inputs)
VLAImportData(inputs)	VLAImportData class for parallelization.
VLARestoreData(inputs)
VLASetjy(inputs)
VlaMstransform(inputs)
Vlassmasking(inputs)	Class for the hifv_vlassmasking pipeline task.
semiFinalBPdcals(inputs)	Class for the semiFinalBPdcals pipeline task.
testBPdcals(inputs)	Class for the testBPdcals pipeline task.

Class Inheritance Diagram

pipeline.hsd.tasks

Classes

FlagDeterALMASingleDish(inputs)
SDATMCorrection(inputs)	Parallel implementation of offline ATM correction task.
SDApplycal(inputs)
SDBLFlag(inputs)
SDBaseline(inputs)	Baseline subtraction task.
SDExportData(inputs)	A class for exporting single dish data to the products subdirectory.
SDImaging(inputs)	SDImaging processing class.
SDImportData(inputs)	SDImportData class for parallelization.
SDK2JyCal(inputs)	Generate calibration table of Jy/K factors.
SDRestoreData(inputs)	Restore flagged and calibrated data produced during a previous pipeline run and archived on disk.
SDSkyCal(inputs)	Class to generate sky calibration table.
SDTsysflag

Class Inheritance Diagram

pipeline.hsdn.tasks

Classes

NROExportData(inputs)	A class for exporting Nobeyama data to the products subdirectory.
NROImportData(inputs)	NRO Data import execution task.
NRORestoreData(inputs)	Restore flagged and calibrated data produced during a previous pipeline run and archived on disk.

Class Inheritance Diagram

Inheritance Diagrams for Pipeline Task/Inputs/Results Classes

ImportData task classes as an example

Task Classes

|taskclasses_diagram|

Inputs Classes

Results Classes