pipeline.infrastructure.utils

utils sub-package.

The utils package contains a set of utility classes and functions that are useful to the pipeline framework and to tasks manipulating pipeline framework objects, Python data types, and CASA data types.

The utils package is intended to be free of any task-specific logic. Code that assumes knowledge or logic beyond that of the task-independent framework should be housed in the h.common package (or hif.common, hifv.common, hsd.common, etc. as appropriate).

Modules:

utils: Core utility functions (imported first to prevent circular imports) caltable_tools: Caltable utilities casa_data: Utilities for handling CASA data structures conversion: Data conversion utilities diagnostics: Diagnostic and debugging tools framework: Pipeline framework utilities imaging: Image processing utilities math: Mathematical functions and algorithms positioncorrection: Position correction utilities parallactic_range: Parallactic range calculation utilties ppr: Pipeline processing request utilities sorting: Sorting algorithms and utilities weblog: Web logging utilities

Functions

absolute_path(name)	Return an absolute path of a given file.
ant_arg_to_id(ms_path, ant_arg, all_antennas)	Convert a string to the corresponding antenna IDs.
antenna_names_from_caltable(caltable)	Returns the antenna names from the specified caltable's ANTENNA table.
approx_equal(x, y[, tol])	Return True if two numbers are equal within the given tolerance.
are_equal(a, b)	Return True if the contents of the given arrays are equal.
build_refantignore([refantignore, ignorerefant])	Return a comma-separated string from refantignore and ignorerefant, ignoring empty strings.
calc_zd_pa(ra, dec, obs_long, obs_lat, date_time)	Computes the zenith distance and parallactic angle chi.
capture_output([file_path])
chan_freq_from_caltable(caltable, spw)	Returns the frequency (in GHz) of the specified spw channel in a caltable.
chan_selection_to_frequencies(img, selection)	Convert channel selection to frequency tuples for a given CASA cube.
change_stream_for_all_streamhandlers(new_stream)	Iterates over existing loggers and updates the stream of their StreamHandlers to the given new_stream.
check_ppr(pprfile)	Check PPR to make sure all tasks exist in the CASA task dictionary.
clear_time_cache()	Clear the time cache in the CASA measures tool.
collect_properties(instance[, ignore])	Return the public properties of an object as a dictionary
commafy(l[, quotes, multi_prefix, ...])	Convert the string list into the textual description.
compute_zenith_distance(field_direction, ...)	Calculate zenith distance for a field at a given time and observatory.
contains_single_dish(context)	Return True if the given context contains single-dish data.
convert_paths_to_basenames(command_string)	Convert all absolute and relative file paths in command string to basenames.
deduplicate(items)	Remove duplicate entries from a list, but preserve the order.
dequote(s)	Remove any kind of quotes from a string to facilitate comparisons.
dict_merge(a, b)	Recursively merge dictionaries.
do_wide_field_pos_cor(fitsname[, date_time, ...])	Applies mean wide field position correction to FITS WCS in place.
enable_fd_logs([interval_secs])	Log file descriptors to the CASA log every n seconds.
enable_memstats()
ensure_products_dir_exists(products_dir)
equal_to_n_digits(x, y[, numdigits])	Approximate equality check up to a given number of digits.
exec_func(fn, *args[, include_client])	Execute the same function on both client and worker (dask-worker or mpiserver) processes.
export_weblog_as_tar(context, products_dir, ...)
field_arg_to_id(ms_path, field_arg, all_fields)	Convert a string to the corresponding field IDs.
fieldname_clean(field)	Indicate if the fieldname is allowed as-is.
fieldname_for_casa(field)	Prepare field string to be used as CASA argument.
filter_intents_for_ms(ms, intents)	Filter string of comma-separated intents to keep only those that are present in the given MS.
find_ranges(data)	Identify numeric ranges in string or list.
flagged_intervals(vec)	Idendity isnads of ones in input array or list.
flatten(l)	Flatten a list of lists into a single list without pipelineaq.QAScore.
flatten_dict(d[, join, lift])
format_datetime(dt[, dp])	Convert a datetime to a formatted string representation.
format_timedelta(td[, dp])	Convert a timedelta to a formatted string representation.
freq_selection_to_channels(img, selection)	Convert frequency selection to channel tuples for a given CASA cube.
frequency_to_velocity(frequency, restfreq)	Convert frequency to radial velocity using radio convention.
from_mjd_to_datetime(mjd)	Convert a MJD float into a datetime
gen_hash(o)	Makes a hash from a dictionary, list, tuple or set to any level, that contains only other hashable types (including any lists, tuples, sets, and dictionaries).
get_ant_ids_from_caltable(caltable)	Returns a list of all unique antenna ids in the caltable
get_calfroms(context, vis[, caltypes])	Get the CalFroms of the requested type from the callibrary.
get_casa_quantity(value)	Wrapper around quanta.quantity() that handles None input.
get_casa_session_details()	Get the current CASA session details.
get_directory_size(directory)	Calculate the total size of a directory in megabytes (MB).
get_epoch_as_datetime(epoch)	Convert a CASA 'epoch' measure into a Python datetime.
get_field_identifiers(ms)	Maps numeric field IDs to field names.
get_file_md5(filename)	Return a readable hex string of the MD5 hash of a given file
get_filename_info(filename)	Get a string with information about the modification date and MD5 hash of a file
get_intervals(context, calapp[, spw_ids])	Get the integration intervals for scans processed by a calibration.
get_iso_mtime(filename)	Return the ISO 8601 datetime string corresponding to the modification time of a given file
get_logrecords(result, loglevel)	Get the logrecords for the result, removing any duplicates
get_num_caltable_polarizations(caltable)	Obtain number of polarisations from calibration table.
get_obj_size(obj[, serialize])	Estimate the size of a Python object.
get_object_info_string(ss_object[, ss_path])	Get the file information (MD5 hash and modification date) for a given Solar System object.
get_origin_input_arg(calapp, attr)	Get a single-valued task input argument from a CalApp.
get_products_dir(context)
get_qascores(result[, lo, hi])
get_receiver_type_for_spws(ms, spwids)	Return dictionary of receiver types for requested spectral window IDs.
get_row_count(table_name, taql)	Return the number of rows in the specified table that match the given TaQL query.
get_si_prefix(value[, select, lztol])	Obtain the best SI unit prefix option for a numeric value.
get_solar_system_model_files(ss_object[, ...])	Return the data files corresponding to a Solar System object
get_spectralspec_to_spwid_map(spws)	Returns a dictionary of spectral specs mapped to corresponding spectral window IDs for requested list of spectral window objects.
get_stats(image_name, metrics[, stokes])	Return a dict of requested statistics for the given image.
get_stokes(imagename)	Get the labels of all stokes planes present in a CASA image.
get_task_result_count(context[, taskname])	Count occurrences of a task result in the context.results list.
get_taskhistory_fromimage(imagename)	Retrieve past CASA/tclean() call parameters from the image history.
get_tracebacks(result)	Get the tracebacks for the result, from any failures that may have occurred during the task.
get_valid_url(env_var, default)	Fetches one or more URLs from an environment variable.
get_vis_from_plots(plots)	Get the name to be used for the MS from the given plots.
get_vlass_image_type(filename[, append_tt])	Determine the VLASS image type based on specific substrings in the filename.
glob_ordered(pattern, *args[, order])	Return a sorted list of paths matching a pathname pattern.
human_file_size(size_in_bytes)	Converts a file size in bytes to a human-readable string format.
ignore_pointing(vis)	A context manager to ignore pointing tables of MSes during I/O operations.
import_module(name[, package])	Import a module.
imstat_items(image[, items, mask])	Extract desired stats properties (per Stokes) using ia.statistics().
intersect_ranges(ranges)	Compute intersection of ranges.
intersect_ranges_by_weight(ranges, delta, ...)	Compute intersection of ranges through weight arrays and a threshold.
invert_dict(input_dict)	Inverts a dictionary so that values become keys and keys become grouped in a list.
is_top_level_task()	Return True if the callee if executing as part of a top-level task.
list_to_str(value)	Convert list or numpy.ndarray into string.
merge_jobs(jobs, task[, merge, ignore])	Merge jobs that are identical apart from the arguments named in ignore.
merge_ranges(ranges)	Merge overlapping and adjacent ranges and yield the merged ranges in order.
merge_td_columns(rows[, num_to_merge, ...])	Merge HTML TD columns with identical values using rowspan.
merge_td_rows(table)	Merge HTML TD rows with identical values using colspan.
mjd_seconds_to_datetime(mjd_secs)	Convert list of MJD seconds to a list of equivalent datetime objects.
mkdir_p(path)	Emulate mkdir -p functionality.
natural_sort(input_list)	Sort a list in natural order, eg.
natural_sort_key(x[, _nsre])	Key to order in natural order with the sort function
nchan_from_caltable(caltable, spw)	Returns the number of channels of the specified spw in a caltable.
nested_dict()
numeric_sort(input_list)	Sort a list numerically, eg.
numeric_sort_key(s[, _nsre])	Key to order in numeric order with the sort function.
obs_long_lat(observatory)	Return longitude and latitude values of the given observatory.
obs_midtime(start_time, end_time)	Returns the mid time in a CASA measures dictionary.
open_with_lock(filename[, mode])	Open a file with an exclusive lock.
ous_parallactic_range(mses, field_name, intent)	Get the parallactic angle range across all measurement sets for field when observed with the specified intent.
pickle_copy(original)
pickle_load(fileobj)
place_repr_source_first(itemlist, repr_source)	Place representative source first in a list of source names or tuples with source name as first tuple element.
plotms_iterate(jobs_and_wrappers[, ...])	Iterate multiple plotms job requests
plots_to_html(plots[, title, alt, caption, ...])	Convert a list of plots to HTML snippets.
predict_kernel(beam, target_beam[, pstol, patol])	Predict the required convolution kernel to each a target restoring beam.
range_to_list(arg)	Expand a numeric range expressed in CASA syntax to the list of integer.
record_to_quantity(record)	Convert a CASA record to an Astropy quantity.
relative_path(name[, start])	Retun a relative path of a given file with respect a given origin.
remove_trailing_string(s, t)	Remove a trailing string if it exists.
request_omp_threading([num_threads])	A context manager to override the session-wise OMP threading setting on CASA MPI client.
reset_logfiles([casacalls_logfile, ...])	Reset CASA/Pipeline logfiles for the current CASA/Pipeline session.
round_half_up(value[, precision])	Provides the "round to given precision with ties going away from zero" behaviour that was the default Python2 rounding behavior.
round_up(value[, precision])	Round to given precision away from zero.
safe_split(fields)	Split a string containing field names into a list.
scale_uv_range(ms)	Return the UV range that captures the inner half of the data.
set_nested_dict(dct, keys, value)	Set a hierarchy of dictionaries with given keys and value for the lowest level key.
shutdown_plotms()	Shutdown the existing plotms process in the current CASA session.
split_spw(spw_string)	Splits an SPW string on '#' characters while retaining them, and inserts a <br/> before the 'ALMA' portion to improve readability.
spw_arg_to_id(ms_path, spw_arg, all_spws)	Convert a string to spectral window IDs and channels.
spw_intersect(spw_range, line_regions)	This utility function takes a frequency range (as numbers with arbitrary but common units) and computes the intersection with a list of frequency ranges defining the regions of spectral lines.
string_to_val(s)	Convert a string to a Python data type.
table_to_html(table[, tableclass, rotate])	Convert a astropy.table.Table object to an HTML table snippet.
task_depth()	Get the number of executing tasks currently on the stack.
to_CASA_intent(ms, intents)	Convert pipeline intents back to the equivalent intents recorded in the measurement set.
to_pipeline_intent(ms, intents)	Convert CASA intents to pipeline intents.
total_time_on_source(scans)	Return the total time on source for the given Scans.
total_time_on_target_on_source(ms[, ...])	Return the nominal total time on target source for the given MeasurementSet excluding OFF-source integrations (REFERENCE).
update_beams_dict(dct, udct)	Update a beams dictionary.
update_sens_dict(dct, udct)	Update a sensitivity dictionary.
validate_url(url)	Validates whether a given URL is properly formatted.
velocity_to_frequency(velocity, restfreq)	Convert radial velocity to frequency using radio convention.
work_directory(workdir[, create, cleanup, ...])	A context manager to temporarily change the working directory.
write_errorexit_file(path, root, extension)

Classes

DirectionDict	Type for CASA direction measure dictionary (from me tool).
EpochDict	Type for CASA epoch measure dictionary (from me tool).
IERSInfo([iers_path, load_on_creation])	Class to store, retrieve and process the information from the IERS geodetic tables
LongLatDict	Type for latitude/longitude pair.
OrderedDefaultdict(*args, **kwargs)	This class behaves as defaultdict from the collections module but maintaining the order of insertion.
PositionDict	Type for CASA position measure dictionary (from me tool).
QuantityDict	Type for CASA quantity dictionary (from qa tool).

Variables

IERS_TABLES_PATH	str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str
SOLAR_SYSTEM_MODELS_PATH	str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str