Run MeerSOLAR Pipeline

Basic run

To run MeerSOLAR pipeline, with default settings for full analysis, run the following command from terminal. Work directory needs not to be created before hand, but the path where it will be created should exist.

run_meersolar </path/to/measurement_set> --workdir </path/to/work_directory> --outdir </path/to/output_product_directory>

Advanced run

For advanced run, user is requested to first check the parameters of run_meersolar.

run_meersolar -h

Click here to see parameters

usage: run_meersolar [-h] --workdir WORKDIR --outdir OUTDIR [--solint SOLINT]
                     [--cal_uvrange CAL_UVRANGE] [--no_polcal]
                     [--target_scans [TARGET_SCANS ...]]
                     [--freqrange FREQRANGE] [--timerange TIMERANGE]
                     [--image_freqres IMAGE_FREQRES]
                     [--image_timeres IMAGE_TIMERES] [--pol POL]
                     [--minuv MINUV] [--weight WEIGHT] [--robust ROBUST]
                     [--no_multiscale] [--clean_threshold CLEAN_THRESHOLD]
                     [--do_pbcor] [--no_apply_parang]
                     [--cutout_rsun CUTOUT_RSUN] [--no_solar_mask]
                     [--no_overlay] [--non_solar_data] [--non_ds]
                     [--do_forcereset_weightflag] [--no_noise_cal]
                     [--no_cal_partition] [--no_cal_flag] [--no_import_model]
                     [--no_basic_cal] [--do_sidereal_cor] [--no_selfcal_split]
                     [--no_selfcal] [--no_ap_selfcal] [--no_solar_selfcal]
                     [--no_target_split] [--no_applycal] [--no_apply_selfcal]
                     [--no_imaging] [--cpu_frac CPU_FRAC]
                     [--mem_frac MEM_FRAC] [--keep_backup]
                     [--no_remote_logger]
                     [--logger_alivetime LOGGER_ALIVETIME] [--n_nodes N_NODES]
                     msname

Run MeerSOLAR for calibration and imaging of solar observations.

options:
  -h, --help            show this help message and exit

###################
Essential parameters
###################:
  msname                Measurement set name
  --workdir WORKDIR     Working directory (default: None)
  --outdir OUTDIR       Output products directory (default: None)

###################
Advanced calibration parameters
###################:
  --solint SOLINT       Solution interval for calibration (e.g. 'int', '10s',
                        '5min', 'inf') (default: 5min)
  --cal_uvrange CAL_UVRANGE
                        UV range to filter data for calibration (e.g.
                        '>100klambda', '100~10000lambda') (default: )
  --no_polcal           Disable polarization calibration

###################
Advanced imaging parameters
###################:
  --target_scans [TARGET_SCANS ...]
                        List of target scans to process (space-separated, e.g.
                        3 5 7) (default: [])
  --freqrange FREQRANGE
                        Frequency range in MHz to select during imaging
                        (comma-seperate, e.g. '100~110,130~140') (default: )
  --timerange TIMERANGE
                        Time range to select during imaging (comma-seperated,
                        e.g. '2014/09/06/09:30:00~2014/09/06/09:45:00,2014/09/
                        06/10:30:00~2014/09/06/10:45:00') (default: )
  --image_freqres IMAGE_FREQRES
                        Output image frequency resolution in MHz (-1 = full)
                        (default: -1)
  --image_timeres IMAGE_TIMERES
                        Output image time resolution in seconds (-1 = full)
                        (default: -1)
  --pol POL             Stokes parameter(s) to image (e.g. 'I', 'XX', 'RR',
                        'IQUV') (default: IQUV)
  --minuv MINUV         Minimum baseline length (in wavelengths) to include in
                        imaging (default: 0)
  --weight WEIGHT       Imaging weighting scheme (e.g. 'briggs', 'natural',
                        'uniform') (default: briggs)
  --robust ROBUST       Robust parameter for Briggs weighting (-2 to +2)
                        (default: 0.0)
  --no_multiscale       Disable multiscale CLEAN for extended structures
  --clean_threshold CLEAN_THRESHOLD
                        Clean threshold in sigma for final deconvolution (Note
                        this is not auto-mask) (default: 1.0)
  --do_pbcor            Apply primary beam correction after imaging
  --no_apply_parang     Disable parallactic angle rotation during imaging
  --cutout_rsun CUTOUT_RSUN
                        Field of view cutout radius in solar radii (default:
                        2.5)
  --no_solar_mask       Disable use solar disk mask during deconvolution
  --no_overlay          Disable overlay plot on GOES SUVI after imaging

###################
Advanced pipeline parameters
###################:
  --non_solar_data      Disable solar data mode
  --non_ds              Disable making solar dynamic spectra
  --do_forcereset_weightflag
                        Force reset of weights and flags (disabled by default)
  --no_noise_cal        Disable noise calibration
  --no_cal_partition    Disable calibrator MS partitioning
  --no_cal_flag         Disable initial flagging of calibrators
  --no_import_model     Disable model import
  --no_basic_cal        Disable basic gain calibration
  --do_sidereal_cor     Sidereal motion correction for Sun (disabled by
                        default)
  --no_selfcal_split    Disable split for self-calibration
  --no_selfcal          Disable self-calibration
  --no_ap_selfcal       Disable amplitude-phase self-calibration
  --no_solar_selfcal    Disable solar-specific self-calibration parameters
  --no_target_split     Disable target data split
  --no_applycal         Disable application of basic calibration solutions
  --no_apply_selfcal    Disable application of self-calibration solutions
  --no_imaging          Disable final imaging

###################
Advanced hardware resource parameters for local system or per node on HPC cluster
###################:
  --cpu_frac CPU_FRAC   Fraction of CPU usuage per node (default: 0.8)
  --mem_frac MEM_FRAC   Fraction of memory usuage per node (default: 0.8)
  --keep_backup         Keep backup of intermediate steps
  --no_remote_logger    Disable remote logger
  --logger_alivetime LOGGER_ALIVETIME
                        Keep remote logger alive for this many hours
                        (Otherwise, logger will be removed after 15 minutes of
                        inactivity) (default: 0)

###################
Advanced HPC settings
###################:
  --n_nodes N_NODES     Number of compute nodes to use (0 means local cluster)
                        (default: 0)

Multiple options demonstrated below can be combined to have all of them together.

  1. To view details of measurement set:

run_meersolar_showms </path/to/measurement_set>

Runs with advanced calibration paramaters

  1. Do calibration with custom calibration parameters. There are two parameters: cal_uvrange and solint which can be changed. Example, run the following command to perform gain solutions at 10second interval and >200lambda data:

run_meersolar </path/to/measurement_set> --solint "10s" --uvrange ">200lambda" --workdir </path/to/work_directory> --outdir </path/to/output_product_directory>

  1. By default for full-polar data, polarization calibration will be performed. To disable it:

run_meersolar </path/to/measurement_set> --no_polcal --workdir </path/to/work_directory> --outdir </path/to/output_product_directory>

Runs with advanced imaging paramaters

  1. Run pipeline to image specific target scans. Default is to use all. Users are requested to check target scan number using run_meersolar_showms first. Example, run the following command to image only target scans 3,8,13 from all available target scans:

run_meersolar </path/to/measurement_set> --target_scans 3 8 13 --workdir </path/to/work_directory> --outdir </path/to/output_product_directory>

  1. Run pipeline to image specific time and frequency range. Default is to use entire time and frequency range. Example for imaging two time ranges given in UTC and frequency ranges given in MHz:

run_meersolar </path/to/measurement_set> --timerange 2024/06/10/09:00:00~2024/06/10/09:30:00,2024/06/10/10:15:00~2024/06/10/10:45:00 --freqrange 600~650,700~800 --workdir </path/to/work_directory> --outdir </path/to/output_product_directory>

  1. Run imaging with custom time and frequency resolution. Default is to use entire observing band and maximum 2 hours (or maximum scan duration) of integration to avoid smearing due to differential rotation of the Sun. Example run for imaging at 2 minutes (120 seconds) time resolution and 10 MHz frequency resolution:

run_meersolar </path/to/measurement_set> --image_timeres 120 --image_freqres 10 --workdir </path/to/work_directory> --outdir </path/to/output_product_directory>

  1. Default is to make only Stokes I images if do_polcal=False and Stokes IQUV, if do_polcal=True. To run only Stokes I imaging, even if do_polcal=True, run:

run_meersolar </path/to/measurement_set> --pol I --workdir </path/to/work_directory> --outdir </path/to/output_product_directory>

Similarly, all other advanced imaging parameters can be used.

Switching off particular pipeline step(s)

By default, all steps will be performed by pipeline. Even pipeline was run upto certain stages and then stopped, all steps from beginning will be performed to avoid any potential issue in failure in previous runs. If user is certain that previous run was successful upto certain stages, those stages can be switched.

Caution

User should not modify any file and directory structure in the work directory. Switching off certain parameters will only allow to run the pipeline forward, if the expected output products from those steps are present with appropriate name in appropriate directory. Otherwise, it will fail.

Take a look at the Advanced pipeline parameters in the help page of run_meersolar. Each parameters are self explanatory. Some examples are given below:

  1. To switch on noise-diode based flux calibration:

run_meersolar </path/to/measurement_set> --no_noise_cal --workdir </path/to/work_directory> --outdir </path/to/output_product_directory>

  1. To switch off self-calibration:

run_meersolar </path/to/measurement_set> --no_selfcal --workdir </path/to/work_directory> --outdir </path/to/output_product_directory>

  1. To stop final imaging:

run_meersolar </path/to/measurement_set> --no_imaging --workdir </path/to/work_directory> --outdir </path/to/output_product_directory>

  1. To switch off self-calibration and final imaging

run_meersolar </path/to/measurement_set> --no_selfcal --no_imaging --workdir </path/to/work_directory> --outdir </path/to/output_product_directory>