pyfstat.utils package

A collection of helpful functions to facilitate ease-of-use of PyFstat.

Most of these are used internally by other parts of the package and are of interest mostly only for developers, but others can also be helpful for end users.

Functions in these modules can be directly acessed via pyfstat.utils without explicitly mentioning the specific module in where they reside. (E.g. just call pyfstat.utils.some_function, not pyfstat.utils.some_topic.some_function.)

Submodules

pyfstat.utils.cli module

pyfstat.utils.cli.run_commandline(cl, raise_error=True, return_output=False)[source]

Run a string command as a subprocess.

Parameters:
  • cl (str) – Command to run

  • raise_error (bool) – If True, raise an error if the subprocess fails. If False, just log the error, continue, and return None.

  • return_output (bool) – If True, return the subprocess.CompletedProcess object. If False, return None.

Returns:

The `subprocess.CompletedProcess of the subprocess if return_output=True. None if return_output=False or on failed execution if raise_error=False.

Return type:

out

pyfstat.utils.cli.match_commandlines(cl1, cl2, be_strict_about_full_executable_path=False)[source]

Check if two commandline strings match element-by-element, regardless of order.

Parameters:
  • cl1 (str) – Commandline strings of executable –key1=val1 –key2=val2 etc format.

  • cl2 (str) – Commandline strings of executable –key1=val1 –key2=val2 etc format.

  • be_strict_about_full_executable_path (bool) – If False (default), only checks the basename of the executable. If True, requires its full path to match.

Returns:

match – Whether the executable and all key=val pairs of the two strings matched.

Return type:

bool

pyfstat.utils.converting module

pyfstat.utils.converting.get_dictionary_from_lines(lines, comments, raise_error)[source]

Return a dictionary of key=val pairs for each line in a list.

Parameters:
  • lines (list of strings) – The list of lines to parse.

  • comments (str or list of strings) – Characters denoting that a row is a comment.

  • raise_error (bool) – If True, raise an error for lines which are not comments, but cannot be read. Note that CFSv2 “loudest” files contain complex numbers which fill raise an error unless this is set to False.

Returns:

d – The key=val pairs as a dictionary.

Return type:

dict

pyfstat.utils.converting.parse_list_of_numbers(val)[source]

Convert a number, list of numbers or comma-separated str into a list of numbers.

This is useful e.g. for sqrtSX inputs where the user can be expected to try either type of input.

Parameters:

val (float, list or str) – The input to be parsed.

Returns:

out – The parsed list.

Return type:

list

pyfstat.utils.converting.gps_to_datestr_utc(gps)[source]

Convert an integer count of GPS seconds to a UTC date-time string.

This uses the locale’s default string formatting as per datetime.strftime(). It is intended just for informing the user and may not be as reliable in all situations as lal[apps]_tconvert. If you want to do any postprocessing of the date-time string, for safety you should probably call that commandline tool.

Parameters:

gps (int) – Integer seconds since GPS seconds.

Returns:

dtstr – A string representation of date-time in UTC and locale format.

Return type:

str

pyfstat.utils.ephemeris module

pyfstat.utils.ephemeris.get_ephemeris_files()[source]

Set the ephemeris files to use for the Earth and Sun.

This looks first for a configuration file ~/.pyfstat.conf giving individual earth/sun file paths like this:

` earth_ephem = '/my/path/earth00-40-DE405.dat.gz' sun_ephem = '/my/path/sun00-40-DE405.dat.gz' `

If such a file is not found or does not conform to that format, then we rely on lal’s recently improved ability to find proper default fallback paths for the [earth/sun]00-40-DE405 ephemerides with both pip- and conda-installed packages,

Alternatively, ephemeris options can be set manually on each class instantiation.

NOTE that the $LALPULSAR_DATADIR environment variable is no longer supported!

Returns:

earth_ephem, sun_ephem – Paths of the two files containing positions of Earth and Sun.

Return type:

str

pyfstat.utils.formatting module

pyfstat.utils.formatting.round_to_n(x, n)[source]

Simple rounding function for getting a fixed number of digits.

Parameters:
  • x (float) – The number to round.

  • n (int) – The number of digits to round to (before plus after the decimal separator).

Returns:

rounded – The rounded number.

Return type:

float

pyfstat.utils.formatting.texify_float(x, d=2)[source]

Format float numbers nicely for LaTeX output, including rounding.

Numbers with absolute values between 0.01 and 100 will be returned in plain float format, while smaller or larger numbers will be returned in powers-of-ten notation.

Parameters:
  • x (float) – The number to round and format.

  • n (int) – The number of digits to round to (before plus after the decimal separator).

Returns:

formatted – The formatted string.

Return type:

str

pyfstat.utils.formatting.get_doppler_params_output_format(keys)[source]

Set a canonical output precision for frequency evolution parameters.

This uses the same format (%.16g) as the write_FstatCandidate_to_fp() function of the ComputeFstatistic_v2 executable.

This assigns that format to each parameter name in keys which matches a hardcoded list of known standard ‘Doppler’ parameters, and ignores any others.

Parameters:

keys (dict) – The parameter keys for which to select formats.

Returns:

fmt – A dictionary assigning the default format to each parameter key from the hardcoded list of standard ‘Doppler’ parameters.

Return type:

dict

pyfstat.utils.gsl module

pyfstat.utils.gsl.convert_array_to_gsl_matrix(array)[source]

Convert a numpy array to a LAL-wrapped GSL matrix.

Parameters:

array (np.ndarray) – The array to convert. array.shape must have 2 dimensions.

Returns:

gsl_matrix – The LAL-wrapped GSL matrix object.

Return type:

lal.gsl_matrix

pyfstat.utils.importing module

pyfstat.utils.importing.initializer(func)[source]

Decorator to automatically assign the parameters of a class instantiation to self.

pyfstat.utils.importing.safe_X_less_plt()[source]

pyfstat.utils.io module

pyfstat.utils.io.read_par(filename=None, label=None, outdir=None, suffix='par', comments=['%', '#'], raise_error=False)[source]

Read in a .par or .loudest file, returns a dictionary of the key=val pairs.

Notes

This can also be used to read in .loudest files produced by the ComputeFstatistic_v2 executable, or any file which has rows of key=val data (in which the val can be understood using eval(val)).

Parameters:
  • filename (str) – Filename (path) containing rows of key=val data to read in.

  • label (str, optional) – If filename is None, form the file to read as outdir/label.suffix.

  • outdir (str, optional) – If filename is None, form the file to read as outdir/label.suffix.

  • suffix (str, optional) – If filename is None, form the file to read as outdir/label.suffix.

  • comments (str or list of strings, optional) – Characters denoting that a row is a comment.

  • raise_error (bool, optional) – If True, raise an error for lines which are not comments, but cannot be read.

Returns:

d – The key=val pairs as a dictionary.

Return type:

dict

pyfstat.utils.io.read_txt_file_with_header(f, names=True, comments='#')[source]

Wrapper to np.genfromtxt with smarter handling of variable-length commented headers.

The header is identified as an uninterrupted block of lines from the beginning of the file, each starting with the given comments character.

After identifying a header of length Nhead, this function then tells np.genfromtxt() to skip Nhead-1 lines (to allow for reading field names from the last commented line before the actual data starts).

Parameters:
  • f (str) – Name of the file to read.

  • names (bool) – Passed on to np.genfromtxt(): If True, the field names are read from the last header line.

  • comments (str) – The character used to indicate the start of a comment. Also passed on to np.genfromtxt().

Returns:

data – The data array read from the file after skipping the header.

Return type:

np.ndarray

pyfstat.utils.io.read_parameters_dict_lines_from_file_header(outfile, comments='#', strip_spaces=True)[source]

Load a list of pretty-printed parameters dictionary lines from a commented file header.

Returns a list of lines from a commented file header that match the pretty-printed parameters dictionary format as generated by BaseSearchClass.get_output_file_header(). The opening/closing bracket lines ({,`}`) are not included. Newline characters at the end of each line are stripped.

Parameters:
  • outfile (str) – Name of a PyFstat-produced output file.

  • comments (str) – Comment character used to start header lines.

  • strip_spaces (bool) – Whether to strip leading/trailing spaces.

Returns:

dict_lines – A list of unparsed pprinted dictionary entries.

Return type:

list

pyfstat.utils.io.get_parameters_dict_from_file_header(outfile, comments='#', eval_values=False)[source]

Load a parameters dict from a commented file header.

Returns a parameters dictionary, as generated by BaseSearchClass.get_output_file_header(), from an output file header. Always returns a proper python dictionary, but the values will be unparsed strings if not requested otherwise.

Parameters:
  • outfile (str) – Name of a PyFstat-produced output file.

  • comments (str) – Comment character used to start header lines.

  • eval_values (bool) – If False, return dictionary values as unparsed strings. If True, evaluate each of them. DANGER! Only do this if you trust the source of the file!

Returns:

params_dict – A dictionary of parameters (with values either as unparsed strings, or evaluated).

Return type:

dictionary

pyfstat.utils.predict module

pyfstat.utils.predict.predict_fstat(h0=None, cosi=None, psi=None, Alpha=None, Delta=None, F0=None, sftfilepattern=None, timestampsFiles=None, minStartTime=None, duration=None, IFOs=None, assumeSqrtSX=None, tempory_filename='fs.tmp', earth_ephem=None, sun_ephem=None, transientWindowType='none', transientStartTime=None, transientTau=None)[source]

Wrapper to PredictFstat executable for predicting expected F-stat values.

Parameters:
  • h0 (float) – Signal parameters, see lalpulsar_PredictFstat –help for more info.

  • cosi (float) – Signal parameters, see lalpulsar_PredictFstat –help for more info.

  • psi (float) – Signal parameters, see lalpulsar_PredictFstat –help for more info.

  • Alpha (float) – Signal parameters, see lalpulsar_PredictFstat –help for more info.

  • Delta (float) – Signal parameters, see lalpulsar_PredictFstat –help for more info.

  • F0 (float or None) – Signal frequency. Only needed for noise floor estimation when given sftfilepattern but assumeSqrtSX=None. The actual F-stat prediction is frequency-independent.

  • sftfilepattern (str or None) – Pattern matching the SFT files to use for inferring detectors, timestamps and/or estimating the noise floor.

  • timestampsFiles (str or None) – Comma-separated list of per-detector files containing timestamps to use. Only used if sftfilepattern=None.

  • minStartTime (int or None) – If sftfilepattern given: used as optional constraints. If timestampsFiles given: ignored. If neither given: used as the interval for prediction.

  • duration (int or None) – If sftfilepattern given: used as optional constraints. If timestampsFiles given: ignored. If neither given: used as the interval for prediction.

  • IFOs (str or None) – Comma-separated list of detectors. Required if sftfilepattern=None, ignored otherwise.

  • assumeSqrtSX (float or str) – Assume stationary per-detector noise-floor instead of estimating from SFTs. Single float or str value: use same for all IFOs. Comma-separated string: must match len(IFOs) and/or the data in sftfilepattern. Detectors will be paired to list elements following alphabetical order. Required if sftfilepattern=None, optional otherwise..

  • tempory_filename (str) – Temporary file used for PredictFstat output, will be deleted at the end.

  • earth_ephem (str or None) – Ephemerides files, defaults will be used if None.

  • sun_ephem (str or None) – Ephemerides files, defaults will be used if None.

  • transientWindowType (str) – Optional parameter for transient signals, see lalpulsar_PredictFstat –help. Default of none means a classical Continuous Wave signal.

  • transientStartTime (int or None) – Optional parameters for transient signals, see lalpulsar_PredictFstat –help.

  • transientTau (int or None) – Optional parameters for transient signals, see lalpulsar_PredictFstat –help.

Returns:

twoF_expected, twoF_sigma – The expectation and standard deviation of 2F.

Return type:

float

pyfstat.utils.predict.get_predict_fstat_parameters_from_dict(signal_parameters, transientWindowType=None)[source]

Extract a subset of parameters as needed for predicting F-stats. Given a dictionary with arbitrary signal parameters, this extracts only those ones required by helper_functions.predict_fstat(): Freq, Alpha, Delta, h0, cosi, psi. Also preserves transient parameters, if included in the input dict.

Parameters:
  • signal_parameters (dict) – Dictionary containing at least those signal parameters required by helper_functions.predict_fstat. This dictionary’s keys must follow the PyFstat convention (e.g. F0 instead of Freq).

  • transientWindowType (str) – Transient window type to store in the output dict. Currently required because the typical input dicts produced by various PyFstat functions tend not to store this property. If there is a key with this name already, its value will be overwritten.

Returns:

predict_fstat_params – The dictionary of selected parameters.

Return type:

dict

pyfstat.utils.runlalsuite module

pyfstat.utils.runlalsuite.get_lal_exec(cmd)[source]

Get a lalpulsar/lalapps executable name with the right prefix.

This is purely to allow for backwards compatibility if, for whatever reason, someone needs to run with old releases (lalapps<9.0.0 and lalpulsar<5.0.0) from before the executables were moved.

Parameters:

cmd (str) – Base executable name without lalapps/lalpulsar prefix.

Returns:

full_cmd – Full executable name with the right prefix.

Return type:

str

pyfstat.utils.runlalsuite.get_covering_band(tref, tstart, tend, F0, F1, F2, F0band=0.0, F1band=0.0, F2band=0.0, maxOrbitAsini=0.0, minOrbitPeriod=0.0, maxOrbitEcc=0.0)[source]

Get the covering band for CW signals for given time and parameter ranges.

This uses the lalpulsar function XLALCWSignalCoveringBand(), accounting for the spin evolution of the signals within the given [F0,F1,F2] ranges, the maximum possible Dopper modulation due to detector motion (i.e. for the worst-case sky locations), and for worst-case binary orbital motion.

Parameters:
  • tref (int) – Reference time (in GPS seconds) for the signal parameters.

  • tstart (int) – Start time (in GPS seconds) for the signal evolution to consider.

  • tend (int) – End time (in GPS seconds) for the signal evolution to consider.

  • F0 (float) – Minimum frequency and spin-down of signals to be covered.

  • F1 (float) – Minimum frequency and spin-down of signals to be covered.

  • F1 – Minimum frequency and spin-down of signals to be covered.

  • F0band (float) – Ranges of frequency and spin-down of signals to be covered.

  • F1band (float) – Ranges of frequency and spin-down of signals to be covered.

  • F1band – Ranges of frequency and spin-down of signals to be covered.

  • maxOrbitAsini (float) – Largest orbital projected semi-major axis to be covered.

  • minOrbitPeriod (float) – Shortest orbital period to be covered.

  • maxOrbitEcc (float) – Highest orbital eccentricity to be covered.

Returns:

minCoverFreq, maxCoverFreq – Estimates of the minimum and maximum frequencies of the signals from the given parameter ranges over the [tstart,tend] duration.

Return type:

float

pyfstat.utils.runlalsuite.generate_loudest_file(max_params, tref, outdir, label, sftfilepattern, minStartTime=None, maxStartTime=None, transientWindowType=None, earth_ephem=None, sun_ephem=None)[source]

Use ComputeFstatistic_v2 executable to produce a .loudest file.

Parameters:
  • max_params (dict) – Dictionary of a single parameter-space point. This needs to already have been translated to lal conventions and must NOT include detection statistic entries!

  • tref (int) – Reference time of the max_params.

  • outdir (str) – Directory to place the .loudest file in.

  • label (str) – Search name bit to be used in the output filename.

  • sftfilepattern (str) – Pattern to match SFTs using wildcards (*?) and ranges [0-9]; mutiple patterns can be given separated by colons.

  • minStartTime (int or None) – GPS seconds of the start time and end time; default: use al available data.

  • maxStartTime (int or None) – GPS seconds of the start time and end time; default: use al available data.

  • transientWindowType (str or None) – optional: transient window type, needs to go with t0 and tau parameters inside max_params.

  • earth_ephem (str or None) – optional: user-set Earth ephemeris file

  • sun_ephem (str or None) – optional: user-set Sun ephemeris file

Returns:

loudest_file – The filename of the CFSv2 output file.

Return type:

str

pyfstat.utils.sft module

pyfstat.utils.sft.get_sft_as_arrays(sftfilepattern, fMin=None, fMax=None, constraints=None)[source]

Read binary SFT files into NumPy arrays.

Parameters:
  • sftfilepattern (str) – Pattern to match SFTs using wildcards (*?) and ranges [0-9]; multiple patterns can be given separated by colons.

  • fMin (Optional[float]) – Restrict frequency range to [fMin, fMax]. If None, retreive the full frequency range.

  • fMax (Optional[float]) – Restrict frequency range to [fMin, fMax]. If None, retreive the full frequency range.

  • constraints (Optional[SFTConstraints]) – Constrains to be fed into XLALSFTdataFind to specify detector, GPS time range or timestamps to be retrieved.

Return type:

Tuple[ndarray, dict, dict]

Returns:

  • freqs – The frequency bins in each SFT. These will be the same for each SFT, so only a single 1D array is returned.

  • times – The SFT start times as a dictionary of 1D arrays, one for each detector. Keys correspond to the official detector names as returned by lalpulsar.ListIFOsInCatalog.

  • data – A dictionary of 2D arrays of the complex Fourier amplitudes of the SFT data for each detector in each frequency bin at each timestamp. Keys correspond to the official detector names as returned by lalpulsar.ListIFOsInCatalog.

pyfstat.utils.sft.get_commandline_from_SFTDescriptor(descriptor)[source]

Extract a commandline from the ‘comment’ entry of a SFT descriptor.

Most LALSuite SFT creation tools save their commandline into that entry, so we can extract it and reuse it to reproduce that data.

Since lalapps 9.0.0 / lalpulsar 5.0.0 the relevant executables have been moved to lalpulsar, but we allow for lalapps backwards compatibility here,

Parameters:

descriptor (SFTDescriptor) – Element of a lalpulsar.SFTCatalog structure.

Returns:

cmd – A lalapps/lalpulsar commandline string, or an empty string if no match in comment.

Return type:

str

Module contents

A collection of helpful functions to facilitate ease-of-use of PyFstat.

Most of these are used internally by other parts of the package and are of interest mostly only for developers, but others can also be helpful for end users.