GEOS ATS

The geos-ats python package includes tools for managing integrated tests for GEOS. It is built using the Automated Test System (ATS) package. The available console scripts for this package and its API are described below.

run_geos_ats

Primary entry point for running integrated tests.

Runs GEOS integrated tests

usage: run_geos_ats [-h] [-w WORKINGDIR] [-b BASELINEDIR] [-y YAML]
                    [--baselineArchiveName BASELINEARCHIVENAME]
                    [--baselineCacheDirectory BASELINECACHEDIRECTORY] [-d]
                    [-u] [-a ACTION] [-c CHECK] [-v VERBOSE]
                    [-r RESTARTCHECKOVERRIDES [RESTARTCHECKOVERRIDES ...]]
                    [--salloc SALLOC] [--sallocoptions SALLOCOPTIONS]
                    [--ats ATS [ATS ...]] [--machine MACHINE]
                    [--machine-dir MACHINE_DIR] [-l LOGS] [-f]
                    [--failIfTestsFail] [-n NUMNODES]
                    geos_bin_dir ats_target

Positional Arguments

geos_bin_dir

GEOS binary directory.

ats_target

ats file

Named Arguments

-w, --workingDir

Root working directory

-b, --baselineDir

Root baseline directory

-y, --yaml

Path to YAML config file

Default: ''

--baselineArchiveName

Baseline archive name

Default: ''

--baselineCacheDirectory

Baseline cache directory

Default: ''

-d, --delete-old-baselines

Automatically delete old baselines

Default: False

-u, --update-baselines

Force baseline file update

Default: False

-a, --action

Test actions options (run,rerun,continue,list,commands,reset,clean,veryclean,check,rebaseline,rebaselinefailed,report,pack_baselines,upload_baselines,download_baselines)

Default: 'run'

-c, --check

Test check options (all,none,stopcheck,curvecheck,restartcheck)

Default: 'all'

-v, --verbose

Log verbosity options (debug,info,warnings,errors)

Default: 'info'

-r, --restartCheckOverrides

Restart check parameter override (name value)

Default: []

--salloc

Used by the chaosM machine to first allocate nodes with salloc, before running the tests

Default: True

--sallocoptions

Used to override all command-line options for salloc. No other options with be used or added.

Default: ''

--ats

pass arguments to ats

Default: []

--machine

name of the machine

--machine-dir

Search path for machine definitions

-l, --logs
-f, --allow-failed-tests

Default: False

--failIfTestsFail

geos_ats normally exits with 0. This will cause it to exit with an error code if there was a failed test.

Default: False

-n, -N, --numNodes

Default: 2

Note

Arguments can be passed to the underlying ATS system with the –ats argument.

Note

Other machine-specific options for ATS can be viewed by running run_geos_ats –ats help

Debugging

If for any reason you need to debug the geos-ats package, we recommend that you create a local copy of this entry point in the build/integratedTests directory: geosPythonPackages/geos-ats_package/geos-ats/debug_geos-ats.py. This script is designed as a debugger entry point, and will read the autogenerated run script that was built during setup. To use it, you must either have geos-ats installed in your target python environment, or a copy of geosPythonPackages in the expected location (/usr/workspace/[username]/geosPythonPackages).

We recommend that you use VSCode with the Python extension to debug geos-ats. To begin the debugging session, you simply need to load the entry script, set any initial breakpoints you desire, then select the Debug run option. Note that this approach can only be used to debug the python code associated with tests, and not the underlying GEOS tests.

API

Restart Check

class geos.ats.helpers.restart_check.FileComparison(file_path, baseline_path, rtol, atol, regex_expressions, output, warnings_are_errors, skip_missing, diff_file=None)

Class that compares two hdf5 files.

compareAttributes(path, attrs, base_attrs)

Compare two sets of attributes.

PATH [in]: The path at which the comparison takes place. ATTRS [in]: The hdf5 AttributeManager to compare. BASE_ATTRS [in]: The hdf5 AttributeManager to compare against.

compareCharArrays(comp_arr, base_arr)

Compare the valid characters of two arrays and return a formatted string showing differences.

COMP_ARR [in]: The hdf5 Dataset to compare. BASE_ARR [in]: The hdf5 Dataset to compare against.

Returns a formatted string highlighting the differing characters.

compareData(path, arr, base_arr)

Compare the numerical portion of two datasets.

PATH [in]: The path at which the comparison takes place. ARR [in]: The hdf5 Dataset to compare. BASE_ARR [in]: The hdf5 Dataset to compare against.

compareDatasets(dset, base_dset)

Compare two datasets.

DSET [in]: The Dataset to compare. BASE_DSET [in]: The Dataset to compare against.

compareFloatArrays(path, arr, base_arr)

Compares two arrays ARR and BASEARR of floating point values. Entries x1 and x2 are considered equal iff:

abs(x1 - x2) <= ATOL * ( 1 + max(abs(x2)) ) or abs(x1 - x2) <= RTOL * abs(x2).

To measure the degree of difference a scaling factor q is introduced. The goal is now to minimize q such that:

abs(x1 - x2) <= ATOL * ( 1 + max(abs(x2)) ) * q or abs(x1 - x2) <= RTOL * abs(x2) * q.

If RTOL * abs(x2) > ATOL * ( 1 + max(abs(x2)) ) q = abs(x1 - x2) / (RTOL * abs(x2)) else q = abs(x1 - x2) / ( ATOL * ( 1 + max(abs(x2)) ) ).

If the maximum value of q over all the entries is greater than 1.0 then the arrays are considered different and an error message is produced.

PATH [in]: The path at which the comparison takes place. ARR [in]: The hdf5 Dataset to compare. BASE_ARR [in]: The hdf5 Dataset to compare against.

compareFloatScalars(path, val, base_val)

Compare floating point scalars.

PATH [in]: The path at which the comparison occurs. VAL [in]: The value to compare. BASE_VAL [in]: The baseline value to compare against.

compareGroups(group, base_group)

Compare hdf5 groups. GROUP [in]: The Group to compare. BASE_GROUP [in]: The Group to compare against.

compareIntArrays(path, arr, base_arr)

Compare two integer datasets. Exact equality is used as the acceptance criteria.

PATH [in]: The path at which the comparison takes place. ARR [in]: The hdf5 Dataset to compare. BASE_ARR [in]: The hdf5 Dataset to compare against.

compareIntScalars(path, val, base_val)

Compare integer scalars.

PATH [in]: The path at which the comparison occurs. VAL [in]: The value to compare. BASE_VAL [in]: The baseline value to compare against.

compareStringArrays(path, arr, base_arr)

Compare two string datasets. Exact equality is used as the acceptance criteria.

PATH [in]: The path at which the comparison takes place. ARR [in]: The hdf5 Dataset to compare. BASE_ARR [in]: The hdf5 Dataset to compare against.

compareStringScalars(path, val, base_val)

Compare string scalars.

PATH [in]: The path at which the comparison occurs. VAL [in]: The value to compare. BASE_VAL [in]: The baseline value to compare against.

errorMsg(path, message, add_to_diff=False)

Issue an error which occurred at PATH in the files with the contents of MESSAGE. Sets self.different to True and rites the error to both stdout and OUTPUT.

PATH [in]: The path in the files at which the error occurred. MESSAGE [in]: The error message.

isExcluded(path)

Return True iff path matches any of the regex expressions in self.regex_expressions.

PATH [in]: The path to match.

warningMsg(path, message)

Issue a warning which occurred at PATH in the files with the contents of MESSAGE. Writes the warning to both stdout and OUTPUT. If WARNINGS_ARE_ERRORS then this is a wrapper around errorMsg.

PATH [in]: The path in the files at which the warning occurred. MESSAGE [in]: The warning message.

geos.ats.helpers.restart_check.findMaxMatchingFile(file_path)

Given a path FILE_PATH where the base name of FILE_PATH is treated as a regular expression find and return the path of the greatest matching file/folder or None if no match is found.

FILE_PATH [in]: The pattern to match.

Examples: ‘.*’ will return the file/folder with the greatest name in the current directory. ‘test/plot_*.hdf5’ will return the file with the greatest name in the ./test directory that begins with ‘plot’ and ends with ‘.hdf5’.

geos.ats.helpers.restart_check.main()

Parses the command line arguments and executes the proper comparison. Writes output to both stdout and a ‘%s.restartcheck’ file where the first part is the path of the file to compare.

Example

The file to compare is ./a/b/c.hdf5 the output will be a ./a/b/c.restartcheck file.

geos.ats.helpers.restart_check.write(output, msg)

Write MSG to both stdout and OUTPUT. OUTPUT [in/out]: File stream to write to. MSG [in]: Message to write.

geos.ats.helpers.restart_check.writeHeader(file_pattern, file_path, baseline_pattern, baseline_path, args)

Write the header.

FILE_PATTERN [in]: The pattern used to find the file to compare. FILE_PATH [in]: The path to the file to compare. BASELINE_PATTERN [in]: The pattern used to find the file to compare against. BASELINE_PATH [in]: THE path to the file to compare against. ARGS [in]: A dictionary of arguments to FileComparison.

Curve Check

geos.ats.helpers.curve_check.check_diff(parameter_name, set_name, target, baseline, tolerance, errors, modifier='baseline')

Compute the L2-norm of the diff and compare to the set tolerance

Parameters:

  • parameter_name (str) – Parameter name

  • set_name (str) – Set name

  • target (np.ndarray) – Target value array

  • baseline (np.ndarray) – Baseline value array

  • tolerance (float) – Required tolerance of diff

  • errors (list) – List to add any errors

Returns:

Interpolated value array

Return type:

np.ndarray

geos.ats.helpers.curve_check.compare_time_history_curves(fname, baseline, curve, tolerance, output, output_n_column, units_time, script_instructions)

Compute time history curves

Parameters:

  • fname (str) – Target curve file name

  • baseline (str) – Baseline curve file name

  • curve (list) – list containing pairs of value and set names to test

  • tolerance (list) – Tolerance for curve diffs

  • output (str) – Path to place output figures

  • ncol (int) – Number of columns to use in the figure

  • units_time (str) – Time units for the figure

  • script_instructions (list) – List of (script, function, parameter, setname) values

Returns:

warnings, errors

Return type:

tuple

geos.ats.helpers.curve_check.curve_check_figure(parameter_name, location_str, set_name, data, data_sizes, output_root, ncol, units_time)

Generate figures associated with the curve check

Parameters:

  • parameter_name (str) – Parameter name

  • set_name (str) – Set name

  • data (dict) – Dictionary of curve data

  • data_sizes (dict) – Dictionary of curve data sizes

  • output_root (str) – Path of the folder to place the figures

  • ncol (int) – Number of columns to use in the figure

  • units_time (str) – Time units for the figure

geos.ats.helpers.curve_check.curve_check_parser()

Build the curve check parser

Returns:

The curve check parser

Return type:

argparse.parser

geos.ats.helpers.curve_check.evaluate_external_script(script, fn, data)

Evaluate an external script to produce the curve

Parameters:

  • script (str) – Path to a python script

  • fn (str) – Name of the function to call

Returns:

Curve values

Return type:

np.ndarray

geos.ats.helpers.curve_check.interpolate_values_time(ta, xa, tb)

Interpolate array values in time

Parameters:

  • ta (np.ndarray) – Target time array

  • xa (np.ndarray) – Target value array

  • tb (np.ndarray) – Baseline time array

Returns:

Interpolated value array

Return type:

np.ndarray

geos.ats.helpers.curve_check.main()

Entry point for the curve check script