Skip to content

Testing

Unit tests

Install project with test dependencies

pip install .[test]

Run test suite located in /tests/unit_tests folder.

pytest tests/unit_tests -v

Regression tests

Engine regression test

Test is located in regression_tests/test_engine_regression.py file.

Engine test is based on comparing two inputs:

  • EngineTestReport class, which content is outside smartreport package. Ideally this report should NOT be changed
  • Expected files, generated previously based on the EngineTestReport. Those reports serve as a source of truth and are considered to be verified.

Test has the following procedure:

  1. New EngineTestReport is generated using the current version of the smartreport package
  2. Generated report is temporarily saved on the disk, to test if export works ok
  3. Generated report is loaded to memory to verify the content
  4. Expected report is loaded to memory from disk
  5. Content of both reports is compared. Exact method of comparison depends on the used output type and test element. Since EngineTestReport is constant, both reports should be the same (or at least very similar).

When engine regression test fails, it means that the report generation engine works differently than in the previous release. If the change is intentional, in order to fix the tests, the regression files must be replaced with newly generated ones.

Generating engine reference files

You can generate the engine reference files with a help of command line script.

To do so run:

python tests/regression_tests/make_reference_engine.py <ENGINE_TYPE>

Where <ENGINE_TYPE> is the type of report output you want to regenerate. Supported ones are word, dash and json. This script will save the new reference files in tests/regression_tests/reference_files/engine/

Example usage:

python tests\regression_tests\make_reference_engine.py json
python tests\regression_tests\make_reference_engine.py dash
python tests\regression_tests\make_reference_engine.py word

Report regression test

Test is located in regression_tests/test_report_regression.py file.

Report regression test is a PyTest case created to verify complete report structure. Its scope is to compare every part of the generated report to the corresponding report that is stored in the tests/regression_tests/reference_files/reports/. These reference reports should be generated every time when report content is changed.

Test parameters corresponds to the reference reports stored in the folder mentioned above.

Test has the following procedure:

For each reference file in tests/regression_tests/reference_files/reports/

  1. Read reference file.
  2. Get report parameters and expected report content from the reference file.
  3. Build new report object based on the report parameters from the report file and generate new report document.
  4. Compare expected report content from the reference file with the newly created report document.

Tests passes when both expected report content and the content of the newly generated report document are the same.

pytest tests/regression_tests/test_report_regression.py -v

Generating report reference files

To generate reference report for regression testing invoke tests/regression_tests/make_reference_report.py

Function takes two required arguments:

  • report_type: - Type of the report to generate. For old reports it accepts: measurement, motor_asset or generic_machine_asset. For new reports it accepts: motor_asset or drive_asset

  • asset_id: Asset Identifier. For old reports it's the SmartSensor ID, for new reports it's the Asset ID in Powertrain API

It also accepts optional arguments:

  • --filename: Name of the report reference file. If not provided then the default name is used
  • --start-date: Reference report start date. Default: now() - 30 days
  • --end-date: Reference report end date. Default: now()
  • --version: Report Version. Accepted: 1, 3. Default: 1
  • --report-date: Report Date. Default: [2025-01-01]

Reference files are stored in tests/regression_tests/reference_files/reports

Example usage:

python tests\regression_tests\make_reference_report.py motor_fleet 7624 --filename MotorFleet7624_August2022 --start-date 2022-08-01 --end-date 2022-09-01