# Integration Testing using MAVSDK
PX4 can be tested end to end to using integration tests based on MAVSDK (opens new window).
The tests are primarily developed against SITL and run in continuous integration (CI). In future we plan to generalise them for any platform/hardware.
The instructions below explain how to setup and run the tests locally.
# Prerequisites
# Setup Developer Environment
If you haven't done so already:
Install the development toolchain for Linux or macOS (Windows not supported). Gazebo Classic is required, and should be installed by default.
-
git clone https://github.com/PX4/PX4-Autopilot.git --recursive cd PX4-Autopilot
# Build PX4 for Testing
To build PX4 source code for simulator testing, use:
DONT_RUN=1 make px4_sitl gazebo-classic mavsdk_tests
# Install the MAVSDK C++ Library
The tests need the MAVSDK C++ library installed system-wide (e.g. in /usr/lib
or /usr/local/lib
).
Install either from binaries or source:
- MAVSDK > C++ > C++ QuickStart (opens new window): Install as a prebuilt library on supported platforms (recommended)
- MAVSDK > C++ Guide > Building from Source (opens new window): Build C++ library from source.
# Run All PX4 Tests
To run all SITL tests as defined in sitl.json (opens new window), do:
test/mavsdk_tests/mavsdk_test_runner.py test/mavsdk_tests/configs/sitl.json --speed-factor 10
This will list all of the tests and then run them sequentially.
To see all possible command line arguments use the -h
argument:
test/mavsdk_tests/mavsdk_test_runner.py -h
usage: mavsdk_test_runner.py [-h] [--log-dir LOG_DIR] [--speed-factor SPEED_FACTOR] [--iterations ITERATIONS] [--abort-early] [--gui] [--model MODEL]
[--case CASE] [--debugger DEBUGGER] [--verbose]
config_file
positional arguments:
config_file JSON config file to use
optional arguments:
-h, --help show this help message and exit
--log-dir LOG_DIR Directory for log files
--speed-factor SPEED_FACTOR
how fast to run the simulation
--iterations ITERATIONS
how often to run all tests
--abort-early abort on first unsuccessful test
--gui display the visualization for a simulation
--model MODEL only run tests for one model
--case CASE only run tests for one case
--debugger DEBUGGER choice from valgrind, callgrind, gdb, lldb
--verbose enable more verbose output
# Run a Single Test
Run a single test by specifying the model
and test case
as command line options.
For example, to test flying a tailsitter in a mission you might run:
test/mavsdk_tests/mavsdk_test_runner.py test/mavsdk_tests/configs/sitl.json --speed-factor 10 --model tailsitter --case 'Fly VTOL mission'
The easiest way to find out the current set of models and their associated test cases is to run all PX4 tests as shown above (note, you can then cancel the build if you wish to test just one).
At time of writing the list generated by running all tests is:
About to run 39 test cases for 3 selected models (1 iteration):
- iris:
- 'Land on GPS lost during mission (baro height mode)'
- 'Land on GPS lost during mission (GPS height mode)'
- 'Continue on mag lost during mission'
- 'Continue on baro lost during mission (baro height mode)'
- 'Continue on baro lost during mission (GPS height mode)'
- 'Continue on baro stuck during mission (baro height mode)'
- 'Continue on baro stuck during mission (GPS height mode)'
- 'Takeoff and Land'
- 'Fly square Multicopter Missions including RTL'
- 'Fly square Multicopter Missions with manual RTL'
- 'Fly straight Multicopter Mission'
- 'Offboard takeoff and land'
- 'Offboard position control'
- 'Fly forward in position control'
- 'Fly forward in altitude control'
- standard_vtol:
- 'Land on GPS lost during mission (baro height mode)'
- 'Land on GPS lost during mission (GPS height mode)'
- 'Continue on mag lost during mission'
- 'Continue on baro lost during mission (baro height mode)'
- 'Continue on baro lost during mission (GPS height mode)'
- 'Continue on baro stuck during mission (baro height mode)'
- 'Continue on baro stuck during mission (GPS height mode)'
- 'Takeoff and Land'
- 'Fly square Multicopter Missions including RTL'
- 'Fly square Multicopter Missions with manual RTL'
- 'Fly forward in position control'
- 'Fly forward in altitude control'
- tailsitter:
- 'Land on GPS lost during mission (baro height mode)'
- 'Land on GPS lost during mission (GPS height mode)'
- 'Continue on mag lost during mission'
- 'Continue on baro lost during mission (baro height mode)'
- 'Continue on baro lost during mission (GPS height mode)'
- 'Continue on baro stuck during mission (baro height mode)'
- 'Continue on baro stuck during mission (GPS height mode)'
- 'Takeoff and Land'
- 'Fly square Multicopter Missions including RTL'
- 'Fly square Multicopter Missions with manual RTL'
- 'Fly forward in position control'
- 'Fly forward in altitude control'
# Notes on implementation
The tests are invoked from the test runner script mavsdk_test_runner.py (opens new window), which is written in Python.
In addition to MAVSDK, this runner starts
px4
as well as Gazebo for SITL tests, and collects the logs of these processes.The test runner is a C++ binary that contains:
- The main (opens new window) function to parse the arguments.
- An abstraction around MAVSDK called autopilot_tester (opens new window).
- The actual tests using the abstraction around MAVSDK as e.g. test_multicopter_mission.cpp (opens new window).
- The tests use the catch2 (opens new window) unit testing framework.
The reasons for using this framework are:
- Asserts (
REQUIRE
) which are needed to abort a test can be inside of functions (and not just in the top level test as is the case with gtest (opens new window)). - Dependency management is easier because catch2 can just be included as a header-only library.
- Catch2 supports tags (opens new window), which allows for flexible composition of tests.
- Asserts (
Terms used:
- "model": This is the selected Gazebo model, e.g.
iris
. - "test case": This is a catch2 test case (opens new window).