Інтеграційне тестування за допомогою MAVSDK

PX4 can be tested end to end using integration tests based on MAVSDK.

The tests are written in Python using the MAVSDK Python package and pytest. They are primarily developed against SITL and run in continuous integration (CI). В майбутньому ми плануємо зробити їх універсальними для будь-якої платформи/обладнання.

The instructions below explain how to set up and run the tests locally, and how to write new ones.

INFO

This is the recommended integration test framework for PX4.

Вимоги

Налаштування середовища розробника

Якщо ви цього ще не зробили:

• Install the development toolchain for Linux or macOS (Windows not supported).

• Get the PX4 source code:

  git clone https://github.com/PX4/PX4-Autopilot.git --recursive
  cd PX4-Autopilot

Збірка PX4 для тестування

The tests run in the SIH simulator, which is built into PX4. To build PX4 for SITL, use:

make px4_sitl_default

Install the Test Dependencies

The tests need the mavsdk and pytest Python packages, in the versions given in Tools/setup/test-requirements.txt. They are installed by the setup scripts, or manually with:

pip install -r Tools/setup/test-requirements.txt

The mavsdk package is a native binding of the MAVSDK C++ library, so no separate MAVSDK installation is needed.

Запуск усіх PX4 тестів

To run all SITL tests using the SIH simulator as defined in sih-sitl.json, do:

test/mavsdk_tests/mavsdk_test_runner.py test/mavsdk_tests/configs/sih-sitl.json --speed-factor 10

This will list all the tests and then run them sequentially. Each test case is run against a freshly started PX4 instance.

To see all possible command line arguments use the -h argument:

test/mavsdk_tests/mavsdk_test_runner.py -h

usage: mavsdk_test_runner.py [-h] [--log-dir LOG_DIR] [--speed-factor SPEED_FACTOR] [--iterations ITERATIONS] [--abort-early]
                             [--gui] [--model MODEL] [--case CASE] [--debugger DEBUGGER] [--upload] [--force-color]
                             [--verbose] [--build-dir BUILD_DIR]

positional arguments:
  config_file           JSON config file to use

options:
  -h, --help            show this help message and exit
  --log-dir LOG_DIR     Directory for log files
  --speed-factor SPEED_FACTOR
                        how fast to run the simulation
  --iterations ITERATIONS
                        how often to run all tests
  --abort-early         abort on first unsuccessful test
  --gui                 display the visualization for a simulation
  --model MODEL         only run tests for one model
  --case CASE           only run tests for one case (or multiple cases with wildcard '*')
  --debugger DEBUGGER   choice from valgrind, callgrind, gdb, lldb
  --upload              Upload logs to logs.px4.io
  --force-color         Force colorized output
  --verbose             enable more verbose output
  --build-dir BUILD_DIR
                        relative path where the built files are stored

Запуск одного тесту

Run a single test by specifying the model and test case as command line options. A test case is a pytest test ID, and wildcards are supported. For example, to test a multicopter mission using the SIH simulator:

test/mavsdk_tests/mavsdk_test_runner.py test/mavsdk_tests/configs/sih-sitl.json --speed-factor 10 --model quadx --case 'test_multicopter_mission.py::test_square_mission_with_return'

Or to run all the failsafe tests:

test/mavsdk_tests/mavsdk_test_runner.py test/mavsdk_tests/configs/sih-sitl.json --speed-factor 10 --model quadx --case 'test_multicopter_failsafe.py::*'

To list all test cases, use pytest from the test directory:

cd test/mavsdk_tests
pytest --collect-only -q

Run a Test Against a Running PX4

While developing a test, it can be quicker to start PX4 yourself and run the test directly with pytest. Start PX4 with SIH in one terminal:

make px4_sitl sihsim_quadx

And run a test in another terminal:

cd test/mavsdk_tests
pytest -v -s 'test_multicopter_mission.py::test_takeoff_and_land'

The -s option shows the test output while it runs. Most tests expect a vehicle that is freshly started and on the ground, so restart PX4 between tests.

Write a Test

Tests are plain pytest functions in test/mavsdk_tests/test_*.py. A test gets a tester that is connected to the vehicle through a fixture, and is selected for vehicles by markers:

import pytest

from autopilot_tester import AutopilotTester, MissionOptions


@pytest.mark.multicopter
def test_square_mission_with_return(tester: AutopilotTester) -> None:
    tester.wait_until_ready()
    tester.prepare_square_mission(MissionOptions(rtl_at_end=True))
    tester.arm()
    tester.execute_mission()
    tester.wait_until_disarmed(180)

The testers wrap MAVSDK with the building blocks the tests need, such as missions, offboard control, failure injection and trajectory checks:

• tester: autopilot_tester.py with the common functionality.
• tester_rtl: autopilot_tester_rtl.py adds rally points and geofences.
• tester_loiter: autopilot_tester_loiter.py adds fixed-wing loiter, reposition and figure eight.
• tester_follow_me: autopilot_tester_follow_me.py adds follow-me.
• tester_vision_target: autopilot_tester_vision_target.py simulates landing target sensors for precision landing.

Things to keep in mind:

• Timeouts and sleeps of the testers are in simulation time, so a test behaves the same at any speed factor.
• A failing assert in a MAVSDK callback is not reported by MAVSDK. Use check_in_callback() in callbacks instead, which fails the test once it finishes.
• MAVLink messages that MAVSDK has no plugin for can be sent and received with MAVSDK's MavlinkDirect plugin, see for example send_command_int().
• Markers have to be listed in pytest.ini. The test_filter of each entry in the config files is a pytest marker expression, such as multicopter or offboard.

The tests are checked in CI with mypy --strict and flake8.

Примітки щодо реалізацій:

• The tests are invoked from the test runner script mavsdk_test_runner.py.

  For each test case, the runner starts px4 (and Gazebo Classic if configured) with a fresh root filesystem, runs the test with pytest, and collects the logs of these processes.

• The config files in test/mavsdk_tests/configs/ define which models are tested with which tests, and per model settings such as the timeout, environment variables or the Gazebo world.

Терміни:

• "model": This is the selected simulation model, e.g. quadx.
• "test case": This is a pytest test, identified by its test ID, e.g. test_multicopter_mission.py::test_takeoff_and_land.