PX4 User Guide (v1.12)

# Computer Vision (Optical Flow, MoCap, VIO, Avoidance)

Computer vision (opens new window) techniques enable computers to use visual data to make sense of their environment.

PX4 uses computer vision systems (primarily running on Companion Computers) in order to support the following features:

  • Pose/Velocity Estimation:
    • Optical Flow provides 2D velocity estimation (using a downward facing camera and a downward facing distance sensor).
    • Motion Capture provides 3D pose estimation using a vision system that is external to the vehicle. It is primarily used for indoor navigation.
    • Visual Inertial Odometry (VIO) provides 3D pose and velocity estimation using an onboard vision system and IMU. It is used for navigation when global position information is absent or unreliable.
  • Avoidance/Path Planning:
    • Obstacle Avoidance provides full navigation around obstacles when flying a planned path (currently missions are supported). This uses PX4/avoidance (opens new window) running on a companion computer.
    • Collision Prevention is used to stop vehicles before they can crash into an obstacle (primarily when flying in manual modes).
    • Safe Landing guides vehicles to find (and land on) flat terrain that is free of stationary obstacles.

TIP

The PX4 Vision Autonomy Development Kit (Holybro) is a robust and inexpensive kit for developers working with computer vision on PX4. It comes with PX4 avoidance (opens new window) software pre-installed, and can be used as the base for your own algorithms.

# External Resources