SoftwarePilot is an open source middleware and API that supports aerial applications. SoftwarePilot allows users to connect consumer DJI drones to programmable Java routines that provide access to the drones flight controller, camera, and navigation system as well as computer vision and deep learning software packages like OpenCV, DLIB, and Tensorflow.
SoftwarePilot is used by researchers to create and test aerial systems that use novel autonomy policies, architectural configurations, and vision algorithms with application domains ranging from agriculture to autonomous photography. Educators also use SoftwarePilot to teach middleschool to university students about drones, autonomous systems, computational thinking, and programming in general.
SoftwarePilot comes with a dockerfile and installation scripts for all requisite software, as well as an Android-x86 virtual machine to communicate with DJI drones from most systems.
SoftwarePilot is developed by the ReRout Lab at The Ohio State University.
SoftwarePilot can be used to create innumerable aerial applications, ranging from simple automated flight to fully autonomous routines. Routines for a range of demos are provided in our Github repository. Below are examples of our autonomous facial recognition routines used for experiments in a recent paper. The first demo performs target recognition and optimization, repositioning itself to take a high quality facial image using reinforcement learning and Dlib facial recognition.
The second demo performs target recognition and avoidance. Some UAV may want to avoid targets that can only be discerned with deep models. This demo uses reinforcement learning and deep facial recognition models to find and avoid humans.
SoftwarePilot makes it easy to program drone actions based on sensed data. Our vision system allows for feature extraction of drone-sensed images using any algorithms that can run on your system. Currently, our vision system explicitly supports DLib, OpenCV, and Tensorflow, with YOLOv3 support coming soon.
The ReRoutlab performs demonstractions of our software and seminars on drones and autonomous systems using SoftwarePilot at schools, universities, and conferences. If you are interested in our educational demonstrations, please contact us.
For comprehensive installation instructions, please view our user guide.
SoftwarePilot provides an easy to use interface for users who wish to run our demos and aerial applications with no development required. Follow our user guide to learn about the suite of demos we provide, commands, and setup.
SoftwarePilot's code is all open source. We encourage users to build their own routines using our API, models, and demo routines. Developers can follow our developement guide to learn more about adding to our API and creating new routines.
Aerial applications can be difficult to implement and even harder test, especially when machine learning and computer vision are involved. The SoftwarePilot project has collected a number of aerial image datasets that can be used to train and test vision and reinforcement learning algorithms meant for execution in real aerial applications.
We have implemented simulators capable of executing test aerial workloads using our datasets, as well as calculating drone and compute energy expenditure from data we have gathered from profiling real systems. Flight simulators abstract away real aerial movement, retrieving sensed data from our image datasets. Energy simulators turn aerial movement traces and profile information into estimated drone and edge energy consumption.
We provide these simulators and datasets free of charge for all to use. Developers can use our tools to create new models, or test autonomous routines. Researchers can use our tools to test the performance of novel autonomy policies, models, and hardware on new and existing aerial applications. For more information about available datasets and simulators, click here.