Skip to content

Quickstart

Run deeperfly on the example clip that ships with the repository: two seconds of a tethered fly walking on a ball, filmed by eight cameras. You get the fly's 3D pose, its joint angles and a video to check them in.

1. Install

You need git, uv and Rust. If cargo --version finds nothing, install Rust, then open a new terminal:

curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh

Install deeperfly:

git clone https://github.com/NeLy-EPFL/deeperfly
cd deeperfly
uv tool install --editable --torch-backend auto ".[detect,ik,ball]"

This installs the deeperfly command from the clone, which also holds the example clip and its config. A GPU is not required, but the example takes a minute and a half with one and about 25 minutes on a 32-core CPU. Installation covers the options.

2. Run the pipeline

From the clone:

deeperfly run examples/octacam -c examples/octacam.toml

The first argument is the folder of videos, and -c is the run config, a TOML file that says what to compute. The first run downloads the detector for this rig (384 MB) and the model that finds the ball (3.4 GB). Add --device cpu on a machine without a GPU.

3. Look at the result

Everything is written to a new folder beside the videos, examples/octacam/deeperfly/:

File What it is
renders/<run>/ball.mp4 The eight views with the fitted pose drawn over them
results/<run>.h5 The arrays: 2D and 3D keypoints, joint angles, the ball

<run> is the time the run started, such as 261011_143005. Watch the video first. Then read the arrays in Python. In the clone, uv run python starts a Python that has deeperfly:

from deeperfly.results import Results

with Results.open("examples/octacam/deeperfly/results/261011_143005.h5") as res:
    points = res.read_animal("pose3d/points", 0)  # (frames, keypoints, 3)
    angles = res.read_animal("ik/angles", 0)  # (frames, joint angles), in radians
    print(res.keypoints)  # the keypoint names, in order

4. Open it in the GUI

deeperfly gui examples/octacam

The GUI opens in the browser. It shows the run's keypoints on the frames, and it is where you correct them and label new frames. deeperfly gui --stop shuts it down.

Next

  • The tutorial goes through the same run step by step: what the config says, how to rerun part of it, and how to draw other videos.
  • Your own recordings explains what you need to run deeperfly on your videos.
  • Labeling and Training a detector are for when the released detectors do not fit your recordings.