Skip to content

Glossary

The words these docs use, grouped by topic.

Projects and videos

  • Project: A folder holding deeperfly.toml. It says which videos it works on, and holds everything deeperfly makes for them: labels, calibrations, runs, rendered videos. It usually sits as deeperfly/ inside the folder of videos.
  • Manifest: A project's deeperfly.toml: its videos, views, skeleton and number of animals.
  • Video: The frames one camera recorded. It may be one file, several files played in order, or a folder of images.
  • View: One camera's look at the animal. A video usually carries one view. On a rig with prisms or mirrors, one video carries several, and the reflected ones are mirrored views.
  • Rig: All the views of a project, in order. This order is the V axis of every array.
  • Skeleton: The keypoints, the bones between them, and which keypoints are left-right pairs. fly38 is the packaged one.
  • Animal: One tracked animal. A project says how many it has, and they are numbered from 0.
  • Collection: Several projects opened together in the GUI, or trained on together. In a collection each project is named after its folder: /data/260730_001/deeperfly is 260730_001.

Cameras

  • Calibration: The position, orientation and optics of every view. A project keeps its calibrations as numbered files under calibrations/.
  • Active calibration: The calibration the GUI uses while labeling. A run uses it only if its config says from = "active".
  • Bundle adjustment: The fit that refines a calibration, by making 2D keypoints from different views agree on 3D positions.
  • Reprojection: A 3D point projected back into a view. The reprojection error is its distance, in pixels, from the 2D keypoint in that view.

Labels

  • Label: A keypoint you placed by hand in one view of one frame. A project's labels are in its labels.h5.
  • Exclude: A mark that keeps a keypoint in one view out of training.
  • Absent: A mark that says a keypoint is not on the animal, as for a missing leg.
  • Complete frame: A labeled frame in which every keypoint that is not absent has a label in at least two views, enough to compute its 3D position.
  • Detections: The 2D keypoints the project's runs found. The GUI shows them, and suggests frames to label from them.

Runs

  • Run: The result of deeperfly run on one project: results/<name>.h5 and the videos under renders/<name>/.
  • Run config: The TOML file given to deeperfly run -c. It says which stages to run and how. It never names the videos, so one config serves every project of a rig.
  • Stage: One step of the pipeline, enabled by its table in the run config: pose2d, bundle_adjustment, pose3d, smoothing, correction, ik, ball, render.
  • Detector: The neural network that finds the keypoints in a view. It is what [pose2d] weights names.
  • Locator: A small network that finds the animal in a whole frame, so that the detector can be given a crop around it.
  • Crop: The rectangle of a frame the detector is given. The run config calls it a window.
  • Stretch: A range of consecutive frames. Stages that work over time, such as smoothing, treat each stretch on its own.
  • Coverage: Which frames of a run a stage has computed.