Skip to content

Citing and credits

Citation

There is no paper on deeperfly yet: please cite the software, as below or with GitHub's "Cite this repository" button, which reads CITATION.cff.

@software{lam_deeperfly_2026,
  author = {Lam, Thomas Ka Chung},
  title  = {deeperfly: markerless {3D} pose estimation of \textit{Drosophila} from multi-camera rigs},
  year   = {2026},
  url    = {https://github.com/NeLy-EPFL/deeperfly},
}

Please also cite the work behind the parts you use:

  • NeuroMechFly v2, if you use the inverse kinematics or the mesh overlay: Wang-Chen, S., Stimpfling, V. A., Lam, T. K. C., Özdil, P. G., Genoud, L., Hurtak, F., & Ramdya, P. (2024). NeuroMechFly v2: simulating embodied sensorimotor control in adult Drosophila. Nature Methods, 21, 2353–2362. doi:10.1038/s41592-024-02497-y
  • DINOv3, if you use the released detectors or train a mvt detector: Siméoni, O., Vo, H. V., Seitzer, M., Baldassarre, F., et al. (2025). DINOv3. arXiv:2508.10104
  • Lightning Pose 3D, if you use [smoothing], whose ensemble Kalman smoother deeperfly reimplements: Aharon, L., Whiteway, M. R., et al. (2026). Lightning Pose 3D: an uncertainty-aware framework for data-efficient multi-view animal pose estimation. bioRxiv. doi:10.64898/2026.04.20.719731
  • DeepFly3D, deeperfly's predecessor and the source of its skeleton, its pictorial-structures 3D method, and a pretraining dataset: Günel, S., Rhodin, H., Morales, D., Campagnolo, J., Ramdya, P., & Fua, P. (2019). DeepFly3D, a deep learning-based approach for 3D limb and appendage tracking in tethered, adult Drosophila. eLife, 8, e48571. doi:10.7554/eLife.48571

Acknowledgments

deeperfly is developed in the Neuroengineering Laboratory (Ramdya lab) at EPFL. It succeeds DeepFly3D and the lab's first version of deeperfly (with a contribution from Jasper Phelps), which it rewrites from scratch, and builds on the following work.

Methods reimplemented from the literature

Work Where in deeperfly
Lightning Pose: Biderman, D., Whiteway, M. R., et al. (2024), Nature Methods 21, 1316–1328, doi:10.1038/s41592-024-02319-1 the pose-PCA and multi-view consistency training losses, the soft-argmax readout, an importer for its label exports, and ideas for the labeling workflow
Lightning Pose 3D: Aharon, L., Whiteway, M. R., et al. (2026), bioRxiv, doi:10.64898/2026.04.20.719731 the nonlinear ensemble Kalman smoother ([smoothing])
DeepFly3D: Günel, S., et al. (2019), eLife 8, e48571, doi:10.7554/eLife.48571 the pictorial-structures 3D method and the fly38 skeleton, which derives from DeepFly3D's
Ispizua, J. I., Abe, E. T. T., et al. (2026), bioRxiv, doi:10.64898/2026.05.03.722293 the design of multi-animal detection: center heatmaps, triangulated across cameras and tracked over time
SimCC: Li, Y., et al. (2022), ECCV, arXiv:2107.03332 the coordinate-classification detector head
DARK: Zhang, F., et al. (2020), CVPR, arXiv:1910.06278 sub-pixel decoding of heatmaps
ViTPose: Xu, Y., et al. (2022), NeurIPS, arXiv:2204.12484 the optional deconvolution heatmap head
Umeyama, S. (1991), IEEE TPAMI 13, 376–380, doi:10.1109/34.88573 the similarity alignment in the inverse kinematics

Models, data, and libraries

Vendored files, each with its license beside it:

  • MuJoCo's WebAssembly build (Apache-2.0) and three.js (MIT), in docs/keypoints/vendor/, for the docs' 3D keypoint viewer.
  • Lucide icons (ISC), in src/deeperfly/gui/web/static/icons.svg.

The example clips in examples/ were recorded in the Ramdya lab.

Use of AI

deeperfly was developed with extensive help from AI coding assistants, mainly Claude Code (Anthropic). Directed by the author, they wrote much of the code, tests, and documentation, including this README, and ran many of the training and evaluation experiments behind the released detectors. The author set the goals, made the design decisions recorded in decisions/, reviewed the changes, and checked the results against data, and is responsible for the software. As with any tool, validate its output on your own recordings before relying on it.