Output¶

What a run writes, and how to read it.

The files¶

A run is written into the project folder:

File	What it is
`results/<name>.h5`	Every array, in one HDF5 file, with one group per stage.
`results/<name>.toml`	The run config, as it ran.
`renders/<name>/<output>.mp4`	One video per `[render.outputs.<output>]` table.

<name> is the run's name: --name, or the config's name, or the time the run started, such as 260925_184059. With --out <folder>, the files go to <folder>/<project>/ instead.

Reading a run in Python¶

The deeperfly command is installed in an environment of its own, so your Python does not see the package yet. Use the clone's Python, or add the clone to your own project, which needs Python 3.14:

uv run python                      # in the clone: a Python that has deeperfly
uv add --editable /path/to/clone   # in your own uv project

from deeperfly.results import Results

with Results.open("examples/octacam/deeperfly/results/tutorial.h5") as res:
    pts = res.read("pose3d/points")  # (N, T, K, 3)
    one = res.read_animal("pose3d/points", 0)  # (T, K, 3)
    front = res.select("pose2d/points", views=["camera_F"])  # by view name
    legs = res.select("smoothing/points", points=["lf_*", "rf_*"])  # by keypoint
    some = res.read_frames("pose3d/points", [10, 20])  # (N, 2, K, 3)
    done = res.covered("pose3d")  # (T,) bool: the frames computed
    skeleton = res.skeleton()  # keypoints and bones
    calib = res.calibration()  # the cameras the 3D was computed with

res.views and res.keypoints are the view and keypoint names, in the order of the V and K axes. res.axes(ref) gives an array's axes, and res.names(ref) the names along a named axis, such as the joint angles of ik/angles.

To export a table, reshape:

import numpy as np

with Results.open(path) as res:
    pts = res.read_animal("pose3d/points", 0)  # (T, K, 3)
    header = ",".join(f"{k}_{c}" for k in res.keypoints for c in "xyz")
np.savetxt("pose3d.csv", pts.reshape(len(pts), -1), delimiter=",", header=header, comments="")

The arrays¶

The axes are N animals, V views, T frames and K keypoints. C and D are counts that depend on the stage. 2D positions are in pixels of the full video frame, and 3D positions in the calibration's length unit. A missing value is NaN.

Array	Axes	What it is
`pose2d/points`	`(N, V, T, K, 2)`	The detector's 2D keypoints.
`pose2d/conf`	`(N, V, T, K)`	Their confidence, from 0 to 1.
`pose2d/crops`	`(N, V, T, 4)`	The crop the detector saw, `[x0, y0, x1, y1]`. With `window = "static"` it is the same in every frame.
`pose2d/candidates/xy`	`(N, V, T, K, C, 2)`	The `C` best guesses per keypoint, when `candidates` is set. The first is `pose2d/points`.
`pose2d/candidates/score`	`(N, V, T, K, C)`	Their scores.
`pose2d/centers`	`(N, T, 3)`	Each tracked animal's 3D center.
`pose2d/track_flags`	`(N, T)`	Per animal and frame: 0 tracked, 1 held through a gap, 2 lost.
`pose2d/center_candidates`	`(V, T, C, 3)`	The animals found in each view before tracking, as `(x, y, score)`.
`bundle_adjustment/reproj_before`	`(V, T, K)`	The reprojection error on the sampled frames, before the calibration was refined.
`bundle_adjustment/reproj_after`	`(V, T, K)`	The same, after.
`pose3d/points`	`(N, T, K, 3)`	The triangulated 3D keypoints. `NaN` where fewer than `min_views` views agree, or where the agreeing views all look along one line and fix no depth.
`pose3d/reproj_error`	`(N, V, T, K)`	The distance in pixels between each 3D point, projected into a view, and the 2D it was fitted to.
`pose3d/inliers`	`(N, V, T, K)`	Which views agreed on each point.
`pose3d/support`	`(N, T, K)`	How well supported each point is: the sum of the squared confidences of the agreeing views. Do not trust a point below 1.5.
`pose3d/selected/points`	`(N, V, T, K, 2)`	The candidate chosen in each view, with `method = "pictorial"` or `"election"`.
`pose3d/selected/conf`	`(N, V, T, K)`	Its score.
`smoothing/points`	`(N, T, K, 3)`	The smoothed 3D keypoints.
`smoothing/var`	`(N, T, K, 3)`	Their variance.
`smoothing/smooth_param`	`(N, K)`	The smoothing strength fitted for each keypoint.
`correction/points`	`(N, T, K, 3)`	The 3D keypoints after the anatomical corrections.
`ik/angles`	`(N, T, D)`	The joint angles of the fitted body model, in radians. `names` are the joint angles.
`ik/points`	`(N, T, K, 3)`	Where the fitted model puts each keypoint.
`ik/chain_scale`	`(N, C)`	The size of each leg and body part relative to the model's. `names` are the parts.
`ik/chain_offset`	`(N, C, 3)`	Where each part attaches, relative to the model's.
`ball/center`	`(T, 3)`	The ball's center.
`ball/center_sd`	`(T, 3)`	The uncertainty of the center along x, y and z.
`ball/radius`	`()`	The ball's radius.
`ball/rotation`	`(T, 3)`	The ball's orientation since the first frame of a continuous stretch, as a rotation vector in radians.
`ball/step`	`(T, 3)`	Its rotation since the previous frame.
`ball/axes`	`(3, 3)`	The animal's forward, left and up directions where it stands on the ball, as rows.
`ball/tracks`	`(T, D)`	spintrack's own columns: `heading`, `speed`, `forward_total`, `dr_lab_x` and the rest. `names` are the columns.

Some things to know:

The N axis is always there. With one animal it has length 1, so code written for one file works on any. res.read_animal(ref, 0) drops it.
Arrays without N describe the recording, not an animal: the calibration errors, pose2d/center_candidates, and the ball.
The three tracking arrays (pose2d/centers, track_flags, center_candidates) exist only with several animals and window = "per_frame".
Reprojections are not stored. Project a 3D array through res.calibration() to get one. A render layer does this by itself.

Frames that were not computed¶

Every array with a T axis has an entry for every frame of the video, whatever frames the run was asked for. A frame that was not computed holds NaN (0 in pose2d/conf and the scores, 2 in pose2d/track_flags). res.covered("pose3d") says which frames a stage has computed.

Running again under the same name computes the frames that are missing, as described in Configuration. The GUI can read a run while it is being written, but two deeperfly run commands cannot write the same run at once.

ball/rotation restarts at zero wherever the frames computed are not contiguous.

Several animals¶

Which animal is number 0 is the tracker's choice, and it can mix two animals up. The GUI's Runs tab swaps them back over a range of frames. That removes the later stages of the run, which were computed from the old assignment; run the same config under the same name to recompute them.

[bundle_adjustment] fits animal 0 only.

Inside the file¶

The file carries what is needed to interpret it without the project. Its root attributes:

Attribute	What it is
`format_version`	The version of this layout, 6 today.
`deeperfly_version`	The deeperfly that wrote it.
`project`	The project's name.
`created`, `written`	When the file was started, and last written.
`views`, `keypoints`	The names along the `V` and `K` axes, as JSON lists.
`n_frames`, `animals`	The lengths of the `T` and `N` axes.
`fps`	The videos' frame rate.
`frame_sizes`	Each view's frame size, as JSON `{view: [height, width]}`.
`videos`	Each video's files, relative to the project folder.
`config`	The run config, as text.
`skeleton`	The skeleton, as text.
`calibration`	The camera parameters the run started from, as text.

Each stage is a group named after it, holding its arrays, a covered dataset of shape (T,), and the attributes created, duration_s, deeperfly_version and fingerprint. The camera parameters the 3D stages used are in the views attribute of the views group: the refined ones if [bundle_adjustment] ran, and otherwise the initial ones.

Without deeperfly¶

A results file is plain HDF5 with gzip compression, so MATLAB (h5read), R (rhdf5) and Julia (HDF5.jl) read it directly, and h5ls -r <name>.h5 lists its contents. Three things to know:

A stage's covered dataset says which frames hold values.
The name lists are JSON strings. In MATLAB: jsondecode(h5readatt(f, "/", "keypoints")).
MATLAB and Julia see the axes in reverse order: pose3d/points, (N, T, K, 3) here, is 3×K×T×N there.