Skip to content

ball

[ball]: the treadmill ball under a tethered animal, in the world frame.

The ball's outline in every view that sees it places the ball, its center on each frame (filtered and smoothed) and its radius (ball.outline); the 3D pose gives the first guess and orients the animal on it (ball.contact); spintrack, an optional extra (ball._spintrack), tracks its rotation in one view's video, which deeperfly brings into its own world frame (ball.track). Neither package reads the other's files (ADR-0144).

CenterTrack dataclass

The ball's center and the views' offsets over a stretch, filtered forward.

Attributes:

Name Type Description
state Float[ndarray, 'T S']

The filtered center and offsets, (T, 3 + V): each from the frames up to it.
cov Float[ndarray, 'T S S']

Their covariances, (T, 3 + V, 3 + V).
seen ndarray

Whether a frame's outline was measured, rather than only predicted.
wander Float[ndarray, S]

The random walks' variances per frame, (3 + V,).
Source code in src/deeperfly/ball/outline.py 
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
@dataclass(frozen=True)
class CenterTrack:
    """The ball's center and the views' offsets over a stretch, filtered forward.

    Attributes:
        state: The filtered center and offsets, `(T, 3 + V)`: each from the frames up
            to it.
        cov: Their covariances, `(T, 3 + V, 3 + V)`.
        seen: Whether a frame's outline was measured, rather than only predicted.
        wander: The random walks' variances per frame, `(3 + V,)`.
    """

    state: Float[np.ndarray, "T S"]
    cov: Float[np.ndarray, "T S S"]
    seen: np.ndarray
    wander: Float[np.ndarray, "S"]

    def smoothed(self) -> tuple[np.ndarray, np.ndarray]:
        """The centers `(T, 3)` and their covariances from every frame (RTS)."""
        state, cov = self.state.copy(), self.cov.copy()
        walk = np.diag(self.wander)
        for t in range(len(state) - 2, -1, -1):
            ahead = self.cov[t] + walk  # the prediction of frame t + 1 from t
            gain = self.cov[t] @ np.linalg.inv(ahead)
            state[t] = self.state[t] + gain @ (state[t + 1] - self.state[t])
            cov[t] = self.cov[t] + gain @ (cov[t + 1] - ahead) @ gain.T
        return state[:, :3], cov[:, :3, :3]

smoothed 

smoothed() -> tuple[np.ndarray, np.ndarray]

The centers (T, 3) and their covariances from every frame (RTS).

Source code in src/deeperfly/ball/outline.py 
204
205
206
207
208
209
210
211
212
213
def smoothed(self) -> tuple[np.ndarray, np.ndarray]:
    """The centers `(T, 3)` and their covariances from every frame (RTS)."""
    state, cov = self.state.copy(), self.cov.copy()
    walk = np.diag(self.wander)
    for t in range(len(state) - 2, -1, -1):
        ahead = self.cov[t] + walk  # the prediction of frame t + 1 from t
        gain = self.cov[t] @ np.linalg.inv(ahead)
        state[t] = self.state[t] + gain @ (state[t + 1] - self.state[t])
        cov[t] = self.cov[t] + gain @ (cov[t + 1] - ahead) @ gain.T
    return state[:, :3], cov[:, :3, :3]

Contact dataclass

The ball, and the animal's axes where it stands on it, in world coordinates.

Attributes:

Name Type Description
axes Float[ndarray, '3 3']

World to animal, (3, 3): its rows are the animal's forward, left and up axes in world coordinates.
center Float[ndarray, 3]

The ball's center, (3,).
radius float

The ball's radius, in world units.
on_surface float

The share of the tips that rest on the ball.
height float

The stance centroid above the surface, in ball radii.
level bool

Whether the world's up stands in for the ball's normal, the animal not believably standing on the ball.
Source code in src/deeperfly/ball/contact.py 
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
@dataclass(frozen=True)
class Contact:
    """The ball, and the animal's axes where it stands on it, in world coordinates.

    Attributes:
        axes: World to animal, `(3, 3)`: its rows are the animal's forward, left and
            up axes in world coordinates.
        center: The ball's center, `(3,)`.
        radius: The ball's radius, in world units.
        on_surface: The share of the tips that rest on the ball.
        height: The stance centroid above the surface, in ball radii.
        level: Whether the world's up stands in for the ball's normal, the animal not
            believably standing on the ball.
    """

    axes: Float[np.ndarray, "3 3"]
    center: Float[np.ndarray, "3"]
    radius: float
    on_surface: float
    height: float
    level: bool = False

    def line(self) -> str:
        """One log line: where the animal stands on the ball."""
        x, _, z = self.axes
        heading = np.degrees(np.arctan2(x[1], x[0]))
        rests = f"{100 * self.on_surface:.0f}% of the leg tips rest on it"
        if self.level:
            return (
                f"{rests} and the body is {self.height:.2f} radii above it, so the "
                f"world's up stands in for its normal; the body points {heading:+.1f} "
                "deg from the world's x"
            )
        tilt = np.degrees(np.arccos(np.clip(z[2], -1.0, 1.0)))
        toward = np.degrees(np.arctan2(z[1], z[0]))
        return (
            f"{rests}; the animal stands {tilt:.1f} deg from its top (toward "
            f"{toward:.0f} deg), {self.height:.2f} radii above it, and points "
            f"{heading:+.1f} deg from the world's x"
        )

line 

line() -> str

One log line: where the animal stands on the ball.

Source code in src/deeperfly/ball/contact.py 
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
def line(self) -> str:
    """One log line: where the animal stands on the ball."""
    x, _, z = self.axes
    heading = np.degrees(np.arctan2(x[1], x[0]))
    rests = f"{100 * self.on_surface:.0f}% of the leg tips rest on it"
    if self.level:
        return (
            f"{rests} and the body is {self.height:.2f} radii above it, so the "
            f"world's up stands in for its normal; the body points {heading:+.1f} "
            "deg from the world's x"
        )
    tilt = np.degrees(np.arccos(np.clip(z[2], -1.0, 1.0)))
    toward = np.degrees(np.arctan2(z[1], z[0]))
    return (
        f"{rests}; the animal stands {tilt:.1f} deg from its top (toward "
        f"{toward:.0f} deg), {self.height:.2f} radii above it, and points "
        f"{heading:+.1f} deg from the world's x"
    )

MissingSpintrack

Bases: ImportError

Raised when the ball is asked for without the ball extra installed.

Source code in src/deeperfly/ball/_spintrack.py 
21
22
class MissingSpintrack(ImportError):
    """Raised when the ball is asked for without the `ball` extra installed."""

OutlineFilter

The ball's center, frame by frame, with each view's offset: random walks.

Each frame predicts the last state, its covariance grown by the walks, and fits the outlines with that prediction as the prior; a frame no view shows the outline in keeps the prediction. Live tracking steps it as frames come.

Source code in src/deeperfly/ball/outline.py 
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
class OutlineFilter:
    """The ball's center, frame by frame, with each view's offset: random walks.

    Each frame predicts the last state, its covariance grown by the walks, and fits the
    outlines with that prediction as the prior; a frame no view shows the outline in
    keeps the prediction. Live tracking steps it as frames come.
    """

    def __init__(
        self,
        views: dict[str, View],
        radius: float,
        center: np.ndarray,
        cov: np.ndarray,
    ) -> None:
        """Start at `center` with covariance `cov`, each offset at 0 within `OFFSET_PX`.

        The radius is held.
        """
        self.views, self.radius = views, radius
        n_views = len(views)
        self.state = np.r_[np.asarray(center, dtype=float), np.zeros(n_views)]
        self.cov = np.zeros((3 + n_views,) * 2)
        self.cov[:3, :3] = cov
        self.cov[3:, 3:] = np.eye(n_views) * OFFSET_PX**2
        self.wander = np.r_[
            np.full(3, (WANDER_RADII * radius) ** 2),
            np.full(n_views, OFFSET_WANDER_PX**2),
        ]
        self._widths: tuple[float, ...] = WIDTHS

    @property
    def center(self) -> np.ndarray:
        """The ball's center now."""
        return self.state[:3]

    def step(self, images: dict[str, np.ndarray]) -> bool:
        """Take one frame's `gray_frame`s; whether its outline was measured.

        A frame the narrow search finds no outline in is searched again wide: a ball
        that jerks moves farther in a frame than the narrow search reaches.
        """
        cov = self.cov + np.diag(self.wander)
        for widths in dict.fromkeys((self._widths, WIDTHS)):
            try:
                fit = fit_outline(
                    self.views,
                    images,
                    self.center,
                    self.radius,
                    prior=(self.state, cov),
                    widths=widths,
                )
            except ValueError:
                continue
            self.state = np.r_[fit.center, fit.offsets]
            self.cov, self._widths = fit.cov, WIDTHS[1:]
            return True
        self.cov = cov
        return False

center property 

center: ndarray

The ball's center now.

__init__ 

__init__(
    views: dict[str, View],
    radius: float,
    center: ndarray,
    cov: ndarray,
) -> None

Start at center with covariance cov, each offset at 0 within OFFSET_PX.

The radius is held.

Source code in src/deeperfly/ball/outline.py 
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
def __init__(
    self,
    views: dict[str, View],
    radius: float,
    center: np.ndarray,
    cov: np.ndarray,
) -> None:
    """Start at `center` with covariance `cov`, each offset at 0 within `OFFSET_PX`.

    The radius is held.
    """
    self.views, self.radius = views, radius
    n_views = len(views)
    self.state = np.r_[np.asarray(center, dtype=float), np.zeros(n_views)]
    self.cov = np.zeros((3 + n_views,) * 2)
    self.cov[:3, :3] = cov
    self.cov[3:, 3:] = np.eye(n_views) * OFFSET_PX**2
    self.wander = np.r_[
        np.full(3, (WANDER_RADII * radius) ** 2),
        np.full(n_views, OFFSET_WANDER_PX**2),
    ]
    self._widths: tuple[float, ...] = WIDTHS

step 

step(images: dict[str, ndarray]) -> bool

Take one frame's gray_frames; whether its outline was measured.

A frame the narrow search finds no outline in is searched again wide: a ball that jerks moves farther in a frame than the narrow search reaches.

Source code in src/deeperfly/ball/outline.py 
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
def step(self, images: dict[str, np.ndarray]) -> bool:
    """Take one frame's `gray_frame`s; whether its outline was measured.

    A frame the narrow search finds no outline in is searched again wide: a ball
    that jerks moves farther in a frame than the narrow search reaches.
    """
    cov = self.cov + np.diag(self.wander)
    for widths in dict.fromkeys((self._widths, WIDTHS)):
        try:
            fit = fit_outline(
                self.views,
                images,
                self.center,
                self.radius,
                prior=(self.state, cov),
                widths=widths,
            )
        except ValueError:
            continue
        self.state = np.r_[fit.center, fit.offsets]
        self.cov, self._widths = fit.cov, WIDTHS[1:]
        return True
    self.cov = cov
    return False

OutlineFit dataclass

A sphere fitted to the outlines.

Attributes:

Name Type Description
center Float[ndarray, 3]

Its center, world coordinates.
radius float

Its radius, world units.
offsets Float[ndarray, V]

Each view's edges' offset from the outline, in pixels, in the views' order.
cov Float[ndarray, 'S S']

The covariance of the center and the offsets, (3 + V, 3 + V), the prior's included.
edges dict[str, int]

How many edges each view gave, by view name.
residual_px float

The median edge offset from the fitted outline, in pixels.
Source code in src/deeperfly/ball/outline.py 
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
@dataclass(frozen=True)
class OutlineFit:
    """A sphere fitted to the outlines.

    Attributes:
        center: Its center, world coordinates.
        radius: Its radius, world units.
        offsets: Each view's edges' offset from the outline, in pixels, in the
            views' order.
        cov: The covariance of the center and the offsets, `(3 + V, 3 + V)`, the
            prior's included.
        edges: How many edges each view gave, by view name.
        residual_px: The median edge offset from the fitted outline, in pixels.
    """

    center: Float[np.ndarray, "3"]
    radius: float
    offsets: Float[np.ndarray, "V"]
    cov: Float[np.ndarray, "S S"]
    edges: dict[str, int]
    residual_px: float

Stance dataclass

The 3D points the ball is guessed from and the animal oriented by.

Attributes:

Name Type Description
tips Float[ndarray, 'P 3']

The finite points that rest on the ball in stance, (P, 3).
body Float[ndarray, 3]

The centroid of the points the animal stands on, NaN if unknown.
body_axis Float[ndarray, 3]

Hind to front, NaN if unknown.
Source code in src/deeperfly/ball/contact.py 
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
@dataclass(frozen=True)
class Stance:
    """The 3D points the ball is guessed from and the animal oriented by.

    Attributes:
        tips: The finite points that rest on the ball in stance, `(P, 3)`.
        body: The centroid of the points the animal stands on, NaN if unknown.
        body_axis: Hind to front, NaN if unknown.
    """

    tips: Float[np.ndarray, "P 3"]
    body: Float[np.ndarray, "3"]
    body_axis: Float[np.ndarray, "3"]

    @classmethod
    def of(
        cls,
        tips: Float[np.ndarray, "T P 3"],
        stand: Float[np.ndarray, "T Q 3"],
        front: Float[np.ndarray, "T F 3"],
        hind: Float[np.ndarray, "T H 3"],
    ) -> "Stance":
        """The stance over some frames of world points, NaN where unknown.

        Each point's median over the frames, then the centroids.
        """
        with warnings.catch_warnings():  # a point never seen is NaN, and stays NaN
            warnings.simplefilter("ignore", RuntimeWarning)
            front_c = np.nanmedian(np.nanmean(front, axis=1), axis=0)
            hind_c = np.nanmedian(np.nanmean(hind, axis=1), axis=0)
            body = np.nanmean(np.nanmedian(stand, axis=0), axis=0)
        tips = tips.reshape(-1, 3)
        return cls(tips[np.isfinite(tips).all(axis=1)], body, front_c - hind_c)

of classmethod 

of(
    tips: Float[ndarray, "T P 3"],
    stand: Float[ndarray, "T Q 3"],
    front: Float[ndarray, "T F 3"],
    hind: Float[ndarray, "T H 3"],
) -> Stance

The stance over some frames of world points, NaN where unknown.

Each point's median over the frames, then the centroids.

Source code in src/deeperfly/ball/contact.py 
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
@classmethod
def of(
    cls,
    tips: Float[np.ndarray, "T P 3"],
    stand: Float[np.ndarray, "T Q 3"],
    front: Float[np.ndarray, "T F 3"],
    hind: Float[np.ndarray, "T H 3"],
) -> "Stance":
    """The stance over some frames of world points, NaN where unknown.

    Each point's median over the frames, then the centroids.
    """
    with warnings.catch_warnings():  # a point never seen is NaN, and stays NaN
        warnings.simplefilter("ignore", RuntimeWarning)
        front_c = np.nanmedian(np.nanmean(front, axis=1), axis=0)
        hind_c = np.nanmedian(np.nanmean(hind, axis=1), axis=0)
        body = np.nanmean(np.nanmedian(stand, axis=0), axis=0)
    tips = tips.reshape(-1, 3)
    return cls(tips[np.isfinite(tips).all(axis=1)], body, front_c - hind_c)

ball_ray 

ball_ray(
    view: View, circle: tuple[float, float, float]
) -> tuple[np.ndarray, np.ndarray, float]

The ray through the ball's image center, and the sine of its angular radius.

The ray is (origin, unit direction) in the world; the angular radius is the mean angle between it and the rays of four points of the circle's rim.

Source code in src/deeperfly/ball/contact.py 
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
def ball_ray(
    view: View, circle: tuple[float, float, float]
) -> tuple[np.ndarray, np.ndarray, float]:
    """The ray through the ball's image center, and the sine of its angular radius.

    The ray is `(origin, unit direction)` in the world; the angular radius is the mean
    angle between it and the rays of four points of the circle's rim.
    """
    cx, cy, r = circle
    origin, ray = view.backproject_ray(np.array([cx, cy]))
    ray = ray / np.linalg.norm(ray)
    angles = []
    for dx, dy in ((r, 0.0), (-r, 0.0), (0.0, r), (0.0, -r)):
        rim = view.backproject_ray(np.array([cx + dx, cy + dy]))[1]
        angles.append(np.arccos(np.clip(rim @ ray / np.linalg.norm(rim), -1.0, 1.0)))
    return origin, ray, float(np.sin(np.mean(angles)))

detect_circle 

detect_circle(
    video: Video, start: int, stop: int
) -> tuple[float, float, float]

The ball's image circle (cx, cy, r) in deeperfly's pixels, from start on.

It is detected in the first frames of start:stop.

Raises:

Type Description
RuntimeError

spintrack's DetectionError, when the ball cannot be trusted.
Source code in src/deeperfly/ball/track.py 
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
def detect_circle(video: Video, start: int, stop: int) -> tuple[float, float, float]:
    """The ball's image circle `(cx, cy, r)` in deeperfly's pixels, from `start` on.

    It is detected in the first frames of `start:stop`.

    Raises:
        RuntimeError: spintrack's `DetectionError`, when the ball cannot be trusted.
    """
    from spintrack.autofit import find_ball

    stop = min(stop, start + SAMPLE_SPAN)
    step = max(1, SAMPLE_SPAN // SAMPLE_COUNT)
    with video.open() as reader:
        frames = [
            _gray(f)
            for f in reader.stream_frames(
                start=start, stop=stop, step=step, gray_ok=True
            )
        ]
    found = find_ball(frames, len(frames))
    return found.cx - HALF_PIXEL, found.cy - HALF_PIXEL, found.r

fit_outline 

fit_outline(
    views: Mapping[str, View],
    images: dict[str, ndarray],
    center: ndarray,
    radius: float,
    *,
    prior: tuple[ndarray, ndarray] | None = None,
    fit_radius: bool = False,
    widths: Sequence[float] = WIDTHS,
) -> OutlineFit

The sphere whose outline matches the ball's edges in images, by view name.

Starts from center and radius, which must put the predicted outline within the first of widths pixels of the edges. prior is the (mean, covariance) of the center and the views' offsets (in views' order) before this frame, which holds what the outlines barely pin; without one, the offsets are within OFFSET_PX of zero and the views must see the ball from more than one line. fit_radius frees the radius. images are gray_frames.

Raises:

Type Description
ValueError

If no view shows enough outline, or the views look along one line and there is no prior, or the radius is asked of them.
Source code in src/deeperfly/ball/outline.py 
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
def fit_outline(
    views: Mapping[str, View],
    images: dict[str, np.ndarray],
    center: np.ndarray,
    radius: float,
    *,
    prior: tuple[np.ndarray, np.ndarray] | None = None,
    fit_radius: bool = False,
    widths: Sequence[float] = WIDTHS,
) -> OutlineFit:
    """The sphere whose outline matches the ball's edges in `images`, by view name.

    Starts from `center` and `radius`, which must put the predicted outline within
    the first of `widths` pixels of the edges. `prior` is the `(mean, covariance)` of
    the center and the views' offsets (in `views`' order) before this frame, which
    holds what the outlines barely pin; without one, the offsets are within
    `OFFSET_PX` of zero and the views must see the ball from more than one line.
    `fit_radius` frees the radius. `images` are `gray_frame`s.

    Raises:
        ValueError: If no view shows enough outline, or the views look along one line
            and there is no prior, or the radius is asked of them.
    """
    names = list(views)
    n_views = len(names)
    x = np.r_[np.asarray(center, dtype=float), radius]
    free = 4 if fit_radius else 3
    # The unknowns: the center's step, the radius's, and each view's offset; the
    # prior is on the center and the offsets.
    held = [0, 1, 2, *range(free, free + n_views)]
    if prior is None:
        mean, info_held = np.zeros(3 + n_views), np.zeros((3 + n_views,) * 2)
        info_held[3:, 3:] = np.eye(n_views) / OFFSET_PX**2
    else:
        mean, info_held = np.asarray(prior[0], dtype=float), np.linalg.inv(prior[1])
    edges: dict[str, int] = {}
    solved = None
    for width in widths:
        rows, offsets, which, seen = [], [], [], []
        for k, name in enumerate(names):
            jac, s = _measure(views[name], images[name], x, width)
            edges[name] = len(s)
            if len(s) >= MIN_EDGES:
                rows.append(jac[:, :free])
                offsets.append(s)
                which.append(np.full(len(s), k))
                seen.append(views[name].position)
        if not rows:
            raise ValueError("no view shows enough of the ball's outline")
        if (prior is None or fit_radius) and _one_line(np.array(seen), x[:3]):
            raise ValueError(
                "the views that show the ball's outline look along one line"
            )
        jac = np.concatenate(rows)
        own = np.zeros((len(jac), n_views))
        own[np.arange(len(jac)), np.concatenate(which)] = 1.0
        info0 = np.zeros((free + n_views,) * 2)
        info0[np.ix_(held, held)] = info_held
        mean0 = np.zeros(free + n_views)
        mean0[held] = np.r_[mean[:3] - x[:3], mean[3:]]
        solved = _solve(np.hstack([jac, own]), np.concatenate(offsets), info0, mean0)
        x[:free] += solved[0][:free]
    if solved is None:
        raise ValueError("no search widths to fit the outline at")
    step, residual, info = solved
    cov = np.linalg.inv(info)[np.ix_(held, held)]
    return OutlineFit(
        center=x[:3],
        radius=float(x[3]),
        offsets=step[free:],
        cov=cov,
        edges=edges,
        residual_px=residual,
    )

gray_frame 

gray_frame(frame: ndarray) -> np.ndarray

A decoded (H, W, C) frame as the blurred float luma the edges are found on.

Source code in src/deeperfly/ball/outline.py 
105
106
107
108
def gray_frame(frame: np.ndarray) -> np.ndarray:
    """A decoded `(H, W, C)` frame as the blurred float luma the edges are found on."""
    gray = frame[..., 0] if frame.shape[-1] == 1 else frame.mean(axis=-1)
    return gaussian_filter(gray.astype(np.float32), SMOOTH_PX)

grazing_circle 

grazing_circle(
    view: View,
    center: ndarray,
    radius: float,
    n: int = N_POINTS,
) -> np.ndarray

(n, 3) world points of the circle where view's rays graze a sphere.

What view images as the sphere's outline, from a view outside the sphere.

Source code in src/deeperfly/ball/outline.py 
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
def grazing_circle(
    view: View, center: np.ndarray, radius: float, n: int = N_POINTS
) -> np.ndarray:
    """`(n, 3)` world points of the circle where `view`'s rays graze a sphere.

    What `view` images as the sphere's outline, from a view outside the sphere.
    """
    origin = view.position
    axis = np.asarray(center, dtype=float) - origin
    distance = float(np.linalg.norm(axis))
    axis = axis / distance
    helper = np.array([0.0, 0.0, 1.0]) if abs(axis[2]) < 0.9 else np.array([1.0, 0, 0])
    u = np.cross(axis, helper)
    u /= np.linalg.norm(u)
    v = np.cross(axis, u)
    sin_a = radius / distance
    cos_a = np.sqrt(1.0 - sin_a**2)
    phi = np.linspace(0.0, 2.0 * np.pi, n, endpoint=False)
    ring = np.cos(phi)[:, None] * u + np.sin(phi)[:, None] * v
    return origin + distance * cos_a * (cos_a * axis + sin_a * ring)

guess_ball 

guess_ball(
    view: View,
    circle: tuple[float, float, float],
    stance: Stance,
) -> tuple[np.ndarray, float] | None

A first (center, radius) of the ball, from one view and the leg tips.

circle is the ball's image (cx, cy, r) in view's pixels. None with too few tips, or no centroid to start the search from.

Source code in src/deeperfly/ball/contact.py 
119
120
121
122
123
124
125
126
127
128
129
130
131
132
def guess_ball(
    view: View, circle: tuple[float, float, float], stance: Stance
) -> tuple[np.ndarray, float] | None:
    """A first `(center, radius)` of the ball, from one view and the leg tips.

    `circle` is the ball's image `(cx, cy, r)` in `view`'s pixels. None with too few
    tips, or no centroid to start the search from.
    """
    if len(stance.tips) < MIN_TIPS or not np.isfinite(stance.body).all():
        return None
    origin, ray, sin_a = ball_ray(view, circle)
    start = float((stance.body - origin) @ ray)
    depth = _ball_depth(stance.tips, origin, ray, sin_a, start)
    return origin + depth * ray, depth * sin_a

require_spintrack cached 

require_spintrack() -> ModuleType

The spintrack module, or MissingSpintrack saying how to install it.

Source code in src/deeperfly/ball/_spintrack.py 
25
26
27
28
29
30
31
32
33
34
35
36
37
@functools.cache
def require_spintrack() -> ModuleType:
    """The `spintrack` module, or [MissingSpintrack][] saying how to install it."""
    try:
        import spintrack
    except ImportError as exc:  # pragma: no cover -- exercised by the install-less path
        raise MissingSpintrack(
            "tracking the ball needs the optional spintrack tracker, which is not "
            f"installed. Install the extra with:\n    {INSTALL_HINT}\n"
            "It builds a Rust extension, so a Rust toolchain is required "
            "(https://rustup.rs)."
        ) from exc
    return spintrack

rim_points 

rim_points(
    view: View, center: ndarray, radius: float, n: int = 16
) -> np.ndarray

(n, 2) pixels of a sphere's outline in view, evenly around it.

Source code in src/deeperfly/ball/outline.py 
305
306
307
308
309
def rim_points(
    view: View, center: np.ndarray, radius: float, n: int = 16
) -> np.ndarray:
    """`(n, 2)` pixels of a sphere's outline in `view`, evenly around it."""
    return _outline(view, np.r_[np.asarray(center, dtype=float), radius], n)

stand_on 

stand_on(
    center: ndarray, radius: float, stance: Stance
) -> tuple[Contact, str | None]

The animal's axes on the ball, and why the world's up stands in, if it does.

Source code in src/deeperfly/ball/contact.py 
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
def stand_on(
    center: np.ndarray, radius: float, stance: Stance
) -> tuple[Contact, str | None]:
    """The animal's axes on the ball, and why the world's up stands in, if it does."""
    gap = np.linalg.norm(stance.tips - center, axis=1) - radius
    share = float(np.mean(np.abs(gap) < ON_SURFACE * radius)) if len(gap) else 0.0
    normal = stance.body - center
    height = float(np.linalg.norm(normal) / radius - 1.0)
    level = not (share >= MIN_ON_SURFACE and 0.0 < height < MAX_HEIGHT)
    up = UP if level else normal / np.linalg.norm(normal)
    contact = Contact(
        axes=_axes(up, stance.body_axis),
        center=np.asarray(center, dtype=float),
        radius=float(radius),
        on_surface=share,
        height=height,
        level=level,
    )
    why = None
    if level:
        why = (
            f"{100 * share:.0f}% of the leg tips rest on the ball, and the body is "
            f"{height:.2f} radii above it"
        )
    return contact, why

track_ball 

track_ball(
    view: View,
    video: Video,
    start: int,
    stop: int,
    axes: ndarray,
    rim: ndarray,
    *,
    tracking: Mapping[str, Any],
    two_pass: bool = False,
) -> tuple[dict[str, np.ndarray], list[str], Any]

Track the ball's rotation over frames start:stop of video, which view sees.

rim is points of the ball's outline at start, (n, 2) in deeperfly's pixels; axes (world to animal) orients spintrack's animal frame. Returns rotation, step and tracks over the stretch (NaN at a frame spintrack dropped), the names of tracks' columns, and spintrack's quality summary.

Source code in src/deeperfly/ball/track.py 
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
def track_ball(
    view: View,
    video: Video,
    start: int,
    stop: int,
    axes: np.ndarray,
    rim: np.ndarray,
    *,
    tracking: Mapping[str, Any],
    two_pass: bool = False,
) -> tuple[dict[str, np.ndarray], list[str], Any]:
    """Track the ball's rotation over frames `start:stop` of `video`, which `view` sees.

    `rim` is points of the ball's outline at `start`, `(n, 2)` in deeperfly's pixels;
    `axes` (world to animal) orients spintrack's animal frame. Returns `rotation`,
    `step` and `tracks` over the stretch (NaN at a frame spintrack dropped), the names
    of `tracks`' columns, and spintrack's quality summary.
    """
    spintrack = require_spintrack()
    height, width = video.size
    fps = video.fps or 0.0
    cfg = spintrack.Config.model_validate({"tracking": dict(tracking)})
    fy = float(view.kmat[1, 1])
    cfg.camera.vfov_deg = float(np.degrees(2.0 * np.arctan(height / (2.0 * fy))))
    to_animal = axes @ view.rmat.T  # camera to animal
    cfg.camera.rotation = tuple(float(v) for v in np.asarray(rmat_to_rvec(to_animal)))
    if fps > 0:
        cfg.camera.fps = fps
    cfg.ball.rim = [(float(x), float(y)) for x, y in np.asarray(rim) + HALF_PIXEL]

    def frames():
        return _Frames(video, start, stop, fps, (width, height))

    result = spintrack.track(cfg, frames=frames, two_pass=two_pass)
    columns = list(result.columns)
    records = np.asarray(result.records, dtype=np.float64)
    n = stop - start
    rows = records[:, columns.index("frame")].astype(np.int64)
    keep = (rows >= 0) & (rows < n)
    rows, records = rows[keep], records[keep]

    def per_frame(names: list[str]) -> np.ndarray:
        out = np.full((n, len(names)), np.nan)
        out[rows] = records[:, [columns.index(c) for c in names]]
        return out

    to_world = view.rmat  # a camera row vector times R is the world one
    rotation = per_frame(["r_cam_x", "r_cam_y", "r_cam_z"]) @ to_world
    step = per_frame(["dr_cam_x", "dr_cam_y", "dr_cam_z"]) @ to_world
    names = [c for c in columns if c != "frame"]
    arrays = {"rotation": rotation, "step": step, "tracks": per_frame(names)}
    return arrays, names, result.quality

track_outline 

track_outline(
    views: dict[str, View],
    frames: Iterable[dict[str, ndarray]],
    center: ndarray,
    cov: ndarray,
    radius: float,
) -> CenterTrack

The ball's center on each of frames (by view name, gray_frames), filtered.

Starts at center with covariance cov, the first frame with the widest search; the radius is held. CenterTrack.smoothed brings in the later frames.

Source code in src/deeperfly/ball/outline.py 
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
def track_outline(
    views: dict[str, View],
    frames: Iterable[dict[str, np.ndarray]],
    center: np.ndarray,
    cov: np.ndarray,
    radius: float,
) -> CenterTrack:
    """The ball's center on each of `frames` (by view name, `gray_frame`s), filtered.

    Starts at `center` with covariance `cov`, the first frame with the widest search;
    the radius is held. `CenterTrack.smoothed` brings in the later frames.
    """
    tracker = OutlineFilter(views, radius, center, cov)
    states, covs, seen = [], [], []
    for images in frames:
        seen.append(tracker.step(images))
        states.append(tracker.state)
        covs.append(tracker.cov)
    size = 3 + len(views)
    return CenterTrack(
        state=np.array(states).reshape(-1, size),
        cov=np.array(covs).reshape(-1, size, size),
        seen=np.array(seen, dtype=bool),
        wander=tracker.wander,
    )

The ball's center and radius from its outline in several views, filtered over time.

A sphere's outline in a view is the projection of the circle where the camera's rays graze it, so each view's outline pins the sphere across that view's rays, and two or more views pin it in depth too: the outline does what one view's apparent size and the leg tips did before, and the radius comes out with the center. Each view's outline is measured as the bright-to-dark edge along the normals of the predicted one, where the ball is not hidden (a leg crossing the rim leaves no edge, or a weak one), and the center (and the radius) is solved by robust Gauss-Newton on the edges' offsets, in pixels, through deeperfly's camera model, distortion included.

A view's edges all sit off the true outline by an offset of their own, its calibration's error and the edge detector's (octacam's views sat -1.5 to +2.3 px off), which is solved for with the center. One view's apparent size then tells distance only as well as its offset is known: alone, the hind camera's size had put the ball's depth 3.5 units off, two thirds of its radius.

Over a stretch of frames the center and the views' offsets are random walks, filtered (Kalman) and smoothed (Rauch-Tung-Striebel). Each frame's fit takes the prediction as its prior, so a direction the outlines pin follows them and one they barely pin keeps the prior. The offsets drift slowly, so they are learned where views on different lines see the ball and carried where they do not: as when a ball drops out of octacam's side views and only its front and hind cameras, which face each other along one line, still see it. Depth along that line is then carried across from the frames on either side, and the covariance says how well each frame's center is known.

CenterTrack dataclass

The ball's center and the views' offsets over a stretch, filtered forward.

Attributes:

Name Type Description
state Float[ndarray, 'T S']

The filtered center and offsets, (T, 3 + V): each from the frames up to it.
cov Float[ndarray, 'T S S']

Their covariances, (T, 3 + V, 3 + V).
seen ndarray

Whether a frame's outline was measured, rather than only predicted.
wander Float[ndarray, S]

The random walks' variances per frame, (3 + V,).
Source code in src/deeperfly/ball/outline.py 
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
@dataclass(frozen=True)
class CenterTrack:
    """The ball's center and the views' offsets over a stretch, filtered forward.

    Attributes:
        state: The filtered center and offsets, `(T, 3 + V)`: each from the frames up
            to it.
        cov: Their covariances, `(T, 3 + V, 3 + V)`.
        seen: Whether a frame's outline was measured, rather than only predicted.
        wander: The random walks' variances per frame, `(3 + V,)`.
    """

    state: Float[np.ndarray, "T S"]
    cov: Float[np.ndarray, "T S S"]
    seen: np.ndarray
    wander: Float[np.ndarray, "S"]

    def smoothed(self) -> tuple[np.ndarray, np.ndarray]:
        """The centers `(T, 3)` and their covariances from every frame (RTS)."""
        state, cov = self.state.copy(), self.cov.copy()
        walk = np.diag(self.wander)
        for t in range(len(state) - 2, -1, -1):
            ahead = self.cov[t] + walk  # the prediction of frame t + 1 from t
            gain = self.cov[t] @ np.linalg.inv(ahead)
            state[t] = self.state[t] + gain @ (state[t + 1] - self.state[t])
            cov[t] = self.cov[t] + gain @ (cov[t + 1] - ahead) @ gain.T
        return state[:, :3], cov[:, :3, :3]

smoothed 

smoothed() -> tuple[np.ndarray, np.ndarray]

The centers (T, 3) and their covariances from every frame (RTS).

Source code in src/deeperfly/ball/outline.py 
204
205
206
207
208
209
210
211
212
213
def smoothed(self) -> tuple[np.ndarray, np.ndarray]:
    """The centers `(T, 3)` and their covariances from every frame (RTS)."""
    state, cov = self.state.copy(), self.cov.copy()
    walk = np.diag(self.wander)
    for t in range(len(state) - 2, -1, -1):
        ahead = self.cov[t] + walk  # the prediction of frame t + 1 from t
        gain = self.cov[t] @ np.linalg.inv(ahead)
        state[t] = self.state[t] + gain @ (state[t + 1] - self.state[t])
        cov[t] = self.cov[t] + gain @ (cov[t + 1] - ahead) @ gain.T
    return state[:, :3], cov[:, :3, :3]

OutlineFilter

The ball's center, frame by frame, with each view's offset: random walks.

Each frame predicts the last state, its covariance grown by the walks, and fits the outlines with that prediction as the prior; a frame no view shows the outline in keeps the prediction. Live tracking steps it as frames come.

Source code in src/deeperfly/ball/outline.py 
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
class OutlineFilter:
    """The ball's center, frame by frame, with each view's offset: random walks.

    Each frame predicts the last state, its covariance grown by the walks, and fits the
    outlines with that prediction as the prior; a frame no view shows the outline in
    keeps the prediction. Live tracking steps it as frames come.
    """

    def __init__(
        self,
        views: dict[str, View],
        radius: float,
        center: np.ndarray,
        cov: np.ndarray,
    ) -> None:
        """Start at `center` with covariance `cov`, each offset at 0 within `OFFSET_PX`.

        The radius is held.
        """
        self.views, self.radius = views, radius
        n_views = len(views)
        self.state = np.r_[np.asarray(center, dtype=float), np.zeros(n_views)]
        self.cov = np.zeros((3 + n_views,) * 2)
        self.cov[:3, :3] = cov
        self.cov[3:, 3:] = np.eye(n_views) * OFFSET_PX**2
        self.wander = np.r_[
            np.full(3, (WANDER_RADII * radius) ** 2),
            np.full(n_views, OFFSET_WANDER_PX**2),
        ]
        self._widths: tuple[float, ...] = WIDTHS

    @property
    def center(self) -> np.ndarray:
        """The ball's center now."""
        return self.state[:3]

    def step(self, images: dict[str, np.ndarray]) -> bool:
        """Take one frame's `gray_frame`s; whether its outline was measured.

        A frame the narrow search finds no outline in is searched again wide: a ball
        that jerks moves farther in a frame than the narrow search reaches.
        """
        cov = self.cov + np.diag(self.wander)
        for widths in dict.fromkeys((self._widths, WIDTHS)):
            try:
                fit = fit_outline(
                    self.views,
                    images,
                    self.center,
                    self.radius,
                    prior=(self.state, cov),
                    widths=widths,
                )
            except ValueError:
                continue
            self.state = np.r_[fit.center, fit.offsets]
            self.cov, self._widths = fit.cov, WIDTHS[1:]
            return True
        self.cov = cov
        return False

center property 

center: ndarray

The ball's center now.

__init__ 

__init__(
    views: dict[str, View],
    radius: float,
    center: ndarray,
    cov: ndarray,
) -> None

Start at center with covariance cov, each offset at 0 within OFFSET_PX.

The radius is held.

Source code in src/deeperfly/ball/outline.py 
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
def __init__(
    self,
    views: dict[str, View],
    radius: float,
    center: np.ndarray,
    cov: np.ndarray,
) -> None:
    """Start at `center` with covariance `cov`, each offset at 0 within `OFFSET_PX`.

    The radius is held.
    """
    self.views, self.radius = views, radius
    n_views = len(views)
    self.state = np.r_[np.asarray(center, dtype=float), np.zeros(n_views)]
    self.cov = np.zeros((3 + n_views,) * 2)
    self.cov[:3, :3] = cov
    self.cov[3:, 3:] = np.eye(n_views) * OFFSET_PX**2
    self.wander = np.r_[
        np.full(3, (WANDER_RADII * radius) ** 2),
        np.full(n_views, OFFSET_WANDER_PX**2),
    ]
    self._widths: tuple[float, ...] = WIDTHS

step 

step(images: dict[str, ndarray]) -> bool

Take one frame's gray_frames; whether its outline was measured.

A frame the narrow search finds no outline in is searched again wide: a ball that jerks moves farther in a frame than the narrow search reaches.

Source code in src/deeperfly/ball/outline.py 
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
def step(self, images: dict[str, np.ndarray]) -> bool:
    """Take one frame's `gray_frame`s; whether its outline was measured.

    A frame the narrow search finds no outline in is searched again wide: a ball
    that jerks moves farther in a frame than the narrow search reaches.
    """
    cov = self.cov + np.diag(self.wander)
    for widths in dict.fromkeys((self._widths, WIDTHS)):
        try:
            fit = fit_outline(
                self.views,
                images,
                self.center,
                self.radius,
                prior=(self.state, cov),
                widths=widths,
            )
        except ValueError:
            continue
        self.state = np.r_[fit.center, fit.offsets]
        self.cov, self._widths = fit.cov, WIDTHS[1:]
        return True
    self.cov = cov
    return False

OutlineFit dataclass

A sphere fitted to the outlines.

Attributes:

Name Type Description
center Float[ndarray, 3]

Its center, world coordinates.
radius float

Its radius, world units.
offsets Float[ndarray, V]

Each view's edges' offset from the outline, in pixels, in the views' order.
cov Float[ndarray, 'S S']

The covariance of the center and the offsets, (3 + V, 3 + V), the prior's included.
edges dict[str, int]

How many edges each view gave, by view name.
residual_px float

The median edge offset from the fitted outline, in pixels.
Source code in src/deeperfly/ball/outline.py 
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
@dataclass(frozen=True)
class OutlineFit:
    """A sphere fitted to the outlines.

    Attributes:
        center: Its center, world coordinates.
        radius: Its radius, world units.
        offsets: Each view's edges' offset from the outline, in pixels, in the
            views' order.
        cov: The covariance of the center and the offsets, `(3 + V, 3 + V)`, the
            prior's included.
        edges: How many edges each view gave, by view name.
        residual_px: The median edge offset from the fitted outline, in pixels.
    """

    center: Float[np.ndarray, "3"]
    radius: float
    offsets: Float[np.ndarray, "V"]
    cov: Float[np.ndarray, "S S"]
    edges: dict[str, int]
    residual_px: float

fit_outline 

fit_outline(
    views: Mapping[str, View],
    images: dict[str, ndarray],
    center: ndarray,
    radius: float,
    *,
    prior: tuple[ndarray, ndarray] | None = None,
    fit_radius: bool = False,
    widths: Sequence[float] = WIDTHS,
) -> OutlineFit

The sphere whose outline matches the ball's edges in images, by view name.

Starts from center and radius, which must put the predicted outline within the first of widths pixels of the edges. prior is the (mean, covariance) of the center and the views' offsets (in views' order) before this frame, which holds what the outlines barely pin; without one, the offsets are within OFFSET_PX of zero and the views must see the ball from more than one line. fit_radius frees the radius. images are gray_frames.

Raises:

Type Description
ValueError

If no view shows enough outline, or the views look along one line and there is no prior, or the radius is asked of them.
Source code in src/deeperfly/ball/outline.py 
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
def fit_outline(
    views: Mapping[str, View],
    images: dict[str, np.ndarray],
    center: np.ndarray,
    radius: float,
    *,
    prior: tuple[np.ndarray, np.ndarray] | None = None,
    fit_radius: bool = False,
    widths: Sequence[float] = WIDTHS,
) -> OutlineFit:
    """The sphere whose outline matches the ball's edges in `images`, by view name.

    Starts from `center` and `radius`, which must put the predicted outline within
    the first of `widths` pixels of the edges. `prior` is the `(mean, covariance)` of
    the center and the views' offsets (in `views`' order) before this frame, which
    holds what the outlines barely pin; without one, the offsets are within
    `OFFSET_PX` of zero and the views must see the ball from more than one line.
    `fit_radius` frees the radius. `images` are `gray_frame`s.

    Raises:
        ValueError: If no view shows enough outline, or the views look along one line
            and there is no prior, or the radius is asked of them.
    """
    names = list(views)
    n_views = len(names)
    x = np.r_[np.asarray(center, dtype=float), radius]
    free = 4 if fit_radius else 3
    # The unknowns: the center's step, the radius's, and each view's offset; the
    # prior is on the center and the offsets.
    held = [0, 1, 2, *range(free, free + n_views)]
    if prior is None:
        mean, info_held = np.zeros(3 + n_views), np.zeros((3 + n_views,) * 2)
        info_held[3:, 3:] = np.eye(n_views) / OFFSET_PX**2
    else:
        mean, info_held = np.asarray(prior[0], dtype=float), np.linalg.inv(prior[1])
    edges: dict[str, int] = {}
    solved = None
    for width in widths:
        rows, offsets, which, seen = [], [], [], []
        for k, name in enumerate(names):
            jac, s = _measure(views[name], images[name], x, width)
            edges[name] = len(s)
            if len(s) >= MIN_EDGES:
                rows.append(jac[:, :free])
                offsets.append(s)
                which.append(np.full(len(s), k))
                seen.append(views[name].position)
        if not rows:
            raise ValueError("no view shows enough of the ball's outline")
        if (prior is None or fit_radius) and _one_line(np.array(seen), x[:3]):
            raise ValueError(
                "the views that show the ball's outline look along one line"
            )
        jac = np.concatenate(rows)
        own = np.zeros((len(jac), n_views))
        own[np.arange(len(jac)), np.concatenate(which)] = 1.0
        info0 = np.zeros((free + n_views,) * 2)
        info0[np.ix_(held, held)] = info_held
        mean0 = np.zeros(free + n_views)
        mean0[held] = np.r_[mean[:3] - x[:3], mean[3:]]
        solved = _solve(np.hstack([jac, own]), np.concatenate(offsets), info0, mean0)
        x[:free] += solved[0][:free]
    if solved is None:
        raise ValueError("no search widths to fit the outline at")
    step, residual, info = solved
    cov = np.linalg.inv(info)[np.ix_(held, held)]
    return OutlineFit(
        center=x[:3],
        radius=float(x[3]),
        offsets=step[free:],
        cov=cov,
        edges=edges,
        residual_px=residual,
    )

gray_frame 

gray_frame(frame: ndarray) -> np.ndarray

A decoded (H, W, C) frame as the blurred float luma the edges are found on.

Source code in src/deeperfly/ball/outline.py 
105
106
107
108
def gray_frame(frame: np.ndarray) -> np.ndarray:
    """A decoded `(H, W, C)` frame as the blurred float luma the edges are found on."""
    gray = frame[..., 0] if frame.shape[-1] == 1 else frame.mean(axis=-1)
    return gaussian_filter(gray.astype(np.float32), SMOOTH_PX)

grazing_circle 

grazing_circle(
    view: View,
    center: ndarray,
    radius: float,
    n: int = N_POINTS,
) -> np.ndarray

(n, 3) world points of the circle where view's rays graze a sphere.

What view images as the sphere's outline, from a view outside the sphere.

Source code in src/deeperfly/ball/outline.py 
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
def grazing_circle(
    view: View, center: np.ndarray, radius: float, n: int = N_POINTS
) -> np.ndarray:
    """`(n, 3)` world points of the circle where `view`'s rays graze a sphere.

    What `view` images as the sphere's outline, from a view outside the sphere.
    """
    origin = view.position
    axis = np.asarray(center, dtype=float) - origin
    distance = float(np.linalg.norm(axis))
    axis = axis / distance
    helper = np.array([0.0, 0.0, 1.0]) if abs(axis[2]) < 0.9 else np.array([1.0, 0, 0])
    u = np.cross(axis, helper)
    u /= np.linalg.norm(u)
    v = np.cross(axis, u)
    sin_a = radius / distance
    cos_a = np.sqrt(1.0 - sin_a**2)
    phi = np.linspace(0.0, 2.0 * np.pi, n, endpoint=False)
    ring = np.cos(phi)[:, None] * u + np.sin(phi)[:, None] * v
    return origin + distance * cos_a * (cos_a * axis + sin_a * ring)

rim_points 

rim_points(
    view: View, center: ndarray, radius: float, n: int = 16
) -> np.ndarray

(n, 2) pixels of a sphere's outline in view, evenly around it.

Source code in src/deeperfly/ball/outline.py 
305
306
307
308
309
def rim_points(
    view: View, center: np.ndarray, radius: float, n: int = 16
) -> np.ndarray:
    """`(n, 2)` pixels of a sphere's outline in `view`, evenly around it."""
    return _outline(view, np.r_[np.asarray(center, dtype=float), radius], n)

track_outline 

track_outline(
    views: dict[str, View],
    frames: Iterable[dict[str, ndarray]],
    center: ndarray,
    cov: ndarray,
    radius: float,
) -> CenterTrack

The ball's center on each of frames (by view name, gray_frames), filtered.

Starts at center with covariance cov, the first frame with the widest search; the radius is held. CenterTrack.smoothed brings in the later frames.

Source code in src/deeperfly/ball/outline.py 
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
def track_outline(
    views: dict[str, View],
    frames: Iterable[dict[str, np.ndarray]],
    center: np.ndarray,
    cov: np.ndarray,
    radius: float,
) -> CenterTrack:
    """The ball's center on each of `frames` (by view name, `gray_frame`s), filtered.

    Starts at `center` with covariance `cov`, the first frame with the widest search;
    the radius is held. `CenterTrack.smoothed` brings in the later frames.
    """
    tracker = OutlineFilter(views, radius, center, cov)
    states, covs, seen = [], [], []
    for images in frames:
        seen.append(tracker.step(images))
        states.append(tracker.state)
        covs.append(tracker.cov)
    size = 3 + len(views)
    return CenterTrack(
        state=np.array(states).reshape(-1, size),
        cov=np.array(covs).reshape(-1, size, size),
        seen=np.array(seen, dtype=bool),
        wander=tracker.wander,
    )

Where the animal stands on the ball, and a first guess of the ball, from the 3D pose.

The ball itself is placed by its outline in every view that sees it (ball.outline); the pose starts it and orients the animal on it. A tethered animal's leg tips rest on the ball in stance, so the first guess is the sphere, centered on one view's ray through the ball's image center, that rests the most tips on its surface, its radius following from that distance and the circle's angular size. The tips cover only a small cap of the ball and one view's apparent size tells distance poorly, so the guess is a start, not the ball (it was 2.5% of a radius off along the view's ray on octacam).

The animal's axes on a ball: up is the ball's normal through the centroid of the points it stands on (the thorax-coxa points), forward the body axis (hind to front) on the tangent plane. When it is not believably standing on the ball, too few tips on the surface or the body not just above it, the world's up stands in for the normal and only the heading comes from the body. On the lab's octacam 73-84% of the tips rest on the ball in the opening frames, scattered about 2% of its radius about its surface.

Contact dataclass

The ball, and the animal's axes where it stands on it, in world coordinates.

Attributes:

Name Type Description
axes Float[ndarray, '3 3']

World to animal, (3, 3): its rows are the animal's forward, left and up axes in world coordinates.
center Float[ndarray, 3]

The ball's center, (3,).
radius float

The ball's radius, in world units.
on_surface float

The share of the tips that rest on the ball.
height float

The stance centroid above the surface, in ball radii.
level bool

Whether the world's up stands in for the ball's normal, the animal not believably standing on the ball.
Source code in src/deeperfly/ball/contact.py 
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
@dataclass(frozen=True)
class Contact:
    """The ball, and the animal's axes where it stands on it, in world coordinates.

    Attributes:
        axes: World to animal, `(3, 3)`: its rows are the animal's forward, left and
            up axes in world coordinates.
        center: The ball's center, `(3,)`.
        radius: The ball's radius, in world units.
        on_surface: The share of the tips that rest on the ball.
        height: The stance centroid above the surface, in ball radii.
        level: Whether the world's up stands in for the ball's normal, the animal not
            believably standing on the ball.
    """

    axes: Float[np.ndarray, "3 3"]
    center: Float[np.ndarray, "3"]
    radius: float
    on_surface: float
    height: float
    level: bool = False

    def line(self) -> str:
        """One log line: where the animal stands on the ball."""
        x, _, z = self.axes
        heading = np.degrees(np.arctan2(x[1], x[0]))
        rests = f"{100 * self.on_surface:.0f}% of the leg tips rest on it"
        if self.level:
            return (
                f"{rests} and the body is {self.height:.2f} radii above it, so the "
                f"world's up stands in for its normal; the body points {heading:+.1f} "
                "deg from the world's x"
            )
        tilt = np.degrees(np.arccos(np.clip(z[2], -1.0, 1.0)))
        toward = np.degrees(np.arctan2(z[1], z[0]))
        return (
            f"{rests}; the animal stands {tilt:.1f} deg from its top (toward "
            f"{toward:.0f} deg), {self.height:.2f} radii above it, and points "
            f"{heading:+.1f} deg from the world's x"
        )

line 

line() -> str

One log line: where the animal stands on the ball.

Source code in src/deeperfly/ball/contact.py 
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
def line(self) -> str:
    """One log line: where the animal stands on the ball."""
    x, _, z = self.axes
    heading = np.degrees(np.arctan2(x[1], x[0]))
    rests = f"{100 * self.on_surface:.0f}% of the leg tips rest on it"
    if self.level:
        return (
            f"{rests} and the body is {self.height:.2f} radii above it, so the "
            f"world's up stands in for its normal; the body points {heading:+.1f} "
            "deg from the world's x"
        )
    tilt = np.degrees(np.arccos(np.clip(z[2], -1.0, 1.0)))
    toward = np.degrees(np.arctan2(z[1], z[0]))
    return (
        f"{rests}; the animal stands {tilt:.1f} deg from its top (toward "
        f"{toward:.0f} deg), {self.height:.2f} radii above it, and points "
        f"{heading:+.1f} deg from the world's x"
    )

Stance dataclass

The 3D points the ball is guessed from and the animal oriented by.

Attributes:

Name Type Description
tips Float[ndarray, 'P 3']

The finite points that rest on the ball in stance, (P, 3).
body Float[ndarray, 3]

The centroid of the points the animal stands on, NaN if unknown.
body_axis Float[ndarray, 3]

Hind to front, NaN if unknown.
Source code in src/deeperfly/ball/contact.py 
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
@dataclass(frozen=True)
class Stance:
    """The 3D points the ball is guessed from and the animal oriented by.

    Attributes:
        tips: The finite points that rest on the ball in stance, `(P, 3)`.
        body: The centroid of the points the animal stands on, NaN if unknown.
        body_axis: Hind to front, NaN if unknown.
    """

    tips: Float[np.ndarray, "P 3"]
    body: Float[np.ndarray, "3"]
    body_axis: Float[np.ndarray, "3"]

    @classmethod
    def of(
        cls,
        tips: Float[np.ndarray, "T P 3"],
        stand: Float[np.ndarray, "T Q 3"],
        front: Float[np.ndarray, "T F 3"],
        hind: Float[np.ndarray, "T H 3"],
    ) -> "Stance":
        """The stance over some frames of world points, NaN where unknown.

        Each point's median over the frames, then the centroids.
        """
        with warnings.catch_warnings():  # a point never seen is NaN, and stays NaN
            warnings.simplefilter("ignore", RuntimeWarning)
            front_c = np.nanmedian(np.nanmean(front, axis=1), axis=0)
            hind_c = np.nanmedian(np.nanmean(hind, axis=1), axis=0)
            body = np.nanmean(np.nanmedian(stand, axis=0), axis=0)
        tips = tips.reshape(-1, 3)
        return cls(tips[np.isfinite(tips).all(axis=1)], body, front_c - hind_c)

of classmethod 

of(
    tips: Float[ndarray, "T P 3"],
    stand: Float[ndarray, "T Q 3"],
    front: Float[ndarray, "T F 3"],
    hind: Float[ndarray, "T H 3"],
) -> Stance

The stance over some frames of world points, NaN where unknown.

Each point's median over the frames, then the centroids.

Source code in src/deeperfly/ball/contact.py 
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
@classmethod
def of(
    cls,
    tips: Float[np.ndarray, "T P 3"],
    stand: Float[np.ndarray, "T Q 3"],
    front: Float[np.ndarray, "T F 3"],
    hind: Float[np.ndarray, "T H 3"],
) -> "Stance":
    """The stance over some frames of world points, NaN where unknown.

    Each point's median over the frames, then the centroids.
    """
    with warnings.catch_warnings():  # a point never seen is NaN, and stays NaN
        warnings.simplefilter("ignore", RuntimeWarning)
        front_c = np.nanmedian(np.nanmean(front, axis=1), axis=0)
        hind_c = np.nanmedian(np.nanmean(hind, axis=1), axis=0)
        body = np.nanmean(np.nanmedian(stand, axis=0), axis=0)
    tips = tips.reshape(-1, 3)
    return cls(tips[np.isfinite(tips).all(axis=1)], body, front_c - hind_c)

ball_ray 

ball_ray(
    view: View, circle: tuple[float, float, float]
) -> tuple[np.ndarray, np.ndarray, float]

The ray through the ball's image center, and the sine of its angular radius.

The ray is (origin, unit direction) in the world; the angular radius is the mean angle between it and the rays of four points of the circle's rim.

Source code in src/deeperfly/ball/contact.py 
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
def ball_ray(
    view: View, circle: tuple[float, float, float]
) -> tuple[np.ndarray, np.ndarray, float]:
    """The ray through the ball's image center, and the sine of its angular radius.

    The ray is `(origin, unit direction)` in the world; the angular radius is the mean
    angle between it and the rays of four points of the circle's rim.
    """
    cx, cy, r = circle
    origin, ray = view.backproject_ray(np.array([cx, cy]))
    ray = ray / np.linalg.norm(ray)
    angles = []
    for dx, dy in ((r, 0.0), (-r, 0.0), (0.0, r), (0.0, -r)):
        rim = view.backproject_ray(np.array([cx + dx, cy + dy]))[1]
        angles.append(np.arccos(np.clip(rim @ ray / np.linalg.norm(rim), -1.0, 1.0)))
    return origin, ray, float(np.sin(np.mean(angles)))

guess_ball 

guess_ball(
    view: View,
    circle: tuple[float, float, float],
    stance: Stance,
) -> tuple[np.ndarray, float] | None

A first (center, radius) of the ball, from one view and the leg tips.

circle is the ball's image (cx, cy, r) in view's pixels. None with too few tips, or no centroid to start the search from.

Source code in src/deeperfly/ball/contact.py 
119
120
121
122
123
124
125
126
127
128
129
130
131
132
def guess_ball(
    view: View, circle: tuple[float, float, float], stance: Stance
) -> tuple[np.ndarray, float] | None:
    """A first `(center, radius)` of the ball, from one view and the leg tips.

    `circle` is the ball's image `(cx, cy, r)` in `view`'s pixels. None with too few
    tips, or no centroid to start the search from.
    """
    if len(stance.tips) < MIN_TIPS or not np.isfinite(stance.body).all():
        return None
    origin, ray, sin_a = ball_ray(view, circle)
    start = float((stance.body - origin) @ ray)
    depth = _ball_depth(stance.tips, origin, ray, sin_a, start)
    return origin + depth * ray, depth * sin_a

stand_on 

stand_on(
    center: ndarray, radius: float, stance: Stance
) -> tuple[Contact, str | None]

The animal's axes on the ball, and why the world's up stands in, if it does.

Source code in src/deeperfly/ball/contact.py 
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
def stand_on(
    center: np.ndarray, radius: float, stance: Stance
) -> tuple[Contact, str | None]:
    """The animal's axes on the ball, and why the world's up stands in, if it does."""
    gap = np.linalg.norm(stance.tips - center, axis=1) - radius
    share = float(np.mean(np.abs(gap) < ON_SURFACE * radius)) if len(gap) else 0.0
    normal = stance.body - center
    height = float(np.linalg.norm(normal) / radius - 1.0)
    level = not (share >= MIN_ON_SURFACE and 0.0 < height < MAX_HEIGHT)
    up = UP if level else normal / np.linalg.norm(normal)
    contact = Contact(
        axes=_axes(up, stance.body_axis),
        center=np.asarray(center, dtype=float),
        radius=float(radius),
        on_surface=share,
        height=height,
        level=level,
    )
    why = None
    if level:
        why = (
            f"{100 * share:.0f}% of the leg tips rest on the ball, and the body is "
            f"{height:.2f} radii above it"
        )
    return contact, why

spintrack on one view's frames, and the rotation it measures in the world frame.

spintrack's camera frame is OpenCV's, as deeperfly's is, so a rotation vector it measures reaches the world as R.T @ v, with R the view's world-to-camera rotation. That holds whatever animal frame spintrack was handed: the animal frame only orients its forward, side and turn, kept verbatim in tracks. spintrack's pixel centers sit at half-integers and deeperfly's at integers (ADR-0055), so its pixels are moved by -0.5.

spintrack models a pinhole with its principal point at the frame's center and no distortion; it is handed the view's vertical field of view and nothing else of the lens. Its rotations are measured on the ball's own image, so the difference moves the camera's axes by the principal point's offset in angle, a fraction of a degree on a well-centered lens.

detect_circle 

detect_circle(
    video: Video, start: int, stop: int
) -> tuple[float, float, float]

The ball's image circle (cx, cy, r) in deeperfly's pixels, from start on.

It is detected in the first frames of start:stop.

Raises:

Type Description
RuntimeError

spintrack's DetectionError, when the ball cannot be trusted.
Source code in src/deeperfly/ball/track.py 
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
def detect_circle(video: Video, start: int, stop: int) -> tuple[float, float, float]:
    """The ball's image circle `(cx, cy, r)` in deeperfly's pixels, from `start` on.

    It is detected in the first frames of `start:stop`.

    Raises:
        RuntimeError: spintrack's `DetectionError`, when the ball cannot be trusted.
    """
    from spintrack.autofit import find_ball

    stop = min(stop, start + SAMPLE_SPAN)
    step = max(1, SAMPLE_SPAN // SAMPLE_COUNT)
    with video.open() as reader:
        frames = [
            _gray(f)
            for f in reader.stream_frames(
                start=start, stop=stop, step=step, gray_ok=True
            )
        ]
    found = find_ball(frames, len(frames))
    return found.cx - HALF_PIXEL, found.cy - HALF_PIXEL, found.r

track_ball 

track_ball(
    view: View,
    video: Video,
    start: int,
    stop: int,
    axes: ndarray,
    rim: ndarray,
    *,
    tracking: Mapping[str, Any],
    two_pass: bool = False,
) -> tuple[dict[str, np.ndarray], list[str], Any]

Track the ball's rotation over frames start:stop of video, which view sees.

rim is points of the ball's outline at start, (n, 2) in deeperfly's pixels; axes (world to animal) orients spintrack's animal frame. Returns rotation, step and tracks over the stretch (NaN at a frame spintrack dropped), the names of tracks' columns, and spintrack's quality summary.

Source code in src/deeperfly/ball/track.py 
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
def track_ball(
    view: View,
    video: Video,
    start: int,
    stop: int,
    axes: np.ndarray,
    rim: np.ndarray,
    *,
    tracking: Mapping[str, Any],
    two_pass: bool = False,
) -> tuple[dict[str, np.ndarray], list[str], Any]:
    """Track the ball's rotation over frames `start:stop` of `video`, which `view` sees.

    `rim` is points of the ball's outline at `start`, `(n, 2)` in deeperfly's pixels;
    `axes` (world to animal) orients spintrack's animal frame. Returns `rotation`,
    `step` and `tracks` over the stretch (NaN at a frame spintrack dropped), the names
    of `tracks`' columns, and spintrack's quality summary.
    """
    spintrack = require_spintrack()
    height, width = video.size
    fps = video.fps or 0.0
    cfg = spintrack.Config.model_validate({"tracking": dict(tracking)})
    fy = float(view.kmat[1, 1])
    cfg.camera.vfov_deg = float(np.degrees(2.0 * np.arctan(height / (2.0 * fy))))
    to_animal = axes @ view.rmat.T  # camera to animal
    cfg.camera.rotation = tuple(float(v) for v in np.asarray(rmat_to_rvec(to_animal)))
    if fps > 0:
        cfg.camera.fps = fps
    cfg.ball.rim = [(float(x), float(y)) for x, y in np.asarray(rim) + HALF_PIXEL]

    def frames():
        return _Frames(video, start, stop, fps, (width, height))

    result = spintrack.track(cfg, frames=frames, two_pass=two_pass)
    columns = list(result.columns)
    records = np.asarray(result.records, dtype=np.float64)
    n = stop - start
    rows = records[:, columns.index("frame")].astype(np.int64)
    keep = (rows >= 0) & (rows < n)
    rows, records = rows[keep], records[keep]

    def per_frame(names: list[str]) -> np.ndarray:
        out = np.full((n, len(names)), np.nan)
        out[rows] = records[:, [columns.index(c) for c in names]]
        return out

    to_world = view.rmat  # a camera row vector times R is the world one
    rotation = per_frame(["r_cam_x", "r_cam_y", "r_cam_z"]) @ to_world
    step = per_frame(["dr_cam_x", "dr_cam_y", "dr_cam_z"]) @ to_world
    names = [c for c in columns if c != "frame"]
    arrays = {"rotation": rotation, "step": step, "tracks": per_frame(names)}
    return arrays, names, result.quality

Access to the optional spintrack tracker, behind one explicit gate.

spintrack runs SAM 3 through torch and transformers and builds a Rust extension, so the deeperfly[ball] extra holds it and a plain install stays without it (ADR-0144). Everything that tracks calls require_spintrack inside a function body, never at module scope; the contact fit imports nothing from here, so it runs without the extra.

MissingSpintrack

Bases: ImportError

Raised when the ball is asked for without the ball extra installed.

Source code in src/deeperfly/ball/_spintrack.py 
21
22
class MissingSpintrack(ImportError):
    """Raised when the ball is asked for without the `ball` extra installed."""

require_spintrack cached 

require_spintrack() -> ModuleType

The spintrack module, or MissingSpintrack saying how to install it.

Source code in src/deeperfly/ball/_spintrack.py 
25
26
27
28
29
30
31
32
33
34
35
36
37
@functools.cache
def require_spintrack() -> ModuleType:
    """The `spintrack` module, or [MissingSpintrack][] saying how to install it."""
    try:
        import spintrack
    except ImportError as exc:  # pragma: no cover -- exercised by the install-less path
        raise MissingSpintrack(
            "tracking the ball needs the optional spintrack tracker, which is not "
            f"installed. Install the extra with:\n    {INSTALL_HINT}\n"
            "It builds a Rust extension, so a Rust toolchain is required "
            "(https://rustup.rs)."
        ) from exc
    return spintrack