rig

The rig: an ordered set of views, and what each one is.

A View is a virtual camera: extrinsics, intrinsics, distortion and whether it is mirrored, with pixels in the raw frame of the video it reads and no frame size (ADR-0056). A Rig orders them, the V axis of every array a run writes. How a view is written, in a run config's [calibration.views] or a calibration file, is a ViewSpec: an orbit placement or the raw parameters a solve works in, resolved against the frame size of its video (ADR-0060).

ViewSpec module-attribute

ViewSpec = spec_union(OrbitView, RawView)

How a view is written: an OrbitView or a RawView, told apart by the keys present.

OrbitView

Bases: _Placement

A view on a ring around look_at: the human way to write a starting guess.

Placed as resolve_extrinsics says, with one guessed focal in pixels (fx == fy) and a principal point defaulting to the center of the frame it reads.

Attributes:

Name	Type	Description
distance	float	Distance from look_at to the camera center, in world units.
focal	float	Focal length in pixels, for both fx and fy.
azimuth_deg	float	Orbit rotation in the world xy-plane, in degrees.
elevation_deg	float	Orbit lift toward +z, in degrees.
roll_deg	float	Rotation about the optical axis, in degrees.
look_at	tuple[float, float, float]	The world point the view aims at.
cx	float | None	Principal-point x in pixels; omitted, (width - 1) / 2.
cy	float | None	Principal-point y in pixels; omitted, (height - 1) / 2.

Source code in src/deeperfly/rig.py

class OrbitView(_Placement):
    """A view on a ring around `look_at`: the human way to write a starting guess.

    Placed as [resolve_extrinsics][deeperfly.rig.resolve_extrinsics] says, with one
    guessed `focal` in pixels (`fx == fy`) and a principal point defaulting to the
    center of the frame it reads.

    Attributes:
        distance: Distance from `look_at` to the camera center, in world units.
        focal: Focal length in pixels, for both `fx` and `fy`.
        azimuth_deg: Orbit rotation in the world xy-plane, in degrees.
        elevation_deg: Orbit lift toward `+z`, in degrees.
        roll_deg: Rotation about the optical axis, in degrees.
        look_at: The world point the view aims at.
        cx: Principal-point x in pixels; omitted, `(width - 1) / 2`.
        cy: Principal-point y in pixels; omitted, `(height - 1) / 2`.
    """

    distance: float
    focal: float
    azimuth_deg: float = 0.0
    elevation_deg: float = 0.0
    roll_deg: float = 0.0
    look_at: tuple[float, float, float] = (0.0, 0.0, 0.0)
    cx: float | None = None
    cy: float | None = None

    def resolve(self, frame_hw: tuple[int, int] | None = None) -> View:
        """This placement as a [View][deeperfly.rig.View], `frame_hw` its video's frame.

        Raises:
            ValueError: If the principal point defaults with no frame size given, or
                the view looks straight along the world up axis.
        """
        if (self.cx is None or self.cy is None) and frame_hw is None:
            raise ValueError(
                "the principal point defaults to the image center, but no image "
                "size is known; open the video, or set cx and cy"
            )
        hw = cast("tuple[int, int]", frame_hw)
        cx = self.cx if self.cx is not None else (hw[1] - 1) / 2
        cy = self.cy if self.cy is not None else (hw[0] - 1) / 2
        rvec, tvec = resolve_extrinsics(
            self.distance,
            look_at=self.look_at,
            azimuth_deg=self.azimuth_deg,
            elevation_deg=self.elevation_deg,
            roll_deg=self.roll_deg,
        )
        intr = np.array([self.focal, self.focal, cx, cy], dtype=float)
        dist = np.asarray(self.dist, dtype=float)
        return View(rvec, tvec, intr, dist, mirrored=self.mirrored)

resolve

resolve(frame_hw: tuple[int, int] | None = None) -> View

This placement as a View, frame_hw its video's frame.

Raises:

Type	Description
ValueError	If the principal point defaults with no frame size given, or the view looks straight along the world up axis.

Source code in src/deeperfly/rig.py

def resolve(self, frame_hw: tuple[int, int] | None = None) -> View:
    """This placement as a [View][deeperfly.rig.View], `frame_hw` its video's frame.

    Raises:
        ValueError: If the principal point defaults with no frame size given, or
            the view looks straight along the world up axis.
    """
    if (self.cx is None or self.cy is None) and frame_hw is None:
        raise ValueError(
            "the principal point defaults to the image center, but no image "
            "size is known; open the video, or set cx and cy"
        )
    hw = cast("tuple[int, int]", frame_hw)
    cx = self.cx if self.cx is not None else (hw[1] - 1) / 2
    cy = self.cy if self.cy is not None else (hw[0] - 1) / 2
    rvec, tvec = resolve_extrinsics(
        self.distance,
        look_at=self.look_at,
        azimuth_deg=self.azimuth_deg,
        elevation_deg=self.elevation_deg,
        roll_deg=self.roll_deg,
    )
    intr = np.array([self.focal, self.focal, cx, cy], dtype=float)
    dist = np.asarray(self.dist, dtype=float)
    return View(rvec, tvec, intr, dist, mirrored=self.mirrored)

RawView

Bases: _Placement

A view by the parameters a solve works in: what a solved rig is stored as.

Attributes:

Name	Type	Description
rvec	tuple[float, float, float]	Axis-angle rotation vector, world to camera.
tvec	tuple[float, float, float]	Translation, world to camera.
intr	tuple[float, float, float] | tuple[float, float, float, float]	Packed intrinsics, [fx, fy, cx, cy] or [f, cx, cy].
image_size	tuple[int, int] | None	The (height, width) of the frame the view was solved in, where known. Its intrinsics are pixels of that frame (ADR-0055), so a video of another size is refused rather than read through them (ADR-0079).

Source code in src/deeperfly/rig.py

class RawView(_Placement):
    """A view by the parameters a solve works in: what a solved rig is stored as.

    Attributes:
        rvec: Axis-angle rotation vector, world to camera.
        tvec: Translation, world to camera.
        intr: Packed intrinsics, `[fx, fy, cx, cy]` or `[f, cx, cy]`.
        image_size: The `(height, width)` of the frame the view was solved in, where
            known. Its intrinsics are pixels of that frame (ADR-0055), so a video of
            another size is refused rather than read through them (ADR-0079).
    """

    rvec: tuple[float, float, float]
    tvec: tuple[float, float, float]
    intr: tuple[float, float, float] | tuple[float, float, float, float]
    image_size: tuple[int, int] | None = None

    @field_validator("intr", mode="before")
    @classmethod
    def _packed(cls, value):
        """Intrinsics are packed one of two ways (see `deeperfly.geometry`)."""
        if isinstance(value, (list, tuple)) and len(value) not in (3, 4):
            raise ValueError(
                f"[fx, fy, cx, cy] or [f, cx, cy], not {len(value)} entries"
            )
        return value

    @classmethod
    def of(cls, view: View, *, image_size: tuple[int, int] | None = None) -> RawView:
        """The spec that resolves back to `view` (trailing zero distortion dropped)."""
        dist = np.asarray(view.dist, dtype=float)
        nonzero = np.flatnonzero(dist)
        keep = int(nonzero[-1]) + 1 if nonzero.size else 0
        rx, ry, rz = map(float, view.rvec)
        tx, ty, tz = map(float, view.tvec)
        return cls(
            mirrored=bool(view.mirrored),
            dist=tuple(float(x) for x in dist[:keep]),
            rvec=(rx, ry, rz),
            tvec=(tx, ty, tz),
            intr=tuple(float(x) for x in view.intr),
            image_size=None
            if image_size is None
            else (int(image_size[0]), int(image_size[1])),
        )

    def resolve(self, frame_hw: tuple[int, int] | None = None) -> View:
        """This placement as a [View][deeperfly.rig.View], `frame_hw` its video's frame.

        Raises:
            ValueError: If the view was solved in a frame of another size.
        """
        if (
            self.image_size is not None
            and frame_hw is not None
            and tuple(frame_hw) != tuple(self.image_size)
        ):
            (h, w), (fh, fw) = self.image_size, frame_hw
            raise ValueError(
                f"solved in a {w}x{h} frame, but its video is {fw}x{fh}; intrinsics "
                "in pixels do not survive a resize (ADR-0079)"
            )
        return View(
            np.asarray(self.rvec, dtype=float),
            np.asarray(self.tvec, dtype=float),
            np.asarray(self.intr, dtype=float),
            np.asarray(self.dist, dtype=float),
            mirrored=self.mirrored,
        )

of classmethod

of(
    view: View, *, image_size: tuple[int, int] | None = None
) -> RawView

The spec that resolves back to view (trailing zero distortion dropped).

Source code in src/deeperfly/rig.py

@classmethod
def of(cls, view: View, *, image_size: tuple[int, int] | None = None) -> RawView:
    """The spec that resolves back to `view` (trailing zero distortion dropped)."""
    dist = np.asarray(view.dist, dtype=float)
    nonzero = np.flatnonzero(dist)
    keep = int(nonzero[-1]) + 1 if nonzero.size else 0
    rx, ry, rz = map(float, view.rvec)
    tx, ty, tz = map(float, view.tvec)
    return cls(
        mirrored=bool(view.mirrored),
        dist=tuple(float(x) for x in dist[:keep]),
        rvec=(rx, ry, rz),
        tvec=(tx, ty, tz),
        intr=tuple(float(x) for x in view.intr),
        image_size=None
        if image_size is None
        else (int(image_size[0]), int(image_size[1])),
    )

resolve

resolve(frame_hw: tuple[int, int] | None = None) -> View

This placement as a View, frame_hw its video's frame.

Raises:

Type	Description
ValueError	If the view was solved in a frame of another size.

Source code in src/deeperfly/rig.py

def resolve(self, frame_hw: tuple[int, int] | None = None) -> View:
    """This placement as a [View][deeperfly.rig.View], `frame_hw` its video's frame.

    Raises:
        ValueError: If the view was solved in a frame of another size.
    """
    if (
        self.image_size is not None
        and frame_hw is not None
        and tuple(frame_hw) != tuple(self.image_size)
    ):
        (h, w), (fh, fw) = self.image_size, frame_hw
        raise ValueError(
            f"solved in a {w}x{h} frame, but its video is {fw}x{fh}; intrinsics "
            "in pixels do not survive a resize (ADR-0079)"
        )
    return View(
        np.asarray(self.rvec, dtype=float),
        np.asarray(self.tvec, dtype=float),
        np.asarray(self.intr, dtype=float),
        np.asarray(self.dist, dtype=float),
        mirrored=self.mirrored,
    )

Rig

An ordered collection of named View objects.

The order is the V axis of every stacked array, positional: observations paired with a rig must follow it. How many videos the rig reads is a separate number.

Source code in src/deeperfly/rig.py

class Rig:
    """An ordered collection of named [View][deeperfly.rig.View] objects.

    The order is the `V` axis of every stacked array, positional: observations paired
    with a rig must follow it. How many videos the rig reads is a separate number.
    """

    def __init__(self, views: dict[str, View]):
        """Wrap an ordered `name -> View` mapping."""
        self.views = dict(views)

    def __repr__(self) -> str:
        """One line: how many views, and the first few names in order."""
        names = self.names
        shown = ", ".join(names[:6]) + (", ..." if len(names) > 6 else "")
        plural = "view" if len(names) == 1 else "views"
        return f"Rig({len(names)} {plural}: {shown})"

    @property
    def names(self) -> list[str]:
        """View names, in order."""
        return list(self.views)

    @classmethod
    def from_arrays(
        cls,
        names: list[str],
        rvecs: Float[np.ndarray, "V 3"],
        tvecs: Float[np.ndarray, "V 3"],
        intrs: Float[np.ndarray, "V P"],
        dists: Float[np.ndarray, "V K"] | Sequence[Sequence[float]],
        mirrored: Bool[np.ndarray, "V"] | Sequence[bool] | None = None,
    ) -> Rig:
        """Build a rig from stacked per-view arrays (e.g. BA output).

        `dists` may be one sequence per view of its own length. `mirrored` defaults to
        none; a caller carrying a rig through a solve passes its own back, since
        bundle adjustment never touches it.

        Raises:
            ValueError: If an array's leading length disagrees with `names`, or the
                names are not unique.
        """
        rvecs, tvecs, intrs = map(np.asarray, (rvecs, tvecs, intrs))
        per_view = [np.asarray(d, dtype=float) for d in dists]
        n = len(names)
        for label, arr in (
            ("rvecs", rvecs),
            ("tvecs", tvecs),
            ("intrs", intrs),
            ("dists", per_view),
        ):
            if len(arr) != n:
                raise ValueError(f"{label} has {len(arr)} rows but there are {n} names")
        if len(set(names)) != n:
            dupes = sorted({m for m in names if names.count(m) > 1})
            raise ValueError(f"duplicate view names {dupes}")
        if mirrored is not None and len(mirrored) != n:
            raise ValueError(
                f"mirrored has {len(mirrored)} entries but there are {n} names"
            )
        flips = list(mirrored) if mirrored is not None else [False] * n
        return cls(
            {
                name: View(
                    rvecs[i],
                    tvecs[i],
                    intrs[i],
                    per_view[i],
                    mirrored=bool(flips[i]),
                )
                for i, name in enumerate(names)
            }
        )

    @property
    def rvecs(self) -> Float[np.ndarray, "V 3"]:
        """Stacked rotation vectors of shape `(V, 3)`."""
        return np.stack([v.rvec for v in self.views.values()])

    @property
    def tvecs(self) -> Float[np.ndarray, "V 3"]:
        """Stacked translation vectors of shape `(V, 3)`."""
        return np.stack([v.tvec for v in self.views.values()])

    @property
    def intrs(self) -> Float[np.ndarray, "V P"]:
        """Stacked packed intrinsics of shape `(V, P)`."""
        return np.stack([v.intr for v in self.views.values()])

    @property
    def dists(self) -> Float[np.ndarray, "V K"]:
        """Per-view distortion, zero-padded to the rig-wide max length."""
        k = max((v.dist.size for v in self.views.values()), default=0)
        out = np.zeros((len(self.views), k))
        for i, view in enumerate(self.views.values()):
            out[i, : view.dist.size] = view.dist
        return out

    @property
    def mirrored(self) -> Bool[np.ndarray, "V"]:
        """Per-view improper-pose flags of shape `(V,)`, in order."""
        return np.array([v.mirrored for v in self.views.values()], dtype=bool)

    @classmethod
    def from_specs(
        cls,
        specs: Mapping[str, ViewSpec],
        frame_hw: Mapping[str, tuple[int, int]] | None = None,
    ) -> Rig:
        """Resolve a `name -> ViewSpec` mapping to a rig, in mapping order.

        `frame_hw` is each view's `(height, width)`, which an orbit view's principal
        point defaults to the center of; a raw view, or an orbit view with `cx` and
        `cy`, needs none.

        Raises:
            ValueError: If an orbit view's principal point defaults with no frame
                size given, naming the view.
        """
        sizes = frame_hw or {}
        views = {}
        for name, spec in specs.items():
            try:
                views[name] = spec.resolve(sizes.get(name))
            except ValueError as exc:
                raise ValueError(f"[views.{name}]: {exc}") from exc
        return cls(views)

    def specs(
        self, frame_hw: Mapping[str, tuple[int, int]] | None = None
    ) -> dict[str, RawView]:
        """Each view as a [RawView][], the form a solved rig is stored in.

        `frame_hw` is each view's `(height, width)`, recorded as the frame it was
        solved in so a video of another size is refused where it resolves (ADR-0079).
        """
        sizes = frame_hw or {}
        return {
            name: RawView.of(view, image_size=sizes.get(name))
            for name, view in self.views.items()
        }

    def project(
        self, pts3d: Float[np.ndarray, "*pts 3"]
    ) -> Float[np.ndarray, "V *pts 2"]:
        """Project `(*pts, 3)` world points through every view: `(V, *pts, 2)`."""
        out = project_full(
            np.asarray(pts3d),
            self.rvecs,
            self.tvecs,
            self.intrs,
            self.dists,
            self.mirrored,
        )
        return np.asarray(out)

dists property

dists: Float[ndarray, 'V K']

Per-view distortion, zero-padded to the rig-wide max length.

intrs property

intrs: Float[ndarray, 'V P']

Stacked packed intrinsics of shape (V, P).

mirrored property

mirrored: Bool[ndarray, V]

Per-view improper-pose flags of shape (V,), in order.

names property

names: list[str]

View names, in order.

rvecs property

rvecs: Float[ndarray, 'V 3']

Stacked rotation vectors of shape (V, 3).

tvecs property

tvecs: Float[ndarray, 'V 3']

Stacked translation vectors of shape (V, 3).

__init__

__init__(views: dict[str, View])

Wrap an ordered name -> View mapping.

Source code in src/deeperfly/rig.py

def __init__(self, views: dict[str, View]):
    """Wrap an ordered `name -> View` mapping."""
    self.views = dict(views)

__repr__

__repr__() -> str

One line: how many views, and the first few names in order.

Source code in src/deeperfly/rig.py

def __repr__(self) -> str:
    """One line: how many views, and the first few names in order."""
    names = self.names
    shown = ", ".join(names[:6]) + (", ..." if len(names) > 6 else "")
    plural = "view" if len(names) == 1 else "views"
    return f"Rig({len(names)} {plural}: {shown})"

from_arrays classmethod

from_arrays(
    names: list[str],
    rvecs: Float[ndarray, "V 3"],
    tvecs: Float[ndarray, "V 3"],
    intrs: Float[ndarray, "V P"],
    dists: Float[ndarray, "V K"]
    | Sequence[Sequence[float]],
    mirrored: Bool[ndarray, V]
    | Sequence[bool]
    | None = None,
) -> Rig

Build a rig from stacked per-view arrays (e.g. BA output).

dists may be one sequence per view of its own length. mirrored defaults to none; a caller carrying a rig through a solve passes its own back, since bundle adjustment never touches it.

Raises:

Type	Description
ValueError	If an array's leading length disagrees with names, or the names are not unique.

Source code in src/deeperfly/rig.py

@classmethod
def from_arrays(
    cls,
    names: list[str],
    rvecs: Float[np.ndarray, "V 3"],
    tvecs: Float[np.ndarray, "V 3"],
    intrs: Float[np.ndarray, "V P"],
    dists: Float[np.ndarray, "V K"] | Sequence[Sequence[float]],
    mirrored: Bool[np.ndarray, "V"] | Sequence[bool] | None = None,
) -> Rig:
    """Build a rig from stacked per-view arrays (e.g. BA output).

    `dists` may be one sequence per view of its own length. `mirrored` defaults to
    none; a caller carrying a rig through a solve passes its own back, since
    bundle adjustment never touches it.

    Raises:
        ValueError: If an array's leading length disagrees with `names`, or the
            names are not unique.
    """
    rvecs, tvecs, intrs = map(np.asarray, (rvecs, tvecs, intrs))
    per_view = [np.asarray(d, dtype=float) for d in dists]
    n = len(names)
    for label, arr in (
        ("rvecs", rvecs),
        ("tvecs", tvecs),
        ("intrs", intrs),
        ("dists", per_view),
    ):
        if len(arr) != n:
            raise ValueError(f"{label} has {len(arr)} rows but there are {n} names")
    if len(set(names)) != n:
        dupes = sorted({m for m in names if names.count(m) > 1})
        raise ValueError(f"duplicate view names {dupes}")
    if mirrored is not None and len(mirrored) != n:
        raise ValueError(
            f"mirrored has {len(mirrored)} entries but there are {n} names"
        )
    flips = list(mirrored) if mirrored is not None else [False] * n
    return cls(
        {
            name: View(
                rvecs[i],
                tvecs[i],
                intrs[i],
                per_view[i],
                mirrored=bool(flips[i]),
            )
            for i, name in enumerate(names)
        }
    )

from_specs classmethod

from_specs(
    specs: Mapping[str, ViewSpec],
    frame_hw: Mapping[str, tuple[int, int]] | None = None,
) -> Rig

Resolve a name -> ViewSpec mapping to a rig, in mapping order.

frame_hw is each view's (height, width), which an orbit view's principal point defaults to the center of; a raw view, or an orbit view with cx and cy, needs none.

Raises:

Type	Description
ValueError	If an orbit view's principal point defaults with no frame size given, naming the view.

Source code in src/deeperfly/rig.py

@classmethod
def from_specs(
    cls,
    specs: Mapping[str, ViewSpec],
    frame_hw: Mapping[str, tuple[int, int]] | None = None,
) -> Rig:
    """Resolve a `name -> ViewSpec` mapping to a rig, in mapping order.

    `frame_hw` is each view's `(height, width)`, which an orbit view's principal
    point defaults to the center of; a raw view, or an orbit view with `cx` and
    `cy`, needs none.

    Raises:
        ValueError: If an orbit view's principal point defaults with no frame
            size given, naming the view.
    """
    sizes = frame_hw or {}
    views = {}
    for name, spec in specs.items():
        try:
            views[name] = spec.resolve(sizes.get(name))
        except ValueError as exc:
            raise ValueError(f"[views.{name}]: {exc}") from exc
    return cls(views)

project

project(
    pts3d: Float[ndarray, "*pts 3"],
) -> Float[np.ndarray, "V *pts 2"]

Project (*pts, 3) world points through every view: (V, *pts, 2).

Source code in src/deeperfly/rig.py

def project(
    self, pts3d: Float[np.ndarray, "*pts 3"]
) -> Float[np.ndarray, "V *pts 2"]:
    """Project `(*pts, 3)` world points through every view: `(V, *pts, 2)`."""
    out = project_full(
        np.asarray(pts3d),
        self.rvecs,
        self.tvecs,
        self.intrs,
        self.dists,
        self.mirrored,
    )
    return np.asarray(out)

specs

specs(
    frame_hw: Mapping[str, tuple[int, int]] | None = None,
) -> dict[str, RawView]

Each view as a RawView, the form a solved rig is stored in.

frame_hw is each view's (height, width), recorded as the frame it was solved in so a video of another size is refused where it resolves (ADR-0079).

Source code in src/deeperfly/rig.py

def specs(
    self, frame_hw: Mapping[str, tuple[int, int]] | None = None
) -> dict[str, RawView]:
    """Each view as a [RawView][], the form a solved rig is stored in.

    `frame_hw` is each view's `(height, width)`, recorded as the frame it was
    solved in so a video of another size is refused where it resolves (ADR-0079).
    """
    sizes = frame_hw or {}
    return {
        name: RawView.of(view, image_size=sizes.get(name))
        for name, view in self.views.items()
    }

View dataclass

One virtual camera: extrinsics, intrinsics, distortion, and its handedness.

Every pixel coordinate is in the raw, whole, full-resolution frame of the video the view reads, so a crop is undone before its coordinates reach a view. A view has no name of its own; the rig that holds it keys it.

Attributes:

Name	Type	Description
rvec	Float[ndarray, 3]	Axis-angle rotation vector of shape (3,).
tvec	Float[ndarray, 3]	Translation vector of shape (3,).
intr	Float[ndarray, P]	Packed intrinsics, [fx, fy, cx, cy] or [f, cx, cy] (intr_to_kmat).
dist	Float[ndarray, K]	OpenCV-ordered distortion coefficients (may be empty).
mirrored	bool	Whether the view sees the world reflected, through a prism: its pose is improper (ADR-0053) and nothing flips the pixels.

Source code in src/deeperfly/rig.py

@dataclass(frozen=True, eq=False)
class View:
    """One virtual camera: extrinsics, intrinsics, distortion, and its handedness.

    Every pixel coordinate is in the raw, whole, full-resolution frame of the video the
    view reads, so a crop is undone before its coordinates reach a view. A view has no
    name of its own; the rig that holds it keys it.

    Attributes:
        rvec: Axis-angle rotation vector of shape `(3,)`.
        tvec: Translation vector of shape `(3,)`.
        intr: Packed intrinsics, `[fx, fy, cx, cy]` or `[f, cx, cy]`
            ([intr_to_kmat][deeperfly.geometry.intr_to_kmat]).
        dist: OpenCV-ordered distortion coefficients (may be empty).
        mirrored: Whether the view sees the world reflected, through a prism: its
            pose is improper (ADR-0053) and nothing flips the pixels.
    """

    rvec: Float[np.ndarray, "3"]
    tvec: Float[np.ndarray, "3"]
    intr: Float[np.ndarray, "P"]
    dist: Float[np.ndarray, "K"]
    mirrored: bool = False

    # The three below are computed once per view, which is immutable: each is a few
    # eager JAX ops (milliseconds), and per-keypoint geometry asks for them per call.

    @cached_property
    def rmat(self) -> Float[np.ndarray, "3 3"]:
        """The effective world-to-camera rotation (rows are the camera axes).

        `R(rvec)`, or the improper `diag(-1, 1, 1) @ R(rvec)` for a mirrored view.
        """
        return _frozen(pose_rmat_one(self.rvec, self.mirrored))

    @cached_property
    def kmat(self) -> Float[np.ndarray, "3 3"]:
        """Intrinsic matrix `K`."""
        return _frozen(intr_to_kmat(self.intr))

    @cached_property
    def position(self) -> Float[np.ndarray, "3"]:
        """The view's center in world coordinates, `-R.T @ tvec`."""
        return _frozen(-self.rmat.T @ self.tvec)

    def project(
        self, pts3d: Float[np.ndarray, "*pts 3"]
    ) -> Float[np.ndarray, "*pts 2"]:
        """Project world points of shape `(*pts, 3)` to this image, `(*pts, 2)`."""
        out = project_full(
            np.asarray(pts3d),
            self.rvec[None],
            self.tvec[None],
            self.intr[None],
            self.dist[None],
            np.array([self.mirrored]),
        )
        return np.asarray(out)[0]

    def backproject_ray(
        self, pixel: Float[np.ndarray, "2"]
    ) -> tuple[Float[np.ndarray, "3"], Float[np.ndarray, "3"]]:
        """The world-frame viewing ray `(origin, direction)` of an image `pixel`.

        The inverse of [project][deeperfly.rig.View.project], `origin` the camera
        center ([backproject_ray_one][deeperfly.geometry.backproject_ray_one]).
        """
        origin, direction = backproject_ray_one(
            jnp.asarray(pixel, dtype=float),
            jnp.asarray(self.rvec),
            jnp.asarray(self.tvec),
            jnp.asarray(self.intr),
            jnp.asarray(self.dist),
            jnp.asarray(self.mirrored),
        )
        return np.asarray(origin), np.asarray(direction)

kmat cached property

kmat: Float[ndarray, '3 3']

Intrinsic matrix K.

position cached property

position: Float[ndarray, 3]

The view's center in world coordinates, -R.T @ tvec.

rmat cached property

rmat: Float[ndarray, '3 3']

The effective world-to-camera rotation (rows are the camera axes).

R(rvec), or the improper diag(-1, 1, 1) @ R(rvec) for a mirrored view.

backproject_ray

backproject_ray(
    pixel: Float[ndarray, 2],
) -> tuple[Float[np.ndarray, 3], Float[np.ndarray, 3]]

The world-frame viewing ray (origin, direction) of an image pixel.

The inverse of project, origin the camera center (backproject_ray_one).

Source code in src/deeperfly/rig.py

def backproject_ray(
    self, pixel: Float[np.ndarray, "2"]
) -> tuple[Float[np.ndarray, "3"], Float[np.ndarray, "3"]]:
    """The world-frame viewing ray `(origin, direction)` of an image `pixel`.

    The inverse of [project][deeperfly.rig.View.project], `origin` the camera
    center ([backproject_ray_one][deeperfly.geometry.backproject_ray_one]).
    """
    origin, direction = backproject_ray_one(
        jnp.asarray(pixel, dtype=float),
        jnp.asarray(self.rvec),
        jnp.asarray(self.tvec),
        jnp.asarray(self.intr),
        jnp.asarray(self.dist),
        jnp.asarray(self.mirrored),
    )
    return np.asarray(origin), np.asarray(direction)

project

project(
    pts3d: Float[ndarray, "*pts 3"],
) -> Float[np.ndarray, "*pts 2"]

Project world points of shape (*pts, 3) to this image, (*pts, 2).

Source code in src/deeperfly/rig.py

def project(
    self, pts3d: Float[np.ndarray, "*pts 3"]
) -> Float[np.ndarray, "*pts 2"]:
    """Project world points of shape `(*pts, 3)` to this image, `(*pts, 2)`."""
    out = project_full(
        np.asarray(pts3d),
        self.rvec[None],
        self.tvec[None],
        self.intr[None],
        self.dist[None],
        np.array([self.mirrored]),
    )
    return np.asarray(out)[0]

resolve_extrinsics

resolve_extrinsics(
    distance: float,
    *,
    look_at: Sequence[float] = (0.0, 0.0, 0.0),
    azimuth_deg: float = 0.0,
    elevation_deg: float = 0.0,
    roll_deg: float = 0.0,
) -> tuple[Float[np.ndarray, 3], Float[np.ndarray, 3]]

Resolve an orbit placement to (rvec, tvec).

The camera sits at look_at + distance * dir(azimuth_deg, elevation_deg) and looks back at look_at with world +z up, then turns by roll_deg about its optical axis; angles in degrees.

Raises:

Type	Description
ValueError	If elevation_deg is +/-90, which leaves the roll undefined.

Source code in src/deeperfly/rig.py

def resolve_extrinsics(
    distance: float,
    *,
    look_at: Sequence[float] = (0.0, 0.0, 0.0),
    azimuth_deg: float = 0.0,
    elevation_deg: float = 0.0,
    roll_deg: float = 0.0,
) -> tuple[Float[np.ndarray, "3"], Float[np.ndarray, "3"]]:
    """Resolve an orbit placement to `(rvec, tvec)`.

    The camera sits at `look_at + distance * dir(azimuth_deg, elevation_deg)` and looks
    back at `look_at` with world `+z` up, then turns by `roll_deg` about its optical
    axis; angles in degrees.

    Raises:
        ValueError: If `elevation_deg` is +/-90, which leaves the roll undefined.
    """
    target = np.asarray(look_at, dtype=float)
    direction = _orbit_direction(azimuth_deg, elevation_deg)
    center = target + float(distance) * direction
    rmat = _roll_matrix(roll_deg) @ _look_rotation(-direction)
    rvec = np.asarray(rmat_to_rvec(rmat), dtype=float)
    return rvec, -rmat @ center

spec_union

spec_union(orbit: type[OrbitView], raw: type[RawView])

The orbit | raw union a table is parsed as, chosen by the keys present.

For ViewSpec, and for any subclass pair that adds keys.

Source code in src/deeperfly/rig.py

def spec_union(orbit: type[OrbitView], raw: type[RawView]):
    """The `orbit | raw` union a table is parsed as, chosen by the keys present.

    For [ViewSpec][deeperfly.rig.ViewSpec], and for any subclass pair that adds keys.
    """
    return Annotated[
        Annotated[orbit, Tag("orbit")] | Annotated[raw, Tag("raw")],
        Discriminator(_placement_kind),
        BeforeValidator(_one_whole_placement),
    ]