Clarification needed: released annotations (2D polylines) vs. spline/3D schema referenced in code and paper

[PRMA-PRMA]

Hi, thanks for releasing Guide3D. I’ve been trying to use the dataset while reading the paper, and I’m having trouble reconciling the released annotation format, the dataset code, and the reconstruction pipeline described in the paper.

From the public repo, the file data/annotations/3d.json appears to contain entries like:

{
"frame_number": 1,
"camera1": {
"image": "...",
"points": [...]
},
"camera2": {
"image": "...",
"points": [...]
}
}

These look like 2D polyline annotations in pixel coordinates for each camera view.

However, several parts of the codebase appear to reference a different annotation schema. For example:

• src/guide3d/dataset/video/b_spline.py references fields such as frame["cameraA"]["tck"], frame["cameraB"]["tck"], and frame["3d"]["tck"]
• dataset_utils.py references fields like img1, img2, and reconstruction

Those fields suggest an annotation structure containing B-spline representations and possibly 3D geometry, which I couldn’t locate in the released dataset.

Separately, the paper describes a calibration and reconstruction pipeline involving:

• undistortion
• marker identification + RANSAC calibration
• Direct Linear Transformation (DLT)
• nonlinear optimization

which would normally produce projection matrices and intrinsic/extrinsic parameters for the biplane setup. I wasn’t able to find any calibration metadata or camera parameters in the dataset release.

So at the moment it’s unclear how these pieces relate:

• the released annotations contain 2D polylines (camera1 / camera2)
• parts of the code reference spline-based annotations (tck, 3d)
• the paper describes calibrated 3D reconstruction

Could you clarify the intended relationship between these?

Specifically:

• Is data/annotations/3d.json the canonical annotation format for the public dataset?
• Are the spline-based annotations (tck, u, 3d) referenced in the code available somewhere, or were they part of an internal preprocessing pipeline?
• Are camera calibration parameters or projection matrices available for the stereo pairs?
• Which annotation schema should users rely on when trying to reproduce the reconstruction setup described in the paper?

I may simply be missing a preprocessing step or additional files, so any clarification would be appreciated. Right now it’s difficult to determine how the released annotations connect to the reconstruction workflow implemented in the code.

Thanks!