Create AprilTag minimal working example with dedicated hardware setup with corresponding Colab

[sgbaird]

This is sort of a follow-up to a mini-hackathon project at POSE 2025.

Include link to AprilTag tutorial notebook

Let's use the vial turntable (probably best, but does require setting it up as IoLT device). Success in the end looks like a notebook where you can pick up a vial from the turntable, rotate the turntable somewhat, recalibrates position with the AprilTag, and run the same workflow picking up the same vial.

Later, we can transfer vials in and out of OT-2 (but OT-2 can't be moved easily to show robustness to slight deviations, so wouldn't be automated demo).

Get camera intrinsics via AprilTag tutorial notebook and hardcode into this notebook.

We'll have a livestream running to help with remote development.

---

Aside: If using MyCobot 280 Pi, see this post for info/videos of it being used

Aside: Consider having the zero 2w camera instead of MyCobot camera.

Possibly with the point and shoot RPi camera (see issue) which also has the benefit of the ring light compatibility.
Consider 3d printing some kind of peg board for radial theta / r movement and calibration, with an AprilTag fixed to a block in the middle.

[sgbaird]

Aside: @yakavetsiv using AprilTags

Generally tries to use two AprilTags to pose ambiguity

Planned use with depth camera that SDL3(?) also has. Custom drilled/installed onto plate crane

Doing something interesting with trying to avoid calibration (maybe didn't fully understand, but I think had to do with having a calibration AprilTag that's fixtured to the starting local reference frame?)

[ZZZhaoz]

I’ve got a few ideas for the program. Let me know if they make sense!

Program Proposal

Since AprilTags can only be attached to the side of the turntable, I propose fixing the camera to the base of the MyCobot, facing the turntable horizontally.

Workflow

1. The turntable rotates slowly after startup.
2. The camera continuously takes photos to detect AprilTags. The camera position or focus can be adjusted to make sure only one or a few tags appear in each frame.
3. Once the target AprilTag is detected, the turntable stops. The corresponding vial is now in a good position for the robot to grab.
4. The system re-detects the tag and converts its position from image coordinates to robot coordinates using a pre-calibrated transformation matrix. The MyCobot then grabs or places the vial from the side.
5. Users can optionally control the rotation speed to increase uncertainty in the stopping position, making the use of AprilTags' position estimation necessary and meaningful.

User Experience

Since we detect AprilTags on the turntable, not on the vials themselves, we can’t directly tell which vial is in which slot. To solve this:

• Each vial gets a physical label (Vial ID).
• Each position on the turntable gets a physical label (Turntable ID) (e.g., placed at the center area).

We maintain two bidirectional mappings in the program:

1. Turntable ID ↔ AprilTag ID
2. Turntable ID ↔ Vial ID

This way, the user can say something like “place vial 3 at position 5,” and the system can automatically figure out which AprilTag to look for to pick up the vial and which one to use to place the vial.

[sgbaird]

I think I'd like to still utilize the camera attached to the cobot arm, since it generalizes to other workflows. I'll give this some more thought though.

Rotating until the desired AprilTag comes into view would be fine I think, especially from a demo perspective. We could also rotate it to the spot we "think" it should be based on a last known position, but also focus on having a calibration routine to find "slot 1", and then assume we know how much to rotate to get to slot 5, for example. In contrast to waiting only until it comes into view, we could take it a step further and try to minimize the distance to the camera (something really simple would probably be fine).

I may be overcomplicating it though. Part of the trouble is that for this workflow, ideally we'd have position feedback built-in for the turntable, so in some sense it becomes a bit contrived that we're circumventing that approach and going straight to computer vision.

Based on your description, I now start to lean slightly towards putting an AprilTag on top of each vial cap.

Predetermining the desired robot gripper position and orientation relative to the camera and the vial will be important. I.e., the transformation matrix you mentioned.

I like the idea of rotation speed.

This helped me think through the workflow. Thanks! Looking forward to chatting tomorrow with the equipment in hand.

[sgbaird]

Apriltags used on vials apriltag_sheet_vial.pdf

Originally posted by @ZZZhaoz in #230

This is using tagCircle49h12

[sgbaird]

We're also deciding whether to use tagStandard41h12 (~2k unique tags) vs. tagStandard52h13 (~48k unique tags)

A brief search indicates that rather than precision, the difference may be more about detection time and false positive rate: https://www.google.com/search?q=accuracy+of+tagStandard41h12+vs.+tagStandard52h13. It wasn't immediately obvious from the AprilTag papers (i.e., these two weren't explicitly mentioned, though there's certainly some discussion around accuracy). It also wasn't immediately obvious which tag family was used in https://ieeexplore.ieee.org/document/9213977.

For now, we'll use tagStandard52h13, with the assumption that we'll scale quite a bit, especially as we get the label printer (#70) fully set up and connected with an inventory system of some kind (#172).

Aside: printing it on a bit thicker paper (I don't remember the weight though).

[ZZZhaoz]

Apriltags used on the turntable apriltag_sheet (34).pdf with tagStandard52h13

[ZZZhaoz]

Updated: apriltag_sheet (38).pdf

[ZZZhaoz]

apriltag_sheet (39).pdf