OT-2 liquid color matching for MSE403H1 practical

[Neil-YL]

SGB FOREWORD: Students in the course will send commands (red, yellow, blue paint dispense volumes) to an Opentrons OT-2 liquid handler and retrieve light sensor data (8 channels) and calculate the distance between the measured spectrum and a target spectrum (e.g., RMSE distance). The job of each student is to find the optimal combination of paint that minimizes the color distance. The students will interact with the system in a "gray-box" format. The students will be provided a Colab notebook that contains a function to send commands, receive the data, and calculate the quality metrics. It will be up to them to choose/design an optimization strategy.

---

I am testing a Hugging face app demo locally for the practical.

Input block:
student id here acts as the session_id in our MQTT setting and there are another experiment_id for each run with the sensor
R and Y will be used to calculate the B (R+Y+B=1)

If there is a queue for the experiment

when your experiment is running

Result:

I will finalise the page tomorrow and deploy it on Hugging face to test remote access.

@sgbaird I might need the admin login for the Nokia router to enable Hugging Face to access the OT-2 via SSH.

Another things to do:

• Colab page for the objective calculation
• The sensor may need adjustment to provide more distinguishable readings. (compared with the light demo, the reading is much smaller). I may also try turning on the light within the OT-2 to see if result will be better.
• any suggestions?

[sgbaird]

Nice! ☺️

• leave as RYB and have the values be in uL. Throw an error if someone requests more than the max allowed in a single well plate
• if you deploy to Hugging Face Spaces and use MQTT (or Prefect) then you shouldn't need SSH access, right?
• please try printing the sensor housing in black
• let's use washable, non-toxic paint instead of food coloring. Can you find something that looks like it would work from Amazon and include the link here?
• how did you implement the queue?

Can you share the code on Hugging Face Spaces and share the code that's running on the OT-2? I'll need that to provide better feedback

Thanks for working on this. Getting closer!

[sgbaird]

I'd like to take a look at how you implemented the queue mechanism.

Aside: @Neil-YL have a look at https://ac-training-lab.readthedocs.io/en/latest/devices/setup_iolt_devices.html#hugging-face-spaces (specifically https://ac-training-lab.readthedocs.io/en/latest/devices/setup_iolt_devices.html#hugging-face-spaces)

Related to the remote access and Prefect: PrefectHQ/prefect#16685

[Neil-YL]

• let's use washable, non-toxic paint instead of food coloring. Can you find something that looks like it would work from Amazon and include the link here?

RYBG paint: no sure what kind of paint is it

Tempera Paint: Tempera is water-based paint and the thickness can be adjusted (just like food coloring), but could become solid leftover in pipette tip if fully dried out.

There are options of tempera RYB color in single bottle as well: red, yellow blue (cannot find a 16oz blue on amazon, this is a 32oz one)

[brendenpelkie]

We've had good luck with acrylic paint thinned in water, probably a 10:1 water:paint ratio, if not weaker. Tempura paint is probably equivalent - I think we just have whatever was on the shelf at a craft store. We've generally used opaque colored liquids to mix. You can also use food coloring + milk in a pinch, but then you need to carefully wash everything or it starts to smell.

[Neil-YL]

We've had good luck with acrylic paint thinned in water, probably a 10:1 water:paint ratio, if not weaker. Tempura paint is probably equivalent - I think we just have whatever was on the shelf at a craft store. We've generally used opaque colored liquids to mix. You can also use food coloring + milk in a pinch, but then you need to carefully wash everything or it starts to smell.

Thank you for sharing your experience!

Originally, acrylic was the first common paint that I ruled out because I heard it could be used to paint on clothing and the drawings would be waterproof. However, your successful experience and suggestion of the 10:1 water-to-paint ratio is really helpful and gives me more confidence to give it a try.

[sgbaird]

I went ahead and ordered some acrylic and some washable. Thanks, both!

Acrylic: https://a.co/d/5aOw6DW
Washable: https://a.co/d/35i7Ost

Should arrive Monday.

[Neil-YL]

I am working on switching the SSH to MQTT on OT-2 and I have some questions about how you think on the queuing and the MQTT topic assigned for the device.

Currently, the queue for students to submit their parameters is managed on Hugging Face. Would you prefer that I manage this queue through MQTT instead? If so, I believe this functionality should be integrated into the OT-2 MQTT setup.

Another thing is the sensor now only subscribes to one same topic (I am using your "absurd-gazelle" course_id for the topic) but it is getting different student_id (which is session_id) submitted by the students and a randomised experiment_id for each run. Is this the intended way to repeatedly use the same MQTT topic or I should assign different course_ids to different students and let the sensor subscribe to all the course_ids topic? I would lean towards reusing same topic and tagged the in/out data with different student/session_id.

[sgbaird]

I think one topic for the sensor, and use a pico_id instead of course_id for differentiating the device.

Based on https://github.com/sparks-baird/self-driving-lab-demo/blob/cadf5cd8f5ccaff3394afa1791b9eac55fed94d0/src/public_mqtt_sdl_demo/main.py#L63C5-L63C42

from ubinascii import hexlify
from machine import unique_id
pico_id = hexlify(unique_id()).decode()
print(f"PICO_ID: {my_id}")

with open("pico_id.txt", "w") as f:
    f.write(pico_id)

You'll need to run this once on the Pico W to see what the ID is (either look at command line or the file that gets written).

You'll only need to hard-code this onto the orchestrator. You can add PICO_ID as a secret on HF spaces and import it into your app.py.

OT2

Since the OT-2 is based on a Raspberry Pi, you can do something similar for a device ID. See https://raspberrypi.stackexchange.com/questions/2086/how-do-i-get-the-serial-number

def getserial():
  # Extract serial from cpuinfo file
  cpuserial = "0000000000000000"
  try:
    f = open('/proc/cpuinfo','r')
    for line in f:
      if line[0:6]=='Serial':
        cpuserial = line[10:26]
    f.close()
  except:
    cpuserial = "ERROR000000000"

  return cpuserial

If this is problematic with the OT-2, feel free to just pick something or ignore (we only have one OT-2 at the moment ;) )

Let's use the OT-2's serial number and also add this as a secret on HF spaces. https://support.opentrons.com/s/article/Checking-your-OT-2-s-serial-number

Possible topic structure:

sensor_data_topic = f"color-mixing/picow/{PICO_ID}/as7341"
command_topic = f"color-mixing/ot2/{OT2_SERIAL}/pipette"

[sgbaird]

Currently, the queue for students to submit their parameters is managed on Hugging Face. Would you prefer that I manage this queue through MQTT instead? If so, I believe this functionality should be integrated into the OT-2 MQTT setup.

The queue can be on the app.py side.

This is how I've implemented it before: https://huggingface.co/spaces/AccelerationConsortium/light-mixing/blob/916acb2bcee4d81d81020c3ee5f46808d2572a77/app.py#L78. I think we can just stick with this for now, and we'll worry about having a per-student budget/restriction a bit later. I need to give this some more thought.

Aside: @gursi26 has implemented a concurrency limit set to 1 on gradio, but this might have been more about temporary access (i.e., here's your five minute access pass):
https://huggingface.co/spaces/AccelerationConsortium/cobot280pi-gradio/blob/39dd3bf0b5dca8e0aebe286594fd8e65d20df40c/app.py#L359

[Neil-YL]

I think we can just stick with this for now, and we'll worry about having a per-student budget/restriction a bit later. I need to give this some more thought.

Currently I have a MongoDB to log-in the data of well-plate status (empty/used) so I think I can also mark down the student_id for the budget consumption purpose, but we may need students sign up for their ID first.

Update for the colors:

The Crayola washable paint is a little bit too viscous to use in our current set up (given the size of pipette tip and well volume), resulting in very visible layering of colors after mixing. I turned to acrylic paint and it mixed better.

The results are quite good with the OT-2 light on.:

C4 appears slightly bluer than C5, and the results for CH440 and CH470 show the most noticeable differences between the two measurements.

Thank you again for @brendenpelkie sharing your successful experience.

Basic colors I have now:

[sgbaird]

Nice! Aside: Can the crayola colors be diluted?

[Neil-YL]

Nice! Aside: Can the crayola colors be diluted?

I diluted it quite a bit (at least 1:15-20) but it still has three layering colors and very sticky bubble after the blow-out process of the tip.

[sgbaird]

Acrylic it is, then! :D Thanks for checking.