Implement Docker container build and publish actions

[d-mcknight]

Changes

This adds support for building and publishing containers ml_api and obico_web which are equivalent to the containers previously built at runtime with docker-compose.yml.

I added a run.sh entrypoint to obico_web so that the container could be run outside of the docker-compose.yml context by setting a new OBICO_CONTAINER envvar while maintaining backwards-compat. with the existing method/documentation.

I also added a symlink and changed the default sqlite database path to use /data within the container and added a volume mount; this way only one volume is needed and the compiled web app remains in the container and not a mounted volume.

I tested a clean setup and confirmed I was able to change the password, with that change persisting between container restarts. I also mounted media and the sqlite db from an existing installation I have and confirmed it loaded.

Issues

Closes #1052
Closes #1055

NOTE: When I access the application in a browser, I get an empty page; however the mobile app renders correctly with my printer and history including images. This is not reproducible with images built from the release branch.

TODO

• Identify browser UI load issue
• Update docker-compose.yml image references
• Confirm default image tags to be used in docker-compose.yml
• Identify and make any needed documentation updates

[d-mcknight]

Identify browser UI load issue

I tried building from the release branch and things appear to be working there, but the master branch resulted in an empty web page. I don't believe the issue is related to these changes

[d-mcknight]

Confirm default image tags to be used in docker-compose.yml

Used latest for now, since that will always be the most recently build image with a note to update to release after an image with that tag is published

[kennethjiang]

Can you let me know how I can trigger a github action to test it? @d-mcknight

[d-mcknight]

Can you let me know how I can trigger a github action to test it? @d-mcknight

Sorry I missed this earlier. Actions can only be run in a repository if they are defined on the default branch. For testing, I usually fork the repository and change the default branch on the fork. You can see runs on my fork (let me know if you need additional permissions to see those for some reason).

[kennethjiang]

No problem. sorry for the delay on my end. Your PR looks quite clean at the first glance. I'll try to find time to test it later today.

[kennethjiang]

@d-mcknight this action succeeded: https://github.com/TheSpaghettiDetective/obico-server/actions/runs/16253425958 Where can I see the uploaded docker images. Sorry, I am not very familiar with how GitHub workflow works.

[kennethjiang]

Remove everything here every time the server starts?

[d-mcknight]

Yes, I updated the compose file to mount the /data directory so that a user can specify where on their host system to save recordings and to keep user data separated from code.

The line after this creates a symlink to /data which is where I also put the sqlite db by default

[kennethjiang]

Not sure if I'm following. My question is if everything in /app/static_build/media (which I believe will include timelapses etc) will be erased every time run.sh runs.

[d-mcknight]

Currently, the static_build files are built on container start; I just kept the existing logic to minimize changes. With this PR, I moved the timelapses to /data/media which is linked to /app/static_build/media on line 14; the directory is removed on line 13 so that the link may be created.

I made this change so that /app doesn't need to be mounted on the host FS. The files will not be erased, because they are mounted to the container at /data/media.

Related, I might be able to simplify the container startup and drop some of this logic if python manage.py collectstatic can be run before python manage.py migrate, but I don't know enough about what those processes are doing to say if that would work.

[d-mcknight]

Where can I see the uploaded docker images. Sorry, I am not very familiar with how GitHub workflow works.

They're listed here; from the repository page on GitHub, there's a "Packages" section in the right column

[kennethjiang]

Do you think that there's a way to structure your code so that it's obvious that the current way of working is not touched at all, and the Docker image is only a new way of running the server? I really don't want to deal with a ton of questions like "I just pulled the update and now my server is bricked".

For instance, our cloud is some extra on top of the existing Dockerfile + docker-compose.yml. Now I launched your PR into our testing env, and it seems that it's now trying to connect to a different db than the one specified by DATABASE_URL env var.

[d-mcknight]

Do you think that there's a way to structure your code so that it's obvious that the current way of working is not touched at all, and the Docker image is only a new way of running the server? I really don't want to deal with a ton of questions like "I just pulled the update and now my server is bricked".

I'll have to think about this some, I agree though that this shouldn't be a breaking change. I see a couple options:

1. I can revert my change which mounts the database and media files to /data. I did this to separate code (/app) from user data/configuration and to keep that code in the container FS, rather than on the host.
2. I can add a migration to move existing media and configuration to /data. This might require some additional testing to identify and handle edge cases.
3. I can revert my changes to docker-compose.yml, and present these pre-built containers as an alternative deployment method. This would solve the immediate question of "why did an update break my setup", but adds a method of deployment, rather than updating one.

Option 3 is probably the best if the goal is to maintain backwards-compat with current workflows.

For instance, our cloud is some extra on top of the existing Dockerfile + docker-compose.yml. Now I launched your PR into our testing env, and it seems that it's now trying to connect to a different db than the one specified by DATABASE_URL env var.

This one is odd, I changed the default value in docker-compose.yml, but nothing should have affected the actual handling of envvars..

[kennethjiang]

Do you think that there's a way to structure your code so that it's obvious that the current way of working is not touched at all, and the Docker image is only a new way of running the server? I really don't want to deal with a ton of questions like "I just pulled the update and now my server is bricked".

I'll have to think about this some, I agree though that this shouldn't be a breaking change. I see a couple options:

1. I can revert my change which mounts the database and media files to `/data`. I did this to separate code (`/app`) from user data/configuration and to keep that code in the container FS, rather than on the host.

2. I can add a migration to move existing media and configuration to `/data`. This might require some additional testing to identify and handle edge cases.

3. I can revert my changes to `docker-compose.yml`, and present these pre-built containers as an alternative deployment method. This would solve the immediate question of "why did an update break my setup", but adds a method of deployment, rather than updating one.

Option 3 is probably the best if the goal is to maintain backwards-compat with current workflows.

For instance, our cloud is some extra on top of the existing Dockerfile + docker-compose.yml. Now I launched your PR into our testing env, and it seems that it's now trying to connect to a different db than the one specified by DATABASE_URL env var.

This one is odd, I changed the default value in docker-compose.yml, but nothing should have affected the actual handling of envvars..

Option 3 sounds like the best one to me too. Also I believe (hope) our cloud won't be affected if we go with option 3.

Thanks again for the contribution to this project!