The project is dead but ready to get into contributor mode

[mazzy89]

The project is officially dead after SUSE acquired Rancher.

However, I would not like to see put in the trash. I'm willing to get the role of contributor or maintainer if official maintainers would allow that.

I've already forked it and continuing to upgrade k3s. I'm limiting myself to upgrade only k3s and not make any changes to the kernel because for my clusters, and I use a baked kernel forked from the RaspberryPi.

Let me know if this would be possible.

[psviderski]

For the context, it seems the Rancher's focus has been shifted from k3os to rancher/os2 that is built using the cOS-toolkit and based on openSUSE.
@ibuildthecloud has recently announced on Twitter that he has left SUSE/Rancher: https://twitter.com/ibuildthecloud/status/1492175776057217026. So I doubt that this experimental os2 project has a bright future (same as k3os) without him pushing it forward. The harvester project has been migrated off k3os to os2 though: harvester/harvester#581

[srgvg]

fyi -
there's a discussion going on here: #838

[alexdrl]

Sorry for probably being a bit off-topic, but as this seems bad news for the future, do you know if there is some kind of way to migrate an existing k3os installation (single-node) to another OS that is not so experimental and has better support?

I have no local storage, just fiddled with some Longhorn volumes and some NFS provisioner PVs...

I could try and set-up a multi-cluster into a ubuntu machine with just k3s, and later keep only one master, but want to avoid the cluster-init if possible, also tested quickly a k3s agent and it was not connecting to the master :(

[mazzy89]

All the discussions in #838 are interesting but at this moment the direction is not very clear.

Sure I believe SUSE will push for its own meat but at the moment I'm fine to push for new releases of k3os. In my fork, I've made available the pre-release that can be used to feed the upgrade controller and hence upgrade fast and automatically the clusters.

[andrewwebber]

I find this interesting as this reminds me of when CoreOS inc got acquired back in the day.

CoreOS got accquired by Redhat and now produces Redhat CoreOS, Fedora CoreOS (cloud optimized kubernetes distribution)

My point being that the CoreOS community maintainers successfully forked to FlatCar Linux https://www.flatcar.org/ (which ultimately got acquired by Microsoft but Microsoft seems to be encouraging them so far). This seems to be successful looking from the outside in.

I moved to K3OS as apposed to migrating to Flarcar Linux due to the K3 developer story. For example development parity between local development in k3s, k3d to k3os.

I would encourage a fork if k3s, k3d integration/maintenance is also being continued.
Taking Flatcar Linux and Redhat CoreOS as a reference I think the maintenance of dependencies would need to be considered and not under estimated (e.g. Rancher Upgrade Controller)

[themgt]

Yeah MicroOS/etc are interesting for sure, but they're not drop-in replacements for k3os and seem intentionally more generic/DIY. Combined with SUSE just abandoning k3os without (afaict) any official warning or notice or migration path, leaving the website up, etc leaves a bad taste in my mouth and doesn't incline me to spending time switching to a new experimental SUSE OS project.

It also seems like (for now at least) k3os can be rebuilt for updated k3s with a few lines of code change (again raising the question of why SUSE refuses to continue support for an EOL grace period), so I'd be very much in favor of a community fork and continuation of k3os.

[alborotogarcia]

@mazzy89 You'd still need to replace the upstream repo in the system upgrade controller for the k3os upgrade plan in

https://github.com/mazzy89/k3os/blob/a9c866997db4f7fcf004e0dedf0aa5cc0dd37d80/overlay/share/rancher/k3s/server/manifests/system-upgrade-plans/k3os-latest.yaml#L15

Also all image builds would need to run make e2e so that it creates a docker-compose env that tests for upgrades.. The only problem I see it concerns all OS packages that may need different rancher products such as longhorn.. The good news is that the repo itself is so well done that apart from that, Dapper and k3s requirements as-is are solid enough..

Hope that @dweomer could give some advice on it..

[mazzy89]

@alborotogarcia yes I know that but at the moment I patch it on my side and it’s working fine. If my fork is going to be adopted by a wider audience then it makes sense to create a git patch and adjust those parts.

[mazzy89]

In the upcoming week once I have some more free time I’m going to wire a release for the k3os with k3s 1.23 to avoid to remain behind with the k8s upstream releases.

[alborotogarcia]

@mazzy89 I'd be willing to contribute, so far k3os runs flawlessly with k3s v1.23.5 at least on arm64 👍

[alexdrl]

@mazzy89 @alborotogarcia hey guys, I'm not much of an expert, but I want to help on getting k3os running, at least with some testing updating my cluster. Updating my master node to MicroOS was a failure 😁

[mazzy89]

@alborotogarcia, that's great. At the moment, I'm running my fork with the release that I've cut here https://github.com/mazzy89/k3os/releases/tag/v0.22.2-k3s2r0. Then, I patched the controller so that updates could be pulled from my fork rather than the official one. It's working without issues. Now you have confirmed that the newer k3s release works too. Then I think it is time to upgrade.

My opinion is that I do not want to make the project die. The project is pretty robust, and the repository is well done. I can't see a valid reason to throw everything into the grass and leave many users uncertain.

Many words have already been spent on some possible replacements, but I could not find something as k3os.

That being said, I could see some attraction and interest by the community to continue to use the project.

I want to move the project into one separated Github organization and start with the two main things:

• cut a new release with the newer k3s version
• patch the controller pointing to the new Git ref
• spin up the existing automation on the new org (Github workflows, e2e tests, etc...)

[bkraegelin]

@mazzy89 could you please document your patches to system upgrade controller? Or even better - how do I modify it using Rancher UI? I'm in the middle of a large project which should be based on K3OS/K3S and RancherUI. The situation seems catastrophic.

The actual OS / the kernel and other binaries is not essential. But we need support for system upgrade controller.

Thanks,
Birger

[mazzy89]

As pointed out above, @bkraegelin, what you need to modify is the Plan at this line https://github.com/mazzy89/k3os/blob/a9c866997db4f7fcf004e0dedf0aa5cc0dd37d80/overlay/share/rancher/k3s/server/manifests/system-upgrade-plans/k3os-latest.yaml#L15

However, since the plan YAML manifest is included in the k3os image, at the reboot of the machine, due to its immutable nature, it will lose the change.

To make it persistent, I would need to make public a new release with the patched Plan manifest and publish the new k3os images.

About your question

how do I modify it using Rancher UI

I do not have much experience with the Rancher UI. I've seen it just a couple of times in my life but I can imagine that it's about to modify a YAML manifest line via the UI.

[mazzy89]

Also another point

publish the new k3os images.

Since I'm not linked to Rancher and/or SUSE, I do not have access to their Docker registry. This means I need to find where to host the new artifacts for my fork. It's still something in WIP.