feat: upgrade to magefile

[fmartingr]

Summary

• Migrating to Mage
• Targets migrated: server, proceesor, webapp, webapp/node_modules, deploy, dist, bundle, watch
• Makefile now calls mage on the migrated targets
• PluginCTL logic has been migrated to mage directly by calling mage pluginctl:command and it's available to other targets as well.

Improvements

• Can run dependendant targets in parallel (server and webapp for a build).
• Friendly error codes for checks (do you have Go? Do you have NPM? Do you have GNU's Tar?) with links to documentation
• Allow the plugin developers to specify other binaries to be built in their magefiles, there's an example in this plugin with the processor.
• Common asset management for plugins is centralized. No more copying and pasting configuration for the editor, golangci lint, github workflows, ...

pluginmage

All logic is bundled to this package which is the one we can move to the public package in the mattermost server for distribution.

$ mage
Targets:
  build:additionalBinaries    builds all additional binaries if set up by the plugin developers
  build:all                   builds both server, additional binaries, and webapp
  build:bundle                creates a distributable bundle of the plugin
  build:server                Build builds the server if it exists
  build:webapp                Build builds the webapp if it exists
  deploy:upload               builds and installs the plugin to a server
  dummy                       prints a hello message and plugin info
  pluginctl:deploy            uploads and enables a plugin via the Client4 API
  pluginctl:disable           disables a plugin via the Client4 API
  pluginctl:enable            enables a plugin via the Client4 API
  pluginctl:reset             disables and re-enables a plugin via the Client4 API
  webapp:dependencies         installs webapp dependencies using npm
  webapp:watch                builds and watches the webapp for changes, rebuilding automatically

Check the included README.

Requires mattermost/actions-workflows#53

[fmartingr]

@wiggin77 @lieut-data Requested you review to check on this first iteration of the effort. Failing e2e tests are due to #138.

[wiggin77]

/update-branch

[wiggin77]

Can we not just delete this line?

[wiggin77]

Shouldn't the mage code be in its own repo so it can be shared across plugins?

[fmartingr]

Shouldn't the mage code be in its own repo so it can be shared across plugins?

The idea is to have this approved so we can move it to the public package in the monorepo, so the pluginapi and pluginmage live togeher.

[fmartingr]

If we plan to move this forward long term I also need confirmation to press on mattermost/actions-workflows#53

[fmartingr]

To continue the discussion here, folks have told me that this may be easier to review if the plugin package and the changes to this repo were split into two. My idea was to have the pluginmage package live alongside the pluginapi package in the plugin repository, but I've heard of using a new external repository/package for this. While I'm not a fan of splitting this into several different codebases, I'm not against if that's what most of the people think it's best.

I want to unblock this so we can get it reviewed, so please share your feedback about this here so we can move this along.

cc: @wiggin77 @lieut-data @hanzei @davidkrauser @BenCookie95

[lieut-data]

I've heard of using a new external repository/package for this. While I'm not a fan of splitting this into several different codebases, I'm not against if that's what most of the people think it's best.

I'd like to propose that we start by putting this in the public package (alongside pluginapi), with all the tradeoffs that brings. It's the quickest way to market, and the worst that can happen is plugins can't opt-in to the new magefile without also upgrading the plugin dependency, etc.

Then, once we have momentum behind this, we can consider breaking it out into a separate repo if the pain of upgrading all of public becomes an issue. @wiggin77 surfaced this to be likely when it comes to fixing broken build pipelines for which we might not want to also update the plugin dependencies, but I'm willing to wager we can cross that bridge when we get there.

Thoughts @fmartingr?

[wiggin77]

I'd like to propose that we start by putting this in the public package (alongside pluginapi), with all the tradeoffs that brings. It's the quickest way to market, and the worst that can happen is plugins can't opt-in to the new magefile without also upgrading the plugin dependency, etc.

I'm fine with this being in the mono-repo. There is some benefit to that in keeping model version in sync. My only concern with it being in the "public" area is the downward compatibility guarantees that package has. This needs to be imported by all supported plugin repos, but I don't want to hand-cuff ourselves into not being able to iterate on it.

[lieut-data]

Incredibly excited with the direction this is heading! A few comments, but let's keep going :)

[lieut-data]

Future: use https://pkg.go.dev/archive/tar and avoid this dep altogether.

[lieut-data]

Can we use logrus and outsource this logic instead?

[fmartingr]

My idea was to use only builtins as much as possible, so we only depend on mage and the model package and prevent conflicts on plugins that use any other libraries. This customization is there for coloring and to easily spot namespace/target combination, which we would need to do for logrus as well.

[fmartingr]

I think the best spot for this is the public package unless we have strong evidence otherwise. Putting it in a separate repository may give us more freedom but we would end up in dependency problems either way, since public would be a dependency of the pluginmage package and the plugin as well.

I think we can land in two areas:

1. the one were we can't update to mage because the plugin has older dependencies and modernize it is not worth the effort (which we already see in some plugins having older monorepo deps)
2. Having the modern tooling require an upgrade of the plugin package leading to faster iteration cycles to maintain the plugins up to date, potentially leading to more improvements to the plugin API as well.