Allow limiting system resource usage

[GrumpyBear57]

I'd like to be able to limit how many system resources the KT runner is allowed to consume. Something like MAX_CPU and MAX_MEM where mem accepts a percent or a unit. I also would like a system variant (MAX_SYS_CPU and MAX_SYS_MEM) that prevents the runner from starting new containers if the system wide utilization would exceed those values.
The runner should also probably default to some sane values (i.e. 50% for runner specific, and 80% for system wide)and check to ensure that there is enough to run a single container (i.e. > 5% CPU, > 500MB memory).

A network utilization limit may also be nice for other people hosting the runner on slower connections, but this is probably lower priority since the network utilization seems rather low anyways.

example:

MAX_CPU=60%
MAX_SYS_CPU=90%
MAX_MEM=50%
MAX_SYS_MEM=80%

or

MAX_MEM=8192MB
MAX_SYS_MEM=80%

or

MAX_MEM=8GB
MAX_SYS_MEM=16GB

[KentoNishi]

You can now set limits using MAX_MEM_PERCENT and MAX_CPU_PERCENT for all KT containers including the runner. It's impossible to estimate how much a container will use once started, so the limit is checked before starting. Ex. if your limit is 8GB/16GB, you would set MAX_MEM_PERCENT=50 and it will only start new containers when usage is below 50%. This means that it could theoretically go above 50% but eh 🤷

[KentoNishi]

also the controller prioritizes hosts based on the relative usage, so the maximum relative load (memory or cpu) is reported to the server

[KentoNishi]

Done in 9ef7bd9. you can now use

MAX_CPU=50%
MAX_MEM=4GB

or

MAX_CPU=50%
MAX_MEM=50%