Better help

[michaeldwan]

This PR revamps help text for the fly command and its subcommands. The goal is to make our help text more helpful, easier to read, and easier to maintain going forward. This only addresses the framework for how help text is rendered. Updating the oodles of individual commands and flags will come in subsequent PRs.

Read on for an overview of what's new.

Improved Formatting

• All command descriptions have proper indentation, and dynamic line wrapping to fit the terminal width. Proper indentation was fixed recently in Consistent help text indentation using heredoc-style formatting #2722, this PR adds it to more things.
• Flag descriptions also have dynamic line wrapping to fit the terminal width.

Old Output, wrapped at 120 characters

$ fly deploy --help | fold -w 120
Deploy Fly applications from source or an image using a local or remote builder.

To disable colorized output and show full Docker build output, set the environment variable NO_COLOR=1.

Usage:
  flyctl deploy [WORKING_DIRECTORY] [flags]

Flags:
  -a, --app string                       Application name
      --auto-confirm                     Will automatically confirm changes when running non-interactively.
      --build-arg stringArray            Set of build time variables in the form of NAME=VALUE pairs. Can be specified m
ultiple times.
      --build-only                       Build but do not deploy
      --build-secret stringArray         Set of build secrets of NAME=VALUE pairs. Can be specified multiple times. See 
https://docs.docker.com/develop/develop-images/build_enhancements/#new-docker-build-secret-information
      --build-target string              Set the target build stage to build if the Dockerfile has more than one stage
  -c, --config string                    Path to application configuration file
      --detach                           Return immediately instead of monitoring deployment progress
      --dockerfile string                Path to a Dockerfile. Defaults to the Dockerfile in the working directory.
  -e, --env stringArray                  Set of environment variables in the form of NAME=VALUE pairs. Can be specified 
multiple times.
      --exclude-regions strings          Deploy to all machines except machines in these regions. Multiple regions can b
e specified with comma separated values or by providing the flag multiple times. --exclude-regions iad,sea --exclude-reg
ions syd will exclude all three iad, sea, and syd regions. Applied after --only-regions. V2 machines platform only.
      --file-literal stringArray         Set of literals in the form of /path/inside/machine=VALUE pairs where VALUE is 
the content. Can be specified multiple times.
      --file-local stringArray           Set of files in the form of /path/inside/machine=<local/path> pairs. Can be spe
cified multiple times.
      --file-secret stringArray          Set of secrets in the form of /path/inside/machine=SECRET pairs where SECRET is
 the name of the secret. Can be specified multiple times.
      --ha                               Create spare machines that increases app availability (default true)
  -h, --help                             help for deploy
      --ignorefile string                Path to a Docker ignore file. Defaults to the .dockerignore file in the working
 directory.
  -i, --image string                     The Docker image to deploy
      --image-label string               Image label to use when tagging and pushing to the fly registry. Defaults to "d
eployment-{timestamp}".
      --lease-timeout int                Seconds to lease individual machines while running deployment. All machines are
 leased at the beginning and released at the end. The lease is refreshed periodically for this same time, which is why i
t is short. flyctl releases leases in most cases. (default 13)
      --local-only                       Only perform builds locally using the local docker daemon
      --max-unavailable float            Max number of unavailable machines during rolling updates. A number between 0 a
nd 1 means percent of total machines (default 0.33)
      --nixpacks                         Deploy using nixpacks to build the image
      --no-cache                         Do not use the build cache when building the image
      --no-extensions                    Do not provision Sentry nor other auto-provisioned extensions
      --no-public-ips                    Do not allocate any new public IP addresses
      --now                              Deploy now without confirmation
      --only-regions strings             Deploy to machines only in these regions. Multiple regions can be specified wit
h comma separated values or by providing the flag multiple times. --only-regions iad,sea --only-regions syd will deploy 
to all three iad, sea, and syd regions. Applied before --exclude-regions. V2 machines platform only.
      --provision-extensions             Provision any extensions assigned as a default to first deployments
      --push                             Push image to registry after build is complete
  -r, --region string                    The target region (see 'flyctl platform regions')
      --release-command-timeout string   Seconds to wait for a release command finish running, or 'none' to disable. (de
fault "300")
      --remote-only                      Perform builds on a remote builder instance instead of using the local docker d
aemon
      --smoke-checks                     Perform smoke checks during deployment (default true)
      --strategy string                  The strategy for replacing running instances. Options are canary, rolling, blue
green, or immediate. Default is canary, or rolling when max-per-region is set.
      --update-only                      Do not create Machines for new process groups
      --vm-cpukind string                The kind of CPU to use ('shared' or 'performance')
      --vm-cpus int                      Number of CPUs
      --vm-memory int                    Memory (in megabytes) to attribute to the VM
      --vm-size string                   The VM size to use when deploying for the first time. See "fly platform vm-size
s" for valid values
      --wait-timeout int                 Seconds to wait for individual machines to transition states and become healthy
. (default 120)

Global Flags:
  -t, --access-token string   Fly API Access Token
      --debug                 Print additional logs and traces
      --verbose               Verbose output

New Output, wrapped at 120 characters

$ go run . deploy --help 
Deploy Fly applications from source or an image using a local or remote builder.  To disable colorized output and show
full Docker build output, set the environment variable NO_COLOR=1.

Usage:
  flyctl deploy [WORKING_DIRECTORY] [flags]

Flags:
  -a, --app string                       Application name
      --build-arg stringArray            Set of build time variables in the form of NAME=VALUE pairs. Can be
                                         specified multiple times.
      --build-only                       Build but do not deploy
      --build-secret stringArray         Set of build secrets of NAME=VALUE pairs. Can be specified multiple
                                         times. See
                                         https://docs.docker.com/engine/reference/commandline/buildx_build/#secret
      --build-target string              Set the target build stage to build if the Dockerfile has more than one stage
  -c, --config string                    Path to application configuration file
      --detach                           Return immediately instead of monitoring deployment progress
      --dockerfile string                Path to a Dockerfile. Defaults to the Dockerfile in the working directory.
  -e, --env stringArray                  Set of environment variables in the form of NAME=VALUE pairs. Can be
                                         specified multiple times.
      --exclude-regions strings          Deploy to all machines except machines in these regions. Multiple
                                         regions can be specified with comma separated values or by providing the
                                         flag multiple times. --exclude-regions iad,sea --exclude-regions syd
                                         will exclude all three iad, sea, and syd regions. Applied after
                                         --only-regions. V2 machines platform only.
      --file-literal stringArray         Set of literals in the form of /path/inside/machine=VALUE pairs where
                                         VALUE is the content. Can be specified multiple times.
      --file-local stringArray           Set of files in the form of /path/inside/machine=<local/path> pairs. Can
                                         be specified multiple times.
      --file-secret stringArray          Set of secrets in the form of /path/inside/machine=SECRET pairs where
                                         SECRET is the name of the secret. Can be specified multiple times.
      --ha                               Create spare machines that increases app availability (default true)
  -h, --help                             help for deploy
      --ignorefile string                Path to a Docker ignore file. Defaults to the .dockerignore file in the
                                         working directory.
  -i, --image string                     The Docker image to deploy
      --image-label string               Image label to use when tagging and pushing to the fly registry.
                                         Defaults to "deployment-{timestamp}".
      --lease-timeout int                Seconds to lease individual machines while running deployment. All
                                         machines are leased at the beginning and released at the end. The lease
                                         is refreshed periodically for this same time, which is why it is short.
                                         flyctl releases leases in most cases. (default 13)
      --local-only                       Only perform builds locally using the local docker daemon
      --max-unavailable float            Max number of unavailable machines during rolling updates. A number
                                         between 0 and 1 means percent of total machines (default 0.33)
      --nixpacks                         Deploy using nixpacks to build the image
      --no-cache                         Do not use the build cache when building the image
      --no-extensions                    Do not provision Sentry nor other auto-provisioned extensions
      --no-public-ips                    Do not allocate any new public IP addresses
      --now                              Deploy now without confirmation
      --only-regions strings             Deploy to machines only in these regions. Multiple regions can be
                                         specified with comma separated values or by providing the flag multiple
                                         times. --only-regions iad,sea --only-regions syd will deploy to all
                                         three iad, sea, and syd regions. Applied before --exclude-regions. V2
                                         machines platform only.
      --provision-extensions             Provision any extensions assigned as a default to first deployments
      --push                             Push image to registry after build is complete
  -r, --region string                    The target region (see 'flyctl platform regions')
      --release-command-timeout string   Seconds to wait for a release command finish running, or 'none' to
                                         disable. (default "300")
      --remote-only                      Perform builds on a remote builder instance instead of using the local
                                         docker daemon
      --smoke-checks                     Perform smoke checks during deployment (default true)
      --strategy string                  The strategy for replacing running instances. Options are canary,
                                         rolling, bluegreen, or immediate. Default is canary, or rolling when
                                         max-per-region is set.
      --update-only                      Do not create Machines for new process groups
      --vm-cpukind string                The kind of CPU to use ('shared' or 'performance')
      --vm-cpus int                      Number of CPUs
      --vm-memory int                    Memory (in megabytes) to attribute to the VM
      --vm-size string                   The VM size to use when deploying for the first time. See "fly platform
                                         vm-sizes" for valid values
      --wait-timeout int                 Seconds to wait for individual machines to transition states and become
                                         healthy. (default 120)
  -y, --yes                              Accept all confirmations

Global Flags:
  -t, --access-token string   Fly API Access Token
      --debug                 Print additional logs and traces
      --verbose               Verbose output

fly "getting started" output

Running fly without any commands, flags, or args has been printing common commands to help users get started for some time. This PR updates the output with more emphasis on the "launch" flow, cleaner formatting, and includes additional resources for help, like the community forum url.

Old output

$ fly
This is flyctl, the Fly.io command line interface.

Here's a few commands to get you started:
  fly launch      Launch a new application
  fly apps        Create and manage apps
  fly postgres    Create and manage Postgres databases
  fly mysql       Create and manage PlanetScale MySQL databases
  fly redis       Create and manage Upstash Redis databases
  fly machines    Create and manage individual Fly.io machines

If you need help along the way:
  fly help            Display a complete list of commands
  fly help <command>  Display help for a specific command, e.g. 'fly help launch'

Visit https://fly.io/docs for additional documentation & guides

New output

$ go run .
This is flyctl, the Fly.io command line interface.

Usage:
  flyctl [flags]
  flyctl [command]

Here's a few commands to get you started:
  flyctl launch    Create and configure a new app from source code or a Docker image.
  flyctl status    Show app status
  flyctl deploy    Deploy Fly applications
  flyctl logs      View app logs
  flyctl apps      Manage apps
  flyctl machine   Commands that manage machines

If you need help along the way:
  Use `fly docs` to open the Fly.io documentation, or visit https://fly.io/docs.
  Use `fly <command> --help` for more information about a command.
  Visit https://community.fly.io for help from the Fly.io community.

For a full list of commands, run `fly help`.

Command grouping for fly help and fly --help

For a while now running fly help would print a manually curated list of commands, grouped by theme. This went a long way to taming the massive list of commands at the root. But there were two issues with it:

• hardcoded groups and commands are out of date; deprecated commands were still listed, and important new commands like fly console were missing
• the grouped output was only shown when running fly help. Running fly --help (which was suggested everywhere) would still show the default output which included an obnoxious list of 54 commands

This PR fixes these issues by taking advantage of cobra's newish command grouping feature. We tag individual commands with a group, and anytime help text is printed, the grouping will reflect it. Formatting and dynamic wrapping apply as well. Commands without a group are still shown in an "Additional Commands" section, so new command won't be inadvertantly hidden. And behavior is consistent between fly help and fly --help.

Old `fly help` output:

$ fly help

Deploying apps and machines:
  apps            Manage apps
  machine         Commands that manage machines
  launch          Create and configure a new app from source code or a Docker image.
  deploy          Deploy Fly applications
  destroy         Permanently destroys an app
  open            Open browser to current deployed application

Scaling and configuring:
  scale           Scale app resources
  regions         V1 APPS ONLY: Manage regions
  secrets         Manage application secrets with the set and unset commands.

Provisioning storage:
  volumes         Volume management commands
  mysql           Provision and manage PlanetScale MySQL databases
  postgres        Manage Postgres clusters.
  redis           Launch and manage Redis databases managed by Upstash.com
  consul          Enable and manage Consul clusters

Networking configuration:
  ips             Manage IP addresses for apps
  wireguard       Commands that manage WireGuard peer connections
  proxy           Proxies connections to a fly VM
  certs           Manage certificates

Monitoring and managing things:
  logs            View app logs
  status          Show app status
  dashboard       Open web browser on Fly Web UI for this app
  dig             Make DNS requests against Fly.io's internal DNS server
  ping            Test connectivity with ICMP ping messages
  ssh             Use SSH to login to or run commands on VMs
  sftp            Get or put files from a remote VM.

Platform overview:
  platform        Fly platform information

Access control:
  orgs            Commands for managing Fly organizations
  auth            Manage authentication
  move            Move an app to another organization

More help:
  docs            View Fly documentation
  doctor          The DOCTOR command allows you to debug your Fly environment
  help commands   A complete list of commands (there are a bunch more)

Old `fly --help` output:

$ fly --help
flyctl is a command line interface to the Fly.io platform.

It allows users to manage authentication, application launch,
deployment, network configuration, logging and more with just the
one command.

* Launch an app with the launch command
* Deploy an app with the deploy command
* View a deployed web application with the open command
* Check the status of an application with the status command

To read more, use the docs command to view Fly's help on the web.

Usage:
  flyctl [flags]
  flyctl [command]

Available Commands:
  agent         Commands that manage the Fly agent, a background process that manages flyctl wireguard connections
  apps          Manage apps
  auth          Manage authentication
  autoscale     V1 APPS ONLY: Autoscaling app resources
  certs         Manage certificates
  checks        Manage health checks
  completion    Generate the autocompletion script for the specified shell
  config        Manage an app's configuration
  console       Run a console in a new or existing machine
  consul        Enable and manage Consul clusters
  curl          Run a performance test against a URL
  dashboard     Open web browser on Fly Web UI for this app
  deploy        Deploy Fly applications
  destroy       Permanently destroys an app
  dig           Make DNS requests against Fly.io's internal DNS server
  dns-records   Manage DNS records
  docs          View Fly documentation
  doctor        The DOCTOR command allows you to debug your Fly environment
  extensions    Extensions are additional functionality that can be added to your Fly apps
  help          Help on flyctl commands
  history       List an app's change history
  image         Manage app image
  ips           Manage IP addresses for apps
  jobs          Show jobs at Fly.io
  launch        Create and configure a new app from source code or a Docker image.
  logs          View app logs
  machine       Commands that manage machines
  migrate-to-v2 Migrate a v1 app to v2
  monitor       Monitor currently running application deployments
  move          Move an app to another organization
  open          Open browser to current deployed application
  orgs          Commands for managing Fly organizations
  ping          Test connectivity with ICMP ping messages
  planetscale   Provision and manage PlanetScale MySQL databases
  platform      Fly platform information
  postgres      Manage Postgres clusters.
  proxy         Proxies connections to a fly VM
  redis         Launch and manage Redis databases managed by Upstash.com
  regions       V1 APPS ONLY: Manage regions
  releases      List app releases
  resume        Resume an application
  scale         Scale app resources
  secrets       Manage application secrets with the set and unset commands.
  services      Show the application's services
  settings      Manage flyctl settings
  sftp          Get or put files from a remote VM.
  ssh           Use SSH to login to or run commands on VMs
  status        Show app status
  suspend       Suspend an application
  tokens        Manage Fly.io API tokens
  turboku       Launch a Heroku app on Fly.io
  version       Show version information for the flyctl command
  vm            Commands that manage VM instances
  volumes       Volume management commands
  wireguard     Commands that manage WireGuard peer connections

Flags:
  -t, --access-token string   Fly API Access Token
      --debug                 Print additional logs and traces
  -h, --help                  help for flyctl
      --verbose               Verbose output

Use "flyctl [command] --help" for more information about a command.

New `fly help` and `fly --help` output:

$ go run . --help
This is flyctl, the Fly.io command line interface.

Usage:
  flyctl [flags]
  flyctl [command]

Deploying apps & machines
  apps          Manage apps
  deploy        Deploy Fly applications
  launch        Create and configure a new app from source code or a Docker image.
  machine       Commands that manage machines
  status        Show app status

Configuration & scaling
  certs         Manage certificates
  config        Manage an app's configuration
  image         Manage app image
  ips           Manage IP addresses for apps
  scale         Scale app resources
  secrets       Manage application secrets with the set and unset commands.
  volumes       Volume management commands

Monitoring & managing things
  checks        Manage health checks
  console       Run a console in a new or existing machine
  dashboard     Open web browser on Fly Web UI for this app
  dig           Make DNS requests against Fly.io's internal DNS server
  logs          View app logs
  ping          Test connectivity with ICMP ping messages
  proxy         Proxies connections to a fly VM
  releases      List app releases
  services      Show the application's services
  sftp          Get or put files from a remote VM.
  ssh           Use SSH to login to or run commands on VMs
  wireguard     Commands that manage WireGuard peer connections

Databases & extensions
  consul        Enable and manage Consul clusters
  extensions    Extensions are additional functionality that can be added to your Fly apps
  planetscale   Provision and manage PlanetScale MySQL databases
  postgres      Manage Postgres clusters.
  redis         Launch and manage Redis databases managed by Upstash.com

Access control
  auth          Manage authentication
  orgs          Commands for managing Fly organizations
  tokens        Manage Fly.io API tokens

Help & troubleshooting
  docs          View Fly documentation
  doctor        The DOCTOR command allows you to debug your Fly environment
  platform      Fly platform information

Apps v1 (deprecated)
  autoscale     V1 APPS ONLY: Autoscaling app resources
  migrate-to-v2 Migrate a v1 app to v2
  monitor       Monitor currently running application deployments
  regions       V1 APPS ONLY: Manage regions
  vm            Commands that manage VM instances

Additional Commands:
  agent         Commands that manage the Fly agent, a background process that manages flyctl wireguard connections
  completion    Generate the autocompletion script for the specified shell
  help          Help about any command
  jobs          Show jobs at Fly.io
  settings      Manage flyctl settings
  turboku       Launch a Heroku app on Fly.io
  version       Show version information for the flyctl command

Flags:
  -t, --access-token string   Fly API Access Token
      --debug                 Print additional logs and traces
  -h, --help                  help for flyctl
      --verbose               Verbose output

Use "flyctl [command] --help" for more information about a command.

Hiding and deprecating commands

In order to keep the root command list tidy, I've marked the following commands as hidden and added a deprecation message if not already set. These commands have all been mentioned as being deprecated, most printed a message already, so no surprises expected. They'll get removed in like a week.

• fly create - deprecated in favor of fly apps create
• fly curl - deprecated, pending removal
• fly destroy - deprecated in favor of fly apps destroy
• fly dns-records and children - deprecated, pending removal
• fly domains and children - deprecated, pending removal
• fly history - deprecated in favor of fly apps releases
• fly move - deprecated in favor of fly apps move
• fly open - deprecated in favor of fly apps open
• fly resume - deprecated in favor of fly scale count
• fly suspend - deprecated in favor of fly scale count

[redjonzaci]

I wonder if there is any tool that checks for broken URLs.

[michaeldwan]

@redjonzaci that's a good idea! I'll add it to the audit tool.

[gwuah]

good set of changes + good PR description.