Dynamically customizable timeout for envd init

[jakubno]

Increasing the timeout didn't affect start times, but the number of initialization requests went down.

---

Note

Envd init now uses a feature-flagged per-request timeout, records init attempt counts and duration with envd version attributes, and adds a logger field for envd version.

• Sandbox/Envd:
  • Make envd init request timeout configurable via feature flag (envd-init-request-timeout-milliseconds); store in internalConfig and use per-request.
  • Refactor initEnvd to use sandbox config directly; doRequestWithInfiniteRetries now takes timeout, returns attempt count, and logs retries with sandbox_id and envd.version.
  • Record successful and failed init attempts via telemetry.EnvdInitCalls with {success: true|false}, envd.version, and timeout_ms attributes.
  • Measure WaitForEnvd duration via telemetry.WaitForEnvdDurationHistogramName with envd.version and timeout_ms.
• Telemetry:
  • Add counter orchestrator.sandbox.envd.init.calls and histogram orchestrator.sandbox.envd.init.duration with units/descriptions.
  • Add telemetry.WithEnvdVersion attribute helper and wire through spans.
• Feature Flags:
  • Introduce EnvdInitTimeoutSeconds (envd-init-request-timeout-milliseconds) int flag with default 50.
• Logging:
  • Add logger.WithEnvdVersion and use it in retry logs and clock-drift warnings.
• Minor:
  • Formatting-only changes in template build phase builders.

^{Written by Cursor Bugbot for commit ce9c48c. This will update automatically on new commits. Configure here.}

[linear]

ENG-3083 Make envd init request time out customizable

[jakubno]

Bug: Envd Initialization Timeout Defaults Incorrectly

If the EnvdInitTimeoutSeconds feature flag fails to load, the envdInitRequestTimeout becomes 0 milliseconds. This causes immediate timeouts for envd initialization, despite the log message indicating a default value is used.

packages/orchestrator/internal/server/sandboxes.go#L107-L112

infra/packages/orchestrator/internal/server/sandboxes.go

Lines 107 to 112 in c347dc8

	// Get timeout from feature flag
	envdInitRequestTimeoutMs, err := s.featureFlags.IntFlag(ctx, featureflags.EnvdInitTimeoutSeconds)
	if err != nil {
	zap.L().Warn("failed to get envd timeout from feature flag, using default", zap.Error(err))
	}
	envdInitRequestTimeout := time.Duration(envdInitRequestTimeoutMs) * time.Millisecond

 

it's not it will use fallback - 50ms

[dobrac]

we won't be able to track how many attempts are required per one sandbox start, correct?

[jakubno]

we won't be able to track how many attempts are required per one sandbox start, correct?

we will - it's total/success