You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: doc/source/serve/production-guide/config.md
+25-1Lines changed: 25 additions & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -51,27 +51,49 @@ applications:
51
51
52
52
The file contains `proxy_location`, `http_options`, `grpc_options`, `logging_config` and `applications`.
53
53
54
+
(proxy-config)=
55
+
56
+
## Proxy config
57
+
54
58
The `proxy_location` field configures where to run proxies to handle traffic to the cluster. You can set `proxy_location` to the following values:
55
59
- EveryNode (default): Run a proxy on every node in the cluster that has at least one replica actor.
56
60
- HeadOnly: Only run a single proxy on the head node.
57
61
- Disabled: Don't run proxies at all. Set this value if you are only making calls to your applications using deployment handles.
58
62
63
+
(http-config)=
64
+
65
+
## HTTP config
66
+
59
67
The `http_options` are as follows. Note that the HTTP config is global to your Ray cluster, and you can't update it during runtime.
60
68
61
69
- **`host`**: The host IP address for Serve's HTTP proxies. This is optional and can be omitted. By default, the `host` is set to `0.0.0.0` to expose your deployments publicly. If you're using Kubernetes, you must set `host` to `0.0.0.0` to expose your deployments outside the cluster.
62
70
- **`port`**: The port for Serve's HTTP proxies. This parameter is optional and can be omitted. By default, the port is set to `8000`.
63
71
- **`request_timeout_s`**: Allows you to set the end-to-end timeout for a request before terminating and retrying at another replica. By default, there is no request timeout.
64
72
- **`keep_alive_timeout_s`**: Allows you to set the keep alive timeout for the HTTP proxy. For more details, see [here](serve-http-guide-keep-alive-timeout)
65
73
74
+
(grpc-config)=
75
+
76
+
## gRPC config
77
+
66
78
The `grpc_options` are as follows. Note that the gRPC config is global to your Ray cluster, and you can't update it during runtime.
67
79
- **`port`**: The port that the gRPC proxies listen on. These are optional settings and can be omitted. By default, the port is
68
80
set to `9000`.
69
81
- **`grpc_servicer_functions`**: List of import paths for gRPC `add_servicer_to_server` functions to add to Serve's gRPC proxy. The servicer functions need to be importable from the context of where Serve is running. This defaults to an empty list, which means the gRPC server isn't started.
70
82
- **`request_timeout_s`**: Allows you to set the end-to-end timeout for a request before terminating and retrying at another replica. By default, there is no request timeout.
71
83
84
+
(logging-config)=
85
+
86
+
## Logging config
87
+
72
88
The `logging_config` is global config, you can configure controller & proxy & replica logs. Note that you can also set application and deployment level logging config, which will take precedence over the global config. See logging config API [here](../../serve/api/doc/ray.serve.schema.LoggingConfig.rst) for more details.
73
89
74
-
These are the fields per application:
90
+
(application-config)=
91
+
92
+
## Application config
93
+
94
+
You configure one or more deployments as part of your Serve application. See [deployment config](serve-configure-deployment).
95
+
96
+
These are the fields per `application`:
75
97
76
98
- **`name`**: The names for each application that are auto-generated by `serve build`. The name of each application must be unique.
77
99
- **`route_prefix`**: An application can be called via HTTP at the specified route prefix. It defaults to `/`. The route prefix for each application must be unique.
@@ -80,6 +102,8 @@ These are the fields per application:
80
102
- **`deployments (optional)`**: A list of deployment options that allows you to override the `@serve.deployment` settings specified in the deployment graph code. Each entry in this list must include the deployment `name`, which must match one in the code. If this section is omitted, Serve launches all deployments in the graph with the parameters specified in the code. See how to [configure serve deployment options](serve-configure-deployment).
81
103
- **`args`**: Arguments that are passed to the [application builder](serve-app-builder-guide).
82
104
105
+
## Example config
106
+
83
107
Below is a config for the [`Text ML Model` example](serve-in-production-example) that follows the format explained above:
0 commit comments