Skip to content

Commit 8e3dec3

Browse files
authored
stable links for Ray serve (#56241) (#56253)
cherrypick in #56241
1 parent 7aabc42 commit 8e3dec3

File tree

1 file changed

+25
-1
lines changed

1 file changed

+25
-1
lines changed

doc/source/serve/production-guide/config.md

Lines changed: 25 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -51,27 +51,49 @@ applications:
5151
5252
The file contains `proxy_location`, `http_options`, `grpc_options`, `logging_config` and `applications`.
5353

54+
(proxy-config)=
55+
56+
## Proxy config
57+
5458
The `proxy_location` field configures where to run proxies to handle traffic to the cluster. You can set `proxy_location` to the following values:
5559
- EveryNode (default): Run a proxy on every node in the cluster that has at least one replica actor.
5660
- HeadOnly: Only run a single proxy on the head node.
5761
- Disabled: Don't run proxies at all. Set this value if you are only making calls to your applications using deployment handles.
5862

63+
(http-config)=
64+
65+
## HTTP config
66+
5967
The `http_options` are as follows. Note that the HTTP config is global to your Ray cluster, and you can't update it during runtime.
6068

6169
- **`host`**: The host IP address for Serve's HTTP proxies. This is optional and can be omitted. By default, the `host` is set to `0.0.0.0` to expose your deployments publicly. If you're using Kubernetes, you must set `host` to `0.0.0.0` to expose your deployments outside the cluster.
6270
- **`port`**: The port for Serve's HTTP proxies. This parameter is optional and can be omitted. By default, the port is set to `8000`.
6371
- **`request_timeout_s`**: Allows you to set the end-to-end timeout for a request before terminating and retrying at another replica. By default, there is no request timeout.
6472
- **`keep_alive_timeout_s`**: Allows you to set the keep alive timeout for the HTTP proxy. For more details, see [here](serve-http-guide-keep-alive-timeout)
6573

74+
(grpc-config)=
75+
76+
## gRPC config
77+
6678
The `grpc_options` are as follows. Note that the gRPC config is global to your Ray cluster, and you can't update it during runtime.
6779
- **`port`**: The port that the gRPC proxies listen on. These are optional settings and can be omitted. By default, the port is
6880
set to `9000`.
6981
- **`grpc_servicer_functions`**: List of import paths for gRPC `add_servicer_to_server` functions to add to Serve's gRPC proxy. The servicer functions need to be importable from the context of where Serve is running. This defaults to an empty list, which means the gRPC server isn't started.
7082
- **`request_timeout_s`**: Allows you to set the end-to-end timeout for a request before terminating and retrying at another replica. By default, there is no request timeout.
7183

84+
(logging-config)=
85+
86+
## Logging config
87+
7288
The `logging_config` is global config, you can configure controller & proxy & replica logs. Note that you can also set application and deployment level logging config, which will take precedence over the global config. See logging config API [here](../../serve/api/doc/ray.serve.schema.LoggingConfig.rst) for more details.
7389

74-
These are the fields per application:
90+
(application-config)=
91+
92+
## Application config
93+
94+
You configure one or more deployments as part of your Serve application. See [deployment config](serve-configure-deployment).
95+
96+
These are the fields per `application`:
7597

7698
- **`name`**: The names for each application that are auto-generated by `serve build`. The name of each application must be unique.
7799
- **`route_prefix`**: An application can be called via HTTP at the specified route prefix. It defaults to `/`. The route prefix for each application must be unique.
@@ -80,6 +102,8 @@ These are the fields per application:
80102
- **`deployments (optional)`**: A list of deployment options that allows you to override the `@serve.deployment` settings specified in the deployment graph code. Each entry in this list must include the deployment `name`, which must match one in the code. If this section is omitted, Serve launches all deployments in the graph with the parameters specified in the code. See how to [configure serve deployment options](serve-configure-deployment).
81103
- **`args`**: Arguments that are passed to the [application builder](serve-app-builder-guide).
82104

105+
## Example config
106+
83107
Below is a config for the [`Text ML Model` example](serve-in-production-example) that follows the format explained above:
84108

85109
```yaml

0 commit comments

Comments
 (0)