From b86eeb4ba442c7bfe740b406601a1f52fe83fec8 Mon Sep 17 00:00:00 2001 From: Tamara Boehm Date: Mon, 3 Nov 2025 14:24:49 +0100 Subject: [PATCH 1/2] Document hash-based routing Co-authored-by: Clemens Hoffmann --- custom-per-route-options.html.md.erb | 156 ++++++++++++++------ deploy-apps/manifest-attributes.html.md.erb | 9 +- hash-based-routing.html.md.erb | 58 ++++++++ 3 files changed, 177 insertions(+), 46 deletions(-) create mode 100644 hash-based-routing.html.md.erb diff --git a/custom-per-route-options.html.md.erb b/custom-per-route-options.html.md.erb index 706878b0..7c715472 100644 --- a/custom-per-route-options.html.md.erb +++ b/custom-per-route-options.html.md.erb @@ -8,10 +8,12 @@ By default, communication between Gorouter and backends is configured through th This topic describes how to specify per-route Gorouter options scoped at the application level. This greater granularity lets developers tailor optimal routing behavior for applications' unique load profiles or other requirements. -Gorouter supports the following per-route option, described in the section below: +Gorouter supports the following per-route options, described in the sections below: - `loadbalancing`: Configures the load balancing algorithm used by Gorouter for this particular route. <%= vars.per_route_lb_version %> - - Settings: `round-robin`, `least-connection`. + - Settings: `round-robin`, `least-connection`, `hash`. +- `hash_header`: Defines the header Gorouter uses for routing decisions on this route. Required when `loadbalancing` is set to `hash`. Cannot be used with other load balancing algorithms. <%= vars.hash_routing_version %> +- `hash_balance`: Sets the float number for the balance factor used by Gorouter to manage load imbalance applying the hash-based routing for this route. Optional when `loadbalancing` is `hash`. Cannot be used with other algorithms. <%= vars.hash_routing_version %> ## Configure Gorouter's Load Balancing Algorithm @@ -19,13 +21,13 @@ Gorouter supports the following per-route option, described in the section below The per-route option `loadbalancing` allows configuring the load balancing algorithm, which defines how the load is distributed between Gorouters and backends. -This option supports two settings for load balancing: +This option supports three settings for load balancing: - `round-robin` distributes the load evenly across all available backends - `least-connection` directs traffic to the backend with the fewest active connections at any given time, optimizing resource utilization +- `hash` distributes requests based on a hash of a specific HTTP header, ensuring requests with the same header are consistently directed to the same backend. <%= vars.hash_routing_version %> - -### Configure Load Balancing in an App Manifest +### Configure Load Balancing using an App Manifest To configure per-route load balancing for an application that has not yet been pushed: @@ -49,91 +51,155 @@ To configure per-route load balancing for an application that has not yet been p cf push -f manifest.yml ``` -### Change Load Balancing Algorithm of an Existing Route +### Create a Route with a Specific Load Balancing Algorithm Using the CF CLI + +To create a route with a per-route `loadbalancing` option, you can use the CLI command `create-route`. +For example: + +```console +cf create-route EXAMPLE.COM --hostname MY-HOST --option loadbalancing=round-robin +``` + +### Map a Route to an Existing App with a Specific Load Balancing Algorithm Using the CF CLI + +To create and map a new route to an existing application with the per-route `loadbalancing` option, you can use the CLI command `map-route`. + +For example: + +```console +cf map-route MY-APP EXAMPLE.COM --hostname MY-HOST --option loadbalancing=round-robin +``` + +

+The command map-route supports the --option flag only for new routes. +To update an existing route, use the command update-route described below.

+ + +### Update the Load Balancing Algorithm of an Existing Route Using the CF CLI -To change the per-route `loadbalancing` option of an existing route, you can use the cli command, `update-route`. +To change the per-route `loadbalancing` option of an existing route, you can use the CLI command `update-route`. For example, to change an app route's algorithm from `least-connection` to `round-robin`, you can run the `update-route` command: ```console cf update-route EXAMPLE.COM --hostname MY-HOST --option loadbalancing=round-robin ``` +### Remove the Specific Load Balancing Algorithm Using the CF CLI -Alternatively, it is also possible to update the per-route load balancing option via the `/v3/routes` API. - -Run the `PATCH` request to the targeted API endpoint: +To remove the `loadbalancing` option from an existing route, run: ```console -cf curl /v3/routes/GUID -X PATCH -H "Content-type: application/json" \ - -d '{ - "options": { - "loadbalancing": "round-robin" - } - }' +cf update-route EXAMPLE.COM --hostname MY-HOST -r loadbalancing ``` -Where `GUID` is the unique identifier for the route. -### Create a Route with a specific Load Balancing Algorithm +## Configure Hash-Based Routing -To create a route with a per-route `loadbalancing` option, you can use the cli command `create-route`. -For example: +<%= vars.hash_routing_version %> + +Hash-Based Routing is a load-balancing method that routes incoming requests to application instances using a hash of a specific HTTP header value. This ensures consistent routing, so requests with the same header value always go to the same instance. The per-route hash options offer detailed control over hash-based load balancing for individual routes. + +### Configure Hash-Based Routing with an App Manifest +1. In the application manifest, include a `route` definition with the following `options` attributes: + - `loadbalancing` set to `hash` + - `hash_header` set to the HTTP header name used for routing decisions + - optionally, `hash_balance` set to a float number for the balance factor used by Gorouter to [manage load imbalance](hash-based-routing.html#handling-imbalance-loads) for this particular route. + + ```yaml + --- + applications: + - name: MY-APP + routes: + - route: MY-HOST.EXAMPLE.COM + options: + loadbalancing: hash + hash_header: HASH-HEADER-NAME + hash_balance: 1.2 + ``` + + Where `MY-APP` is the name of your app, `MY-HOST.EXAMPLE.COM` is the route you want to map to your app and `HASH-HEADER-NAME` is the HTTP header name. + +1. Push the app with the manifest: + + ```console + cf push -f manifest.yml + ``` + +### Create a Route with Hash-Based Options Using the CF CLI + +To create a route with hash-specific options, you can use the CLI command `create-route`. For example: ```console -cf create-route EXAMPLE.COM --hostname MY-HOST --option loadbalancing=round-robin +cf create-route EXAMPLE.COM --hostname MY-HOST --option loadbalancing=hash --option hash_header=HASH-HEADER-NAME --option hash_balance=1.2 ``` -Alternatively, it is also possible to create a route with a per-route load balancing option via the `/v3/routes` API: +Alternatively, you can use the shorthand `-o` for `--option`. Since `hash_balance` is optional, you can omit it: ```console -cf curl /v3/routes -X POST -H "Content-type: application/json" \ - -d '{ - "host": "MY-HOST", - "path": "MY-PATH", - ... - "options": { - "loadbalancing": "round-robin" - } - }' +cf create-route EXAMPLE.COM --hostname MY-HOST -o loadbalancing=hash -o hash_header=HASH-HEADER-NAME ``` -### Map a Route to an Existing App with specific Load Balancing Algorithm +### Map a Route with Hash Options to an Existing App Using the CF CLI -To create and map a new route to an existing application with the per-route `loadbalancing` option, you can use the cli command `map-route`. +To create a new route suitable for hash-based routing and map it to an existing application, you can use the CLI command `map-route`. For example: ```console -cf map-route MY-APP EXAMPLE.COM --hostname MY-HOST --option loadbalancing=round-robin +cf map-route MY-APP EXAMPLE.COM --hostname MY-HOST -o loadbalancing=hash -o hash_header=HASH-HEADER-NAME -o hash_balance=1.2 ```

The command map-route supports the --option flag only for new routes. -To update an existing route, the command update-route must be used as described before.

+To update an existing route, see the instructions for update-route below. -### Retrieve Route Options +### Update an Existing Route with Hash Options Using the CF CLI -To read route options, you can query the route using the `route` command: +You can change an existing route that uses the default load balancing algorithm to the hash load balancing algorithm. + +For example, to change an app route's algorithm from default `round-robin` to `hash` and set `hash_header` to HASH-HEADER-NAME without a balance factor, you can run the `update-route` command: ```console -cf route EXAMPLE.COM --hostname MY-HOST +cf update-route EXAMPLE.COM --hostname MY-HOST -o loadbalancing=hash -o hash_header=HASH-HEADER-NAME ``` - The response lists the chosen `loadbalancing` algorithm option, e.g. `least-connection`: +To add a balance factor along with the previous settings, you can later run the `update-route` command, for example, with the `hash_balance` option set to `1.5`: ```console -options: {loadbalancing=least-connection} +cf update-route EXAMPLE.COM --hostname MY-HOST -o hash_balance=1.5 ``` - Alternatively, you can query the `routes` API endpoint for a route: +To unset the balance factor, run the `update-route` command with `hash_balance` set to 0. ```console -cf curl /v3/routes/?hosts=MY-HOST +cf update-route EXAMPLE.COM --hostname MY-HOST -o hash_balance=0 ``` -Where `MY-HOST` is the host attribute of the route. The response lists the chosen `loadbalancing` algorithm option as well: +Setting the balance factor to 0 indicates to Gorouter that load imbalance is accepted and all requests for a particular hash should be routed to the same instance as long as it's healthy, without redirecting to other predetermined instances. + +### Revert Hash Options Using the CF CLI + +Running the `update-route` command with the `-r` flag for the option `loadbalancing` removes all hash options from the route, returning to the default load balancing algorithm: + +```console +cf update-route EXAMPLE.COM --hostname MY-HOST -r loadbalancing +``` + +## Retrieve Route Options + +To view route options, you can query the route using the `route` command: ```console - "options": {"loadbalancing": "least-connection"} +cf route EXAMPLE.COM --hostname MY-HOST ``` -To retrieve all the routes with the corresponding options in a space of an organization, you can use the `routes` command. +The response lists the chosen `loadbalancing` algorithm option, e.g. `least-connection`: + +```console +options: {loadbalancing=least-connection} +``` + +Or `hash` with its related options: + +```console +options: {hash_balance=1.2, hash_header=HASH-HEADER-NAME, loadbalancing=hash} +``` diff --git a/deploy-apps/manifest-attributes.html.md.erb b/deploy-apps/manifest-attributes.html.md.erb index 7e63735a..1aeeb2cf 100644 --- a/deploy-apps/manifest-attributes.html.md.erb +++ b/deploy-apps/manifest-attributes.html.md.erb @@ -583,7 +583,9 @@ Under each route, you can optionally include an `options` attribute to configure Available options are: -- `loadbalancing` - defines how Gorouter distributes requests across the application backends. Valid values are `round-robin` and `least-connection`. +- `loadbalancing` - defines how Gorouter distributes requests across the application backends. Valid values are `round-robin`, `least-connection` and `hash`. +- `hash_header` - defines the header used for hash-based routing decisions. Required when `loadbalancing` is set to `hash`. +- `hash_balance` - defines the balance factor used to manage load imbalance for hash-based routing. Optional when `loadbalancing` is set to `hash`. Values in the 1.1-2.0 range provide the best balance of even distribution and performance. Omitting `hash_balance` or setting it explicitly to 0 indicates that the load situation will not be considered. For example: @@ -597,6 +599,11 @@ For example: - route: example2.com options: loadbalancing: least-connection + - route: example3.com + options: + loadbalancing: hash + hash_header: Hash-Relevant-Header + hash_balance: 1.25 ``` #### Manifest attributes diff --git a/hash-based-routing.html.md.erb b/hash-based-routing.html.md.erb new file mode 100644 index 00000000..cc67cce7 --- /dev/null +++ b/hash-based-routing.html.md.erb @@ -0,0 +1,58 @@ +--- +title: Hash-Based Routing +owner: CF for VMs Networking +--- + +## Purpose + +Hash-Based Routing is a load-balancing algorithm that distributes incoming requests to application instances based on a hash of a specific HTTP header value, e.g., `X-Resource-ID` or `Tenant-ID`. This ensures consistent routing behavior where requests containing the same header value are always directed to the same instance. + +## Prerequisites + +To use Hash-Based Routing, ensure that + +- your <%= vars.app_runtime_abbr %> deployment is running version <%= vars.hash_routing_version %> or later. +- platform operators activate the feature so that the CF feature flag `hash_based_routing` is set to `true`. See [Feature Flags](https://docs.cloudfoundry.org/adminguide/listing-feature-flags.html#flags) for more details. + +## Key Features + +- **Configurable via Per-Route Options**: Hash-Based load-balancing setup through application route options +- **Configurable Hash Header**: The HTTP header to consider for Hash-Based Routing is configurable for each route +- **Session Affinity Precedence**: Session affinity (sticky sessions) is prioritized over Hash-Based Routing +- **No availability zones preference**: The global properties `locallyOptimistic` and `localAvailabilityZone` are ignored when using Hash-Based Routing +- **Consistent Hashing**: Implementation of the Maglev Algorithm (see [Maglev: A Fast and Reliable Software Network Load Balancer](https://storage.googleapis.com/gweb-research2023-media/pubtools/2904.pdf) for details) +- **Minimal Rehashing**: Usage of the Maglev Algorithm to map application instances by hash ensures that hash positions are shifted to other instances as little as possible when application instances are added or removed +- **Handling imbalanced loads**: Detection and mitigation of imbalanced load on single instances prevents overloading while keeping instances for a particular hash at a minimum + +Hash-Based Routing implements a clear precedence hierarchy: + +1. **Sticky Sessions**: First checks if a sticky session is used and the sticky session endpoint is available +2. **Hash-Based Routing**: Calculates the hash of the specified HTTP header value and routes the requests to a pre-determined application instance +3. **Default Load Balancing**: Falls back to default load balancing if the header configured in the application route for Hash-Based Routing is absent in the request + + +## Hash-Based Routing vs. Session Affinity + +While Session Affinity provides a way to keep session consistency, it works at the session level and can cause scalability issues. Heavy users might be routed to the same instance and stay pinned to it through session affinity, which could lead to overload. Session Affinity usually depends on session cookies (see more details in the [Session Affinity](https://docs.cloudfoundry.org/concepts/http-routing.html#-session-affinity) documentation), which requires implementation effort for web applications. + +In contrast, Hash-Based Routing offers a more scalable and flexible approach by consistently distributing requests based on a hash of any configurable HTTP header value, not just session identifiers. This allows for load distribution based on tenant IDs, resource IDs, or other business-relevant identifiers. Configurable load imbalance handling enables spillover to other instances (see [Handling imbalanced loads](#handling-imbalance-loads)). + +Unlike Session Affinity, Hash-Based Routing requires no code changes to the application. + +## Handling imbalanced loads + +Hash-Based Routing includes mechanisms to detect imbalanced load across application instances. An imbalanced load occurs when certain hashes receive more traffic, such as when a specific tenant generates many requests, resulting in heavier use of their mapped instances than others’. Additionally, multiple high-traffic targets might be assigned to the same instance. + +To prevent overloading specific instances while others remain underutilized, the acceptable threshold for load imbalance can be configured using the `hash_balance` property. +This setting determines whether an instance is handling more traffic than its fair share based on the average load across all instances, measured by the number of in-flight requests. For example, with a balance factor of 1.25, no single instance should handle more than 125% of the average number of in-flight requests across all instances managed by the current router. When this threshold is exceeded, the router redirects subsequent requests to other, less-loaded instances. +Values of the balance factor in the 1.1-2.0 range offer a good balance between even distribution and performance. However, optimal values depend on the application's traffic patterns and load characteristics. + +This approach ensures that a minimum number of instances process requests for a particular hash while preventing any single instance from becoming overloaded. + +## Minimal Rehashing + +The Maglev algorithm used in Hash-Based Routing minimizes rehashing when application instances are added or removed. When a new instance is added, only a small subset of hashes is remapped to it, while most continue to route to their original instances. Similarly, when an instance is removed, only the hashes mapped to that instance are reassigned. This design minimizes disruption and maintains consistent routing behavior as the application scales up or down. + +## Retries in Hash-Based Routing + +For idempotent requests, Hash-Based Routing supports a retry mechanism. If a request fails due to a network error or a 5xx response from the application instance, the router retries the request with a different, predetermined application instance. The next entry in the Maglev lookup table determines this instance. This approach aligns with the approach to handling imbalances. It ensures that the retry mechanism adhere to the principles of Hash-Based Routing while providing resilience against transient failures. From 6eb97da31e221c4e32d3e95d9cc1a62a886ee4a8 Mon Sep 17 00:00:00 2001 From: App Platform Runtime Working Group CI Bot Date: Fri, 27 Mar 2026 15:47:24 +0100 Subject: [PATCH 2/2] Address review and improve structure and wording --- custom-per-route-options.html.md.erb | 97 ++----------- deploy-apps/manifest-attributes.html.md.erb | 2 +- hash-based-routing.html.md.erb | 148 +++++++++++++++++--- routing-index.html.md.erb | 2 + 4 files changed, 136 insertions(+), 113 deletions(-) diff --git a/custom-per-route-options.html.md.erb b/custom-per-route-options.html.md.erb index 7c715472..56381ee8 100644 --- a/custom-per-route-options.html.md.erb +++ b/custom-per-route-options.html.md.erb @@ -12,7 +12,7 @@ Gorouter supports the following per-route options, described in the sections bel - `loadbalancing`: Configures the load balancing algorithm used by Gorouter for this particular route. <%= vars.per_route_lb_version %> - Settings: `round-robin`, `least-connection`, `hash`. -- `hash_header`: Defines the header Gorouter uses for routing decisions on this route. Required when `loadbalancing` is set to `hash`. Cannot be used with other load balancing algorithms. <%= vars.hash_routing_version %> +- `hash_header`: Defines the HTTP header used for hash-based routing decisions. Required when `loadbalancing` is set to `hash`. Cannot be used with other load balancing algorithms. <%= vars.hash_routing_version %> - `hash_balance`: Sets the float number for the balance factor used by Gorouter to manage load imbalance applying the hash-based routing for this route. Optional when `loadbalancing` is `hash`. Cannot be used with other algorithms. <%= vars.hash_routing_version %> ## Configure Gorouter's Load Balancing Algorithm @@ -21,11 +21,11 @@ Gorouter supports the following per-route options, described in the sections bel The per-route option `loadbalancing` allows configuring the load balancing algorithm, which defines how the load is distributed between Gorouters and backends. -This option supports three settings for load balancing: +This option supports the following settings for load balancing: - `round-robin` distributes the load evenly across all available backends - `least-connection` directs traffic to the backend with the fewest active connections at any given time, optimizing resource utilization -- `hash` distributes requests based on a hash of a specific HTTP header, ensuring requests with the same header are consistently directed to the same backend. <%= vars.hash_routing_version %> +- `hash` distributes requests based on a specific HTTP header value, ensuring requests with the same header value are consistently directed to the same backend. See [Hash-Based Routing](hash-based-routing.html) for details. <%= vars.hash_routing_version %> ### Configure Load Balancing using an App Manifest @@ -84,7 +84,8 @@ For example, to change an app route's algorithm from `least-connection` to `roun ```console cf update-route EXAMPLE.COM --hostname MY-HOST --option loadbalancing=round-robin ``` -### Remove the Specific Load Balancing Algorithm Using the CF CLI + +### Remove the Specific Load Balancing Algorithm Using the CF CLI To remove the `loadbalancing` option from an existing route, run: @@ -96,93 +97,11 @@ cf update-route EXAMPLE.COM --hostname MY-HOST -r loadbalancing <%= vars.hash_routing_version %> -Hash-Based Routing is a load-balancing method that routes incoming requests to application instances using a hash of a specific HTTP header value. This ensures consistent routing, so requests with the same header value always go to the same instance. The per-route hash options offer detailed control over hash-based load balancing for individual routes. - -### Configure Hash-Based Routing with an App Manifest -1. In the application manifest, include a `route` definition with the following `options` attributes: - - `loadbalancing` set to `hash` - - `hash_header` set to the HTTP header name used for routing decisions - - optionally, `hash_balance` set to a float number for the balance factor used by Gorouter to [manage load imbalance](hash-based-routing.html#handling-imbalance-loads) for this particular route. - - ```yaml - --- - applications: - - name: MY-APP - routes: - - route: MY-HOST.EXAMPLE.COM - options: - loadbalancing: hash - hash_header: HASH-HEADER-NAME - hash_balance: 1.2 - ``` - - Where `MY-APP` is the name of your app, `MY-HOST.EXAMPLE.COM` is the route you want to map to your app and `HASH-HEADER-NAME` is the HTTP header name. - -1. Push the app with the manifest: - - ```console - cf push -f manifest.yml - ``` - -### Create a Route with Hash-Based Options Using the CF CLI - -To create a route with hash-specific options, you can use the CLI command `create-route`. For example: - -```console -cf create-route EXAMPLE.COM --hostname MY-HOST --option loadbalancing=hash --option hash_header=HASH-HEADER-NAME --option hash_balance=1.2 -``` - -Alternatively, you can use the shorthand `-o` for `--option`. Since `hash_balance` is optional, you can omit it: - -```console -cf create-route EXAMPLE.COM --hostname MY-HOST -o loadbalancing=hash -o hash_header=HASH-HEADER-NAME -``` - -### Map a Route with Hash Options to an Existing App Using the CF CLI +Hash-Based Routing is a load-balancing method that routes incoming requests to application instances based on a specific HTTP header value. This ensures consistent routing, so requests with the same header value are always routed to the same instance. -To create a new route suitable for hash-based routing and map it to an existing application, you can use the CLI command `map-route`. +For details on Hash-Based Routing concepts, features, and how it compares to Session Affinity, see [Hash-Based Routing](hash-based-routing.html). -For example: - -```console -cf map-route MY-APP EXAMPLE.COM --hostname MY-HOST -o loadbalancing=hash -o hash_header=HASH-HEADER-NAME -o hash_balance=1.2 -``` - -

-The command map-route supports the --option flag only for new routes. -To update an existing route, see the instructions for update-route below. - -### Update an Existing Route with Hash Options Using the CF CLI - -You can change an existing route that uses the default load balancing algorithm to the hash load balancing algorithm. - -For example, to change an app route's algorithm from default `round-robin` to `hash` and set `hash_header` to HASH-HEADER-NAME without a balance factor, you can run the `update-route` command: - -```console -cf update-route EXAMPLE.COM --hostname MY-HOST -o loadbalancing=hash -o hash_header=HASH-HEADER-NAME -``` - -To add a balance factor along with the previous settings, you can later run the `update-route` command, for example, with the `hash_balance` option set to `1.5`: - -```console -cf update-route EXAMPLE.COM --hostname MY-HOST -o hash_balance=1.5 -``` - -To unset the balance factor, run the `update-route` command with `hash_balance` set to 0. - -```console -cf update-route EXAMPLE.COM --hostname MY-HOST -o hash_balance=0 -``` - -Setting the balance factor to 0 indicates to Gorouter that load imbalance is accepted and all requests for a particular hash should be routed to the same instance as long as it's healthy, without redirecting to other predetermined instances. - -### Revert Hash Options Using the CF CLI - -Running the `update-route` command with the `-r` flag for the option `loadbalancing` removes all hash options from the route, returning to the default load balancing algorithm: - -```console -cf update-route EXAMPLE.COM --hostname MY-HOST -r loadbalancing -``` +For instructions on configuring hash-based routing using the app manifest or the CF CLI, see [Configure Hash-Based Routing](hash-based-routing.html#configure). ## Retrieve Route Options diff --git a/deploy-apps/manifest-attributes.html.md.erb b/deploy-apps/manifest-attributes.html.md.erb index 1aeeb2cf..e57e1d50 100644 --- a/deploy-apps/manifest-attributes.html.md.erb +++ b/deploy-apps/manifest-attributes.html.md.erb @@ -584,7 +584,7 @@ Under each route, you can optionally include an `options` attribute to configure Available options are: - `loadbalancing` - defines how Gorouter distributes requests across the application backends. Valid values are `round-robin`, `least-connection` and `hash`. -- `hash_header` - defines the header used for hash-based routing decisions. Required when `loadbalancing` is set to `hash`. +- `hash_header` - defines the HTTP header used for hash-based routing decisions. Required when `loadbalancing` is set to `hash`. - `hash_balance` - defines the balance factor used to manage load imbalance for hash-based routing. Optional when `loadbalancing` is set to `hash`. Values in the 1.1-2.0 range provide the best balance of even distribution and performance. Omitting `hash_balance` or setting it explicitly to 0 indicates that the load situation will not be considered. For example: diff --git a/hash-based-routing.html.md.erb b/hash-based-routing.html.md.erb index cc67cce7..50a02e40 100644 --- a/hash-based-routing.html.md.erb +++ b/hash-based-routing.html.md.erb @@ -5,49 +5,151 @@ owner: CF for VMs Networking ## Purpose -Hash-Based Routing is a load-balancing algorithm that distributes incoming requests to application instances based on a hash of a specific HTTP header value, e.g., `X-Resource-ID` or `Tenant-ID`. This ensures consistent routing behavior where requests containing the same header value are always directed to the same instance. +Hash-Based Routing is a load-balancing algorithm that distributes incoming requests to application instances based on the value of a configured HTTP header. Typically, this is a header that identifies a user, resource, or tenant, such as `X-Resource-ID` or `Tenant-ID`. This ensures consistent routing behavior where requests containing the same header value are always directed to the same instance. ## Prerequisites To use Hash-Based Routing, ensure that -- your <%= vars.app_runtime_abbr %> deployment is running version <%= vars.hash_routing_version %> or later. +- your <%= vars.app_runtime_abbr %> deployment meets the minimum version requirements. <%= vars.hash_routing_version %> - platform operators activate the feature so that the CF feature flag `hash_based_routing` is set to `true`. See [Feature Flags](https://docs.cloudfoundry.org/adminguide/listing-feature-flags.html#flags) for more details. ## Key Features -- **Configurable via Per-Route Options**: Hash-Based load-balancing setup through application route options -- **Configurable Hash Header**: The HTTP header to consider for Hash-Based Routing is configurable for each route -- **Session Affinity Precedence**: Session affinity (sticky sessions) is prioritized over Hash-Based Routing -- **No availability zones preference**: The global properties `locallyOptimistic` and `localAvailabilityZone` are ignored when using Hash-Based Routing -- **Consistent Hashing**: Implementation of the Maglev Algorithm (see [Maglev: A Fast and Reliable Software Network Load Balancer](https://storage.googleapis.com/gweb-research2023-media/pubtools/2904.pdf) for details) -- **Minimal Rehashing**: Usage of the Maglev Algorithm to map application instances by hash ensures that hash positions are shifted to other instances as little as possible when application instances are added or removed -- **Handling imbalanced loads**: Detection and mitigation of imbalanced load on single instances prevents overloading while keeping instances for a particular hash at a minimum +- **Consistent Hashing**: Uses the Maglev Algorithm to determine instance assignments (see [Maglev: A Fast and Reliable Software Network Load Balancer](https://storage.googleapis.com/gweb-research2023-media/pubtools/2904.pdf) for details) +- **Minimal Rehashing**: The Maglev Algorithm ensures that the mapping of header values to application instances remains as consistent as possible when instances are added or removed +- **Configurable Hash Header**: The HTTP header used for Hash-Based Routing is configurable for each route +- **Configurable via Per-Route Options**: Hash-Based load balancing is set up through [per-route options](custom-per-route-options.html) +- **Handling imbalanced loads**: Detection and mitigation of imbalanced load on single instances prevents overloading while keeping the number of instances for a particular hash at a minimum +- **Session Affinity Precedence**: Session Affinity (sticky sessions) is prioritized over Hash-Based Routing +- **No availability zone preference**: The global properties `locallyOptimistic` and `localAvailabilityZone` are ignored when using Hash-Based Routing -Hash-Based Routing implements a clear precedence hierarchy: +Hash-Based Routing implements the following precedence hierarchy: -1. **Sticky Sessions**: First checks if a sticky session is used and the sticky session endpoint is available -2. **Hash-Based Routing**: Calculates the hash of the specified HTTP header value and routes the requests to a pre-determined application instance -3. **Default Load Balancing**: Falls back to default load balancing if the header configured in the application route for Hash-Based Routing is absent in the request +1. **Session Affinity**: If a sticky session cookie is present and the target instance is available, the request is routed to that instance +2. **Hash-Based Routing**: If a hash header is configured for the route and present in the request, Gorouter routes to the instance determined by the header value's hash +3. **Default Load Balancing**: If the hash header is absent from the request, Gorouter falls back to the platform's default load balancing algorithm +## Configure Hash-Based Routing -## Hash-Based Routing vs. Session Affinity +<%= vars.hash_routing_version %> + +The per-route hash options offer detailed control over hash-based load balancing for individual routes. For a full list of per-route options, see [Configuring per-route options](custom-per-route-options.html). + +### Configure Hash-Based Routing with an App Manifest + +1. In the application manifest, include a `route` definition with the following `options` attributes: + - `loadbalancing` set to `hash` + - `hash_header` set to the HTTP header name used for routing decisions + - optionally, `hash_balance` set to a float number for the balance factor used by Gorouter to [manage load imbalance](#handling-imbalance-loads) for this particular route. + + ```yaml + --- + applications: + - name: MY-APP + routes: + - route: MY-HOST.EXAMPLE.COM + options: + loadbalancing: hash + hash_header: HASH-HEADER-NAME + hash_balance: 1.2 + ``` + + Where `MY-APP` is the name of your app, `MY-HOST.EXAMPLE.COM` is the route you want to map to your app and `HASH-HEADER-NAME` is the HTTP header name. + +1. Push the app with the manifest: + + ```console + cf push -f manifest.yml + ``` + +### Create a Route with Hash-Based Options Using the CF CLI + +To create a route with hash-specific options, you can use the CLI command `create-route`. For example: + +```console +cf create-route EXAMPLE.COM --hostname MY-HOST --option loadbalancing=hash --option hash_header=HASH-HEADER-NAME --option hash_balance=1.2 +``` + +Alternatively, you can use the shorthand `-o` for `--option`. Since `hash_balance` is optional, you can omit it: + +```console +cf create-route EXAMPLE.COM --hostname MY-HOST -o loadbalancing=hash -o hash_header=HASH-HEADER-NAME +``` + +### Map a Route with Hash Options to an Existing App Using the CF CLI + +To create a new route suitable for hash-based routing and map it to an existing application, you can use the CLI command `map-route`. + +For example: + +```console +cf map-route MY-APP EXAMPLE.COM --hostname MY-HOST -o loadbalancing=hash -o hash_header=HASH-HEADER-NAME -o hash_balance=1.2 +``` + +

+The command map-route supports the --option flag only for new routes. +To update an existing route, see the instructions for update-route below.

-While Session Affinity provides a way to keep session consistency, it works at the session level and can cause scalability issues. Heavy users might be routed to the same instance and stay pinned to it through session affinity, which could lead to overload. Session Affinity usually depends on session cookies (see more details in the [Session Affinity](https://docs.cloudfoundry.org/concepts/http-routing.html#-session-affinity) documentation), which requires implementation effort for web applications. +### Update an Existing Route with Hash Options Using the CF CLI + +You can change an existing route that uses the default load balancing algorithm to the hash load balancing algorithm. + +For example, to change an app route's algorithm from default `round-robin` to `hash` and set `hash_header` to HASH-HEADER-NAME without a balance factor, you can run the `update-route` command: + +```console +cf update-route EXAMPLE.COM --hostname MY-HOST -o loadbalancing=hash -o hash_header=HASH-HEADER-NAME +``` + +To add a balance factor along with the previous settings, you can later run the `update-route` command, for example, with the `hash_balance` option set to `1.5`: + +```console +cf update-route EXAMPLE.COM --hostname MY-HOST -o hash_balance=1.5 +``` + +To remove the balance factor, run the `update-route` command with the `-r` flag: + +```console +cf update-route EXAMPLE.COM --hostname MY-HOST -r hash_balance +``` + +Removing the balance factor indicates to Gorouter that load imbalance is accepted and all requests for a particular hash should be routed to the same instance as long as it is healthy, without redirecting to other predetermined instances. + +### Revert Hash Options Using the CF CLI + +Running the `update-route` command with the `-r` flag for the option `loadbalancing` removes all hash options from the route, returning to the default load balancing algorithm: + +```console +cf update-route EXAMPLE.COM --hostname MY-HOST -r loadbalancing +``` + +### Retrieve Hash-Based Route Options + +To view route options, you can query the route using the `route` command: + +```console +cf route EXAMPLE.COM --hostname MY-HOST +``` + +The response lists the chosen options, for example: + +```console +options: {hash_balance=1.2, hash_header=HASH-HEADER-NAME, loadbalancing=hash} +``` + +## Hash-Based Routing vs. Session Affinity -In contrast, Hash-Based Routing offers a more scalable and flexible approach by consistently distributing requests based on a hash of any configurable HTTP header value, not just session identifiers. This allows for load distribution based on tenant IDs, resource IDs, or other business-relevant identifiers. Configurable load imbalance handling enables spillover to other instances (see [Handling imbalanced loads](#handling-imbalance-loads)). +Session Affinity works at the session level and relies on session cookies (see [Session Affinity](https://docs.cloudfoundry.org/concepts/http-routing.html#-session-affinity) for details). Heavy users can be pinned to a single instance, leading to uneven load distribution. Implementing Session Affinity in web applications requires handling session cookies. -Unlike Session Affinity, Hash-Based Routing requires no code changes to the application. +Hash-Based Routing provides a more scalable approach by consistently routing requests based on any configurable HTTP header value, not just session identifiers. This allows distribution based on tenant IDs, resource IDs, or other business-relevant identifiers. When a single instance receives disproportionate load, requests can spill over to other predetermined instances (see [Handling imbalanced loads](#handling-imbalance-loads)). Unlike Session Affinity, Hash-Based Routing requires no code changes to the application. ## Handling imbalanced loads -Hash-Based Routing includes mechanisms to detect imbalanced load across application instances. An imbalanced load occurs when certain hashes receive more traffic, such as when a specific tenant generates many requests, resulting in heavier use of their mapped instances than others’. Additionally, multiple high-traffic targets might be assigned to the same instance. +Imbalanced load can occur when certain header values receive disproportionately more traffic. For example, a particular tenant may generate a large number of requests, resulting in heavier use of the instance assigned to that tenant's hash. Additionally, multiple high-traffic header values might map to the same instance. -To prevent overloading specific instances while others remain underutilized, the acceptable threshold for load imbalance can be configured using the `hash_balance` property. -This setting determines whether an instance is handling more traffic than its fair share based on the average load across all instances, measured by the number of in-flight requests. For example, with a balance factor of 1.25, no single instance should handle more than 125% of the average number of in-flight requests across all instances managed by the current router. When this threshold is exceeded, the router redirects subsequent requests to other, less-loaded instances. -Values of the balance factor in the 1.1-2.0 range offer a good balance between even distribution and performance. However, optimal values depend on the application's traffic patterns and load characteristics. +To prevent overloading specific instances while others remain underutilized, the acceptable threshold for load imbalance can be configured using the `hash_balance` route option. This factor determines whether an instance is handling more traffic than its fair share compared to the average load across all instances, measured by the number of in-flight requests. For example, with a balance factor of 1.25, no single instance should handle more than 125% of the average number of in-flight requests across all instances managed by the current Gorouter. When this threshold is exceeded, the router redirects subsequent requests to other predetermined, less-loaded instances. -This approach ensures that a minimum number of instances process requests for a particular hash while preventing any single instance from becoming overloaded. +Values in the 1.1–2.0 range offer a good balance between even distribution and performance. Optimal values depend on the application's traffic patterns. Omitting `hash_balance` or setting it to 0 means load imbalance is accepted and all requests for a particular hash are routed to the same instance as long as it is healthy. ## Minimal Rehashing @@ -55,4 +157,4 @@ The Maglev algorithm used in Hash-Based Routing minimizes rehashing when applica ## Retries in Hash-Based Routing -For idempotent requests, Hash-Based Routing supports a retry mechanism. If a request fails due to a network error or a 5xx response from the application instance, the router retries the request with a different, predetermined application instance. The next entry in the Maglev lookup table determines this instance. This approach aligns with the approach to handling imbalances. It ensures that the retry mechanism adhere to the principles of Hash-Based Routing while providing resilience against transient failures. +For idempotent requests, Hash-Based Routing supports a retry mechanism. When a request fails due to a network error or a 5xx response from the application instance, the router retries the request with a different, predetermined application instance. The next entry in the Maglev lookup table determines this instance — the same approach used for handling imbalanced loads. This ensures that retries adhere to the principles of Hash-Based Routing while providing resilience against temporary failures such as instance restarts or network interruptions. diff --git a/routing-index.html.md.erb b/routing-index.html.md.erb index 37e392d9..10fbad60 100644 --- a/routing-index.html.md.erb +++ b/routing-index.html.md.erb @@ -9,6 +9,8 @@ These topics contain information about configuring routes and domains: * [Configuring per-route options](custom-per-route-options.html) +* [Hash-Based Routing](hash-based-routing.html) + * [Configuring <%= vars.app_runtime_abbr %> to route traffic to apps on custom ports](custom-ports.html) * [Routing HTTP/2 and gRPC traffic to apps](http2-protocol.html)