Skip to content

Commit 325beaa

Browse files
zhaohuabingzhaohuabing
committed
fix: derive the protocol of active health checks from the backend
Envoy sends HTTP health check requests with a codec that is fixed when the health checker is created, and health check connections are created outside of the connection pool, so they don't inherit the ALPN protocols that the pool derives from the backend protocol settings. Envoy Gateway configures neither, so health checks are sent as HTTP/1.1 and fail against a backend that speaks HTTP/2 on its health check endpoint, marking all of its endpoints unhealthy. Derive both from the backend protocol: `codec_client_type` is set to HTTP2 for backends that use HTTP/2 and to HTTP1 otherwise, and for backends that use TLS, `tls_options.alpn_protocols` is pinned to the protocol matching the codec, so the protocol negotiated during the handshake is the one Envoy uses to send the request. The ALPN isn't pinned when the backend TLS settings ask for a protocol that the codec can't speak, such as `istio`, since overriding it would break the handshake. gRPC health checks are always sent over HTTP/2, so TLS backends get `h2`. Add `healthCheck.active.http.version` to select the version explicitly, for backends whose health check endpoint uses a different protocol than their application endpoint. Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com> Signed-off-by: Huabing (Robin) Zhao <huabing@tetrate.io>
1 parent 2b40195 commit 325beaa

29 files changed

Lines changed: 2487 additions & 4 deletions

‎api/v1alpha1/healthcheck_types.go‎

Lines changed: 46 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -197,6 +197,37 @@ type HTTPActiveHealthChecker struct {
197197
// +kubebuilder:validation:MinLength=1
198198
// +kubebuilder:validation:MaxLength=1024
199199
Path string `json:"path" yaml:"path"`
200+
// Version defines the HTTP protocol version used to send active health check
201+
// requests to the backend.
202+
//
203+
// Envoy sends health check requests over a dedicated connection, using a fixed
204+
// protocol version that is not negotiated per request. If that version does not
205+
// match the protocol the backend speaks, every health check fails and all
206+
// endpoints of the backend are marked unhealthy.
207+
//
208+
// Defaults to Auto, which resolves the version from the effective upstream protocol
209+
// of the backend: HTTP2 if the backend is configured to use HTTP/2, through a
210+
// `kubernetes.io/h2c`, `gateway.envoyproxy.io/h2c` or `grpc` appProtocol, or by being
211+
// the backend of a GRPCRoute, and HTTP1 otherwise.
212+
//
213+
// How the resolved version is applied depends on the backend:
214+
//
215+
// - For plaintext backends, the resolved version is the version used, since no
216+
// protocol is negotiated on the connection.
217+
// - For backends that use TLS, the resolved version is also the version used, and
218+
// the ALPN offered on health check connections is constrained to the matching
219+
// protocol so that the handshake can't settle on a different one. This may
220+
// change to the protocol negotiated during the handshake once
221+
// https://github.com/envoyproxy/envoy/issues/46848 lands in Envoy.
222+
//
223+
// Set this field explicitly when the health check endpoint and the application
224+
// endpoint of the backend use different protocols, for example when the backend
225+
// serves HTTP/1.1 traffic but only accepts HTTP/2 health check requests. An
226+
// explicitly configured version is always the version used.
227+
//
228+
// +kubebuilder:default=Auto
229+
// +optional
230+
Version *HTTPHealthCheckVersion `json:"version,omitempty" yaml:"version,omitempty"`
200231
// Method defines the HTTP method used for health checking.
201232
// Defaults to GET
202233
// +kubebuilder:validation:MaxLength=16
@@ -220,6 +251,21 @@ type HTTPActiveHealthChecker struct {
220251
RequestBody *ActiveHealthCheckPayload `json:"requestBody,omitempty" yaml:"requestBody,omitempty"`
221252
}
222253

254+
// HTTPHealthCheckVersion specifies the HTTP protocol version used to send active
255+
// HTTP health check requests to the backend.
256+
// +kubebuilder:validation:Enum=Auto;HTTP1;HTTP2
257+
type HTTPHealthCheckVersion string
258+
259+
const (
260+
// HTTPHealthCheckVersionAuto derives the health check protocol version from the
261+
// effective upstream protocol of the backend.
262+
HTTPHealthCheckVersionAuto HTTPHealthCheckVersion = "Auto"
263+
// HTTPHealthCheckVersionHTTP1 sends health check requests using HTTP/1.1.
264+
HTTPHealthCheckVersionHTTP1 HTTPHealthCheckVersion = "HTTP1"
265+
// HTTPHealthCheckVersionHTTP2 sends health check requests using HTTP/2.
266+
HTTPHealthCheckVersionHTTP2 HTTPHealthCheckVersion = "HTTP2"
267+
)
268+
223269
// TCPActiveHealthChecker defines the settings of tcp health check.
224270
type TCPActiveHealthChecker struct {
225271
// Send defines the request payload.

‎api/v1alpha1/zz_generated.deepcopy.go‎

Lines changed: 5 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎charts/gateway-crds-helm/templates/generated/gateway.envoyproxy.io_backendtrafficpolicies.yaml‎

Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -801,6 +801,41 @@ spec:
801801
minimum: 100
802802
type: integer
803803
type: array
804+
version:
805+
default: Auto
806+
description: |-
807+
Version defines the HTTP protocol version used to send active health check
808+
requests to the backend.
809+
810+
Envoy sends health check requests over a dedicated connection, using a fixed
811+
protocol version that is not negotiated per request. If that version does not
812+
match the protocol the backend speaks, every health check fails and all
813+
endpoints of the backend are marked unhealthy.
814+
815+
Defaults to Auto, which resolves the version from the effective upstream protocol
816+
of the backend: HTTP2 if the backend is configured to use HTTP/2, through a
817+
`kubernetes.io/h2c`, `gateway.envoyproxy.io/h2c` or `grpc` appProtocol, or by being
818+
the backend of a GRPCRoute, and HTTP1 otherwise.
819+
820+
How the resolved version is applied depends on the backend:
821+
822+
- For plaintext backends, the resolved version is the version used, since no
823+
protocol is negotiated on the connection.
824+
- For backends that use TLS, the resolved version is also the version used, and
825+
the ALPN offered on health check connections is constrained to the matching
826+
protocol so that the handshake can't settle on a different one. This may
827+
change to the protocol negotiated during the handshake once
828+
https://github.com/envoyproxy/envoy/issues/46848 lands in Envoy.
829+
830+
Set this field explicitly when the health check endpoint and the application
831+
endpoint of the backend use different protocols, for example when the backend
832+
serves HTTP/1.1 traffic but only accepts HTTP/2 health check requests. An
833+
explicitly configured version is always the version used.
834+
enum:
835+
- Auto
836+
- HTTP1
837+
- HTTP2
838+
type: string
804839
required:
805840
- path
806841
type: object

‎charts/gateway-crds-helm/templates/generated/gateway.envoyproxy.io_envoyextensionpolicies.yaml‎

Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -704,6 +704,41 @@ spec:
704704
minimum: 100
705705
type: integer
706706
type: array
707+
version:
708+
default: Auto
709+
description: |-
710+
Version defines the HTTP protocol version used to send active health check
711+
requests to the backend.
712+
713+
Envoy sends health check requests over a dedicated connection, using a fixed
714+
protocol version that is not negotiated per request. If that version does not
715+
match the protocol the backend speaks, every health check fails and all
716+
endpoints of the backend are marked unhealthy.
717+
718+
Defaults to Auto, which resolves the version from the effective upstream protocol
719+
of the backend: HTTP2 if the backend is configured to use HTTP/2, through a
720+
`kubernetes.io/h2c`, `gateway.envoyproxy.io/h2c` or `grpc` appProtocol, or by being
721+
the backend of a GRPCRoute, and HTTP1 otherwise.
722+
723+
How the resolved version is applied depends on the backend:
724+
725+
- For plaintext backends, the resolved version is the version used, since no
726+
protocol is negotiated on the connection.
727+
- For backends that use TLS, the resolved version is also the version used, and
728+
the ALPN offered on health check connections is constrained to the matching
729+
protocol so that the handshake can't settle on a different one. This may
730+
change to the protocol negotiated during the handshake once
731+
https://github.com/envoyproxy/envoy/issues/46848 lands in Envoy.
732+
733+
Set this field explicitly when the health check endpoint and the application
734+
endpoint of the backend use different protocols, for example when the backend
735+
serves HTTP/1.1 traffic but only accepts HTTP/2 health check requests. An
736+
explicitly configured version is always the version used.
737+
enum:
738+
- Auto
739+
- HTTP1
740+
- HTTP2
741+
type: string
707742
required:
708743
- path
709744
type: object

0 commit comments

Comments
 (0)