docs: document Kubernetes service registry incompat with K8s 1.32

Fixes #9980

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>

Author: smira

pkg/machinery/config/schemas/config.schema.json

@@ -1753,9 +1753,9 @@
         "kubernetes": {
           "$ref": "#/$defs/v1alpha1.RegistryKubernetesConfig",
           "title": "kubernetes",
-          "description": "Kubernetes registry uses Kubernetes API server to discover cluster members and stores additional information\nas annotations on the Node resources.\n",
-          "markdownDescription": "Kubernetes registry uses Kubernetes API server to discover cluster members and stores additional information\nas annotations on the Node resources.",
-          "x-intellij-html-description": "\u003cp\u003eKubernetes registry uses Kubernetes API server to discover cluster members and stores additional information\nas annotations on the Node resources.\u003c/p\u003e\n"
+          "description": "Kubernetes registry uses Kubernetes API server to discover cluster members and stores additional information\nas annotations on the Node resources.\n\nThis feature is deprecated as it is not compatible with Kubernetes 1.32+.\nSee https://github.com/siderolabs/talos/issues/9980 for more information.\n",
+          "markdownDescription": "Kubernetes registry uses Kubernetes API server to discover cluster members and stores additional information\nas annotations on the Node resources.\n\nThis feature is deprecated as it is not compatible with Kubernetes 1.32+.\nSee https://github.com/siderolabs/talos/issues/9980 for more information.",
+          "x-intellij-html-description": "\u003cp\u003eKubernetes registry uses Kubernetes API server to discover cluster members and stores additional information\nas annotations on the Node resources.\u003c/p\u003e\n\n\u003cp\u003eThis feature is deprecated as it is not compatible with Kubernetes 1.32+.\nSee \u003ca href=\"https://github.com/siderolabs/talos/issues/9980\" target=\"_blank\"\u003ehttps://github.com/siderolabs/talos/issues/9980\u003c/a\u003e for more information.\u003c/p\u003e\n"
         },
         "service": {
           "$ref": "#/$defs/v1alpha1.RegistryServiceConfig",

pkg/machinery/config/types/v1alpha1/v1alpha1_types.go

@@ -2413,6 +2413,9 @@ type DiscoveryRegistriesConfig struct {
 	// description: |
 	//   Kubernetes registry uses Kubernetes API server to discover cluster members and stores additional information
 	//   as annotations on the Node resources.
+	//
+	//   This feature is deprecated as it is not compatible with Kubernetes 1.32+.
+	//   See https://github.com/siderolabs/talos/issues/9980 for more information.
 	RegistryKubernetes RegistryKubernetesConfig `yaml:"kubernetes"`
 	// description: |
 	//   Service registry is using an external service to push and pull information about cluster members.

pkg/machinery/config/types/v1alpha1/v1alpha1_types_doc.go

@@ -3830,7 +3830,7 @@ DiscoveryRegistriesConfig struct configures cluster membership discovery.
 
 | Field | Type | Description | Value(s) |
 |-------|------|-------------|----------|
-|`kubernetes` |<a href="#Config.cluster.discovery.registries.kubernetes">RegistryKubernetesConfig</a> |<details><summary>Kubernetes registry uses Kubernetes API server to discover cluster members and stores additional information</summary>as annotations on the Node resources.</details>  | |
+|`kubernetes` |<a href="#Config.cluster.discovery.registries.kubernetes">RegistryKubernetesConfig</a> |<details><summary>Kubernetes registry uses Kubernetes API server to discover cluster members and stores additional information</summary>as annotations on the Node resources.<br /><br />This feature is deprecated as it is not compatible with Kubernetes 1.32+.<br />See https://github.com/siderolabs/talos/issues/9980 for more information.</details>  | |
 |`service` |<a href="#Config.cluster.discovery.registries.service">RegistryServiceConfig</a> |Service registry is using an external service to push and pull information about cluster members.  | |
 
 

website/content/v1.10/reference/configuration/v1alpha1/config.md

@@ -1753,9 +1753,9 @@
         "kubernetes": {
           "$ref": "#/$defs/v1alpha1.RegistryKubernetesConfig",
           "title": "kubernetes",
-          "description": "Kubernetes registry uses Kubernetes API server to discover cluster members and stores additional information\nas annotations on the Node resources.\n",
-          "markdownDescription": "Kubernetes registry uses Kubernetes API server to discover cluster members and stores additional information\nas annotations on the Node resources.",
-          "x-intellij-html-description": "\u003cp\u003eKubernetes registry uses Kubernetes API server to discover cluster members and stores additional information\nas annotations on the Node resources.\u003c/p\u003e\n"
+          "description": "Kubernetes registry uses Kubernetes API server to discover cluster members and stores additional information\nas annotations on the Node resources.\n\nThis feature is deprecated as it is not compatible with Kubernetes 1.32+.\nSee https://github.com/siderolabs/talos/issues/9980 for more information.\n",
+          "markdownDescription": "Kubernetes registry uses Kubernetes API server to discover cluster members and stores additional information\nas annotations on the Node resources.\n\nThis feature is deprecated as it is not compatible with Kubernetes 1.32+.\nSee https://github.com/siderolabs/talos/issues/9980 for more information.",
+          "x-intellij-html-description": "\u003cp\u003eKubernetes registry uses Kubernetes API server to discover cluster members and stores additional information\nas annotations on the Node resources.\u003c/p\u003e\n\n\u003cp\u003eThis feature is deprecated as it is not compatible with Kubernetes 1.32+.\nSee \u003ca href=\"https://github.com/siderolabs/talos/issues/9980\" target=\"_blank\"\u003ehttps://github.com/siderolabs/talos/issues/9980\u003c/a\u003e for more information.\u003c/p\u003e\n"
         },
         "service": {
           "$ref": "#/$defs/v1alpha1.RegistryServiceConfig",

website/content/v1.10/schemas/config.schema.json

@@ -20,6 +20,8 @@ Sidero Labs runs a public external registry service, which is enabled by default
 The Kubernetes registry service is disabled by default.
 The advantage of the external registry service is that it is not dependent on etcd, and thus can inform you of cluster membership even when Kubernetes is down.
 
+> Note: Kubernetes registry is deprecated as it is not compatible with Kubernetes 1.32 and later versions in the default configuration.
+
 ## Video Walkthrough
 
 To see a live demo of Cluster Discovery, see the video below:
@@ -46,6 +48,8 @@ Disabling all registries effectively disables member discovery.
 
 > Note: An enabled discovery service is required for [KubeSpan]({{< relref "../talos-guides/network/kubespan/" >}}) to function correctly.
 
+### Kubernetes Registry
+
 The `Kubernetes` registry uses Kubernetes `Node` resource data and additional Talos annotations:
 
 ```sh
@@ -56,15 +60,21 @@ Annotations:        cluster.talos.dev/node-id: Utoh3O0ZneV0kT2IUBrh7TgdouRcUW2yz
 ...
 ```
 
+> Note: Starting with Kubernetes 1.32, the feature gate `AuthorizeNodeWithSelectors` enables additional authorization for `Node` resource read access via `system:node:*` role.
+> This prevents Talos Kubernetes registry from functioning correctly.
+> The workaround is to disable the feature gate on the API server, but it's not recommended as it disables also other important security protections.
+> For this reason, the Kubernetes registry is deprecated and disabled by default.
+
+### Discovery Service Registry
+
 The `Service` registry by default uses a public external Discovery Service to exchange encrypted information about cluster members.
 
 > Note: Talos supports operations when Discovery Service is disabled, but some features will rely on Kubernetes API availability to discover
 > controlplane endpoints, so in case of a failure disabled Discovery Service makes troubleshooting much harder.
 
-## Discovery Service
-
 Sidero Labs maintains a public discovery service at `https://discovery.talos.dev/` whereby cluster members use a shared key that is globally unique to coordinate basic connection information (i.e. the set of possible "endpoints", or IP:port pairs).
 We call this data "affiliate data."
+This data is encrypted by Talos Linux before being sent to the discovery service, and it can only be decrypted by the cluster members.
 
 > Note: If KubeSpan is enabled the data has the addition of the WireGuard public key.
 
@@ -73,7 +83,7 @@ Each node submits its own data, plus the endpoints it sees from other peers, to
 The discovery service aggregates the data, deduplicates the endpoints, and sends updates to each connected peer.
 Each peer receives information back from the discovery service, decrypts it and uses it to drive KubeSpan and cluster discovery.
 
-Data is stored in memory only.
+Data is stored in memory only (and snapshotted to disk in encrypted way to facilitate quick recovery on restarts).
 The cluster ID is used as a key to select the affiliates (so that different clusters see different affiliates).
 
 To summarize, the discovery service knows the client version, cluster ID, the number of affiliates, some encrypted data for each affiliate, and a list of encrypted endpoints.

website/content/v1.10/talos-guides/discovery.md

@@ -20,6 +20,8 @@ Sidero Labs runs a public external registry service, which is enabled by default
 The Kubernetes registry service is disabled by default.
 The advantage of the external registry service is that it is not dependent on etcd, and thus can inform you of cluster membership even when Kubernetes is down.
 
+> Note: Kubernetes registry is deprecated as it is not compatible with Kubernetes 1.32 and later versions in the default configuration.
+
 ## Video Walkthrough
 
 To see a live demo of Cluster Discovery, see the video below:
@@ -46,6 +48,8 @@ Disabling all registries effectively disables member discovery.
 
 > Note: An enabled discovery service is required for [KubeSpan]({{< relref "../talos-guides/network/kubespan/" >}}) to function correctly.
 
+### Kubernetes Registry
+
 The `Kubernetes` registry uses Kubernetes `Node` resource data and additional Talos annotations:
 
 ```sh
@@ -56,15 +60,21 @@ Annotations:        cluster.talos.dev/node-id: Utoh3O0ZneV0kT2IUBrh7TgdouRcUW2yz
 ...
 ```
 
+> Note: Starting with Kubernetes 1.32, the feature gate `AuthorizeNodeWithSelectors` enables additional authorization for `Node` resource read access via `system:node:*` role.
+> This prevents Talos Kubernetes registry from functioning correctly.
+> The workaround is to disable the feature gate on the API server, but it's not recommended as it disables also other important security protections.
+> For this reason, the Kubernetes registry is deprecated and disabled by default.
+
+### Discovery Service Registry
+
 The `Service` registry by default uses a public external Discovery Service to exchange encrypted information about cluster members.
 
 > Note: Talos supports operations when Discovery Service is disabled, but some features will rely on Kubernetes API availability to discover
 > controlplane endpoints, so in case of a failure disabled Discovery Service makes troubleshooting much harder.
 
-## Discovery Service
-
 Sidero Labs maintains a public discovery service at `https://discovery.talos.dev/` whereby cluster members use a shared key that is globally unique to coordinate basic connection information (i.e. the set of possible "endpoints", or IP:port pairs).
 We call this data "affiliate data."
+This data is encrypted by Talos Linux before being sent to the discovery service, and it can only be decrypted by the cluster members.
 
 > Note: If KubeSpan is enabled the data has the addition of the WireGuard public key.
 
@@ -73,7 +83,7 @@ Each node submits its own data, plus the endpoints it sees from other peers, to
 The discovery service aggregates the data, deduplicates the endpoints, and sends updates to each connected peer.
 Each peer receives information back from the discovery service, decrypts it and uses it to drive KubeSpan and cluster discovery.
 
-Data is stored in memory only.
+Data is stored in memory only (and snapshotted to disk in encrypted way to facilitate quick recovery on restarts).
 The cluster ID is used as a key to select the affiliates (so that different clusters see different affiliates).
 
 To summarize, the discovery service knows the client version, cluster ID, the number of affiliates, some encrypted data for each affiliate, and a list of encrypted endpoints.