Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
33 changes: 33 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,7 @@ are deleted they are no longer visible on the `/metrics` endpoint.
* [Horizontal sharding](#horizontal-sharding)
* [Automated sharding](#automated-sharding)
* [Daemonset sharding for pod metrics](#daemonset-sharding-for-pod-metrics)
* [Per-resource label selectors](#per-resource-label-selectors)
* [Resource filtering](#resource-filtering)
* [Setup](#setup)
* [Building the Docker container](#building-the-docker-container)
Expand Down Expand Up @@ -304,6 +305,38 @@ spec:

Other metrics can be sharded via [Horizontal sharding](#horizontal-sharding).

### Per-resource label selectors

To restrict which objects kube-state-metrics watches for selected builtin resources, use the repeatable `--label-selector` flag:

* `--label-selector=nodes=tenant=team-a`
* `--label-selector='pods=app in (frontend,api)'`

Each flag applies a Kubernetes `LabelSelector` to a single resource type. Resource names use the same plural form as `--resources`, such as `pods`, `nodes`, or `namespaces`. Resources without a matching `--label-selector` flag remain unfiltered.

Filtered resources expose metrics only for the selected subset of objects and do not preserve cluster-wide semantics.

`--label-selector=pods=...` can be combined with `--node=$(NODE_NAME)`. In that case both filters apply to Pod metrics.

Examples:

```yaml
apiVersion: apps/v1
kind: Deployment
spec:
template:
spec:
containers:
- image: registry.k8s.io/kube-state-metrics/kube-state-metrics:IMAGE_TAG
name: kube-state-metrics
args:
- --resources=nodes,pods
- --label-selector=nodes=tenant=team-a
- --label-selector=pods=app=frontend
```

This flag currently supports builtin resources only. Custom resource metrics use their own configuration paths.

#### Resource Filtering

The `/metrics` endpoint supports filtering by resource type using the `resources` query parameter. This allows you to scrape only the metrics for specific Kubernetes resources, which can be useful for reducing the amount of data scraped or for creating separate scraping jobs for different resource types.
Expand Down
33 changes: 33 additions & 0 deletions README.md.tpl
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,7 @@ are deleted they are no longer visible on the `/metrics` endpoint.
* [Horizontal sharding](#horizontal-sharding)
* [Automated sharding](#automated-sharding)
* [Daemonset sharding for pod metrics](#daemonset-sharding-for-pod-metrics)
* [Per-resource label selectors](#per-resource-label-selectors)
* [Resource filtering](#resource-filtering)
* [Setup](#setup)
* [Building the Docker container](#building-the-docker-container)
Expand Down Expand Up @@ -305,6 +306,38 @@ spec:

Other metrics can be sharded via [Horizontal sharding](#horizontal-sharding).

### Per-resource label selectors

To restrict which objects kube-state-metrics watches for selected builtin resources, use the repeatable `--label-selector` flag:

* `--label-selector=nodes=tenant=team-a`
* `--label-selector='pods=app in (frontend,api)'`

Each flag applies a Kubernetes `LabelSelector` to a single resource type. Resource names use the same plural form as `--resources`, such as `pods`, `nodes`, or `namespaces`. Resources without a matching `--label-selector` flag remain unfiltered.

Filtered resources expose metrics only for the selected subset of objects and do not preserve cluster-wide semantics.

`--label-selector=pods=...` can be combined with `--node=$(NODE_NAME)`. In that case both filters apply to Pod metrics.

Examples:

```yaml
apiVersion: apps/v1
kind: Deployment
spec:
template:
spec:
containers:
- image: registry.k8s.io/kube-state-metrics/kube-state-metrics:IMAGE_TAG
name: kube-state-metrics
args:
- --resources=nodes,pods
- --label-selector=nodes=tenant=team-a
- --label-selector=pods=app=frontend
```

This flag currently supports builtin resources only. Custom resource metrics use their own configuration paths.

#### Resource Filtering

The `/metrics` endpoint supports filtering by resource type using the `resources` query parameter. This allows you to scrape only the metrics for specific Kubernetes resources, which can be useful for reducing the amount of data scraped or for creating separate scraping jobs for different resource types.
Expand Down
1 change: 1 addition & 0 deletions docs/developer/cli-arguments.md
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,7 @@ Flags:
-h, --help Print Help text
--host string Host to expose metrics on. (default "::")
--kubeconfig string Absolute path to the kubeconfig file
--label-selector string Repeatable resource-specific Kubernetes label selectors in the form 'resource=labelSelector'. Resources use the same plural names as --resources. Examples: '--label-selector=pods=app=frontend' or '--label-selector=nodes=tenant in (team-a,team-b)'.
--legacy_stderr_threshold_behavior If true, stderrthreshold is ignored when logtostderr=true (legacy behavior). If false, stderrthreshold is honored even when logtostderr=true (default true)
--log_backtrace_at traceLocation when logging hits line file:N, emit a stack trace (default :0)
--log_dir string If non-empty, write log files in this directory (no effect when -logtostderr=true)
Expand Down
50 changes: 48 additions & 2 deletions internal/store/builder.go
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,9 @@ import (
policyv1 "k8s.io/api/policy/v1"
rbacv1 "k8s.io/api/rbac/v1"
storagev1 "k8s.io/api/storage/v1"
"k8s.io/apimachinery/pkg/api/meta"
"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured"
"k8s.io/apimachinery/pkg/runtime/schema"
clientset "k8s.io/client-go/kubernetes"
"k8s.io/client-go/tools/cache"
"k8s.io/klog/v2"
Expand Down Expand Up @@ -76,6 +78,7 @@ type Builder struct {
buildCustomResourceStoresFunc ksmtypes.BuildCustomResourceStoresFunc
allowAnnotationsList map[string][]string
allowLabelsList map[string][]string
labelSelectorFilters map[string]string
utilOptions *options.Options
// namespaceFilter is inside fieldSelectorFilter
fieldSelectorFilter string
Expand Down Expand Up @@ -129,6 +132,25 @@ func (b *Builder) WithFieldSelectorFilter(fieldSelectorFilter string) {
b.fieldSelectorFilter = fieldSelectorFilter
}

// WithLabelSelectorFilters sets resource-specific label selectors for builtin resources.
func (b *Builder) WithLabelSelectorFilters(labelSelectorFilters map[string]string) error {
if len(labelSelectorFilters) == 0 {
b.labelSelectorFilters = nil
return nil
}

validatedFilters := make(map[string]string, len(labelSelectorFilters))
for resource, selector := range labelSelectorFilters {
if !resourceExists(resource) {
return fmt.Errorf("resource %s does not exist. Available resources: %s", resource, strings.Join(availableResources(), ","))
}
validatedFilters[resource] = selector
}

b.labelSelectorFilters = validatedFilters
return nil
}

// WithNamespaces sets the namespaces property of a Builder.
func (b *Builder) WithNamespaces(n options.NamespaceList) {
b.namespaces = n
Expand Down Expand Up @@ -522,6 +544,7 @@ func (b *Builder) buildStores(
metricFamilies = generator.FilterFamilyGenerators(b.familyGeneratorFilter, metricFamilies)
composedMetricGenFuncs := generator.ComposeMetricGenFuncs(metricFamilies)
familyHeaders := generator.ExtractMetricFamilyHeaders(metricFamilies)
labelSelector := b.labelSelectorForExpectedType(expectedType)

if b.namespaces.IsAllNamespaces() {
store := metricsstore.NewMetricsStore(
Expand All @@ -531,7 +554,10 @@ func (b *Builder) buildStores(
if b.fieldSelectorFilter != "" {
klog.InfoS("FieldSelector is used", "fieldSelector", b.fieldSelectorFilter)
}
listWatcher := listWatchFunc(b.kubeClient, v1.NamespaceAll, b.fieldSelectorFilter)
if labelSelector != "" {
klog.InfoS("LabelSelector is used", "labelSelector", labelSelector)
}
listWatcher := withLabelSelector(listWatchFunc(b.kubeClient, v1.NamespaceAll, b.fieldSelectorFilter), labelSelector)
b.startReflector(expectedType, store, listWatcher, useAPIServerCache, objectLimit, b.kubeClient)
return []cache.Store{store}
}
Expand All @@ -545,7 +571,10 @@ func (b *Builder) buildStores(
if b.fieldSelectorFilter != "" {
klog.InfoS("FieldSelector is used", "fieldSelector", b.fieldSelectorFilter)
}
listWatcher := listWatchFunc(b.kubeClient, ns, b.fieldSelectorFilter)
if labelSelector != "" {
klog.InfoS("LabelSelector is used", "labelSelector", labelSelector)
}
listWatcher := withLabelSelector(listWatchFunc(b.kubeClient, ns, b.fieldSelectorFilter), labelSelector)
b.startReflector(expectedType, store, listWatcher, useAPIServerCache, objectLimit, b.kubeClient)
stores = append(stores, store)
}
Expand Down Expand Up @@ -637,3 +666,20 @@ func cacheStoresToMetricStores(cStores []cache.Store) []*metricsstore.MetricsSto

return mStores
}

func (b *Builder) labelSelectorForExpectedType(expectedType interface{}) string {
if len(b.labelSelectorFilters) == 0 {
return ""
}

t := reflect.TypeOf(expectedType)
if t == nil {
return ""
}
if t.Kind() == reflect.Ptr {
t = t.Elem()
}

resource, _ := meta.UnsafeGuessKindToResource(schema.GroupVersionKind{Kind: t.Name()})
return b.labelSelectorFilters[resource.Resource]
}
93 changes: 93 additions & 0 deletions internal/store/builder_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -17,10 +17,17 @@ limitations under the License.
package store

import (
"context"
"reflect"
"slices"
"testing"

v1 "k8s.io/api/core/v1"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/apimachinery/pkg/runtime"
"k8s.io/apimachinery/pkg/watch"
"k8s.io/client-go/tools/cache"

"k8s.io/kube-state-metrics/v2/pkg/options"
)

Expand All @@ -32,6 +39,21 @@ type expectedError struct {
expectedNotEqual bool
}

type fakeListerWatcher struct {
listOptions []metav1.ListOptions
watchOptions []metav1.ListOptions
}

func (r *fakeListerWatcher) ListWithContext(_ context.Context, options metav1.ListOptions) (runtime.Object, error) {
r.listOptions = append(r.listOptions, options)
return &v1.PodList{}, nil
}

func (r *fakeListerWatcher) WatchWithContext(_ context.Context, options metav1.ListOptions) (watch.Interface, error) {
r.watchOptions = append(r.watchOptions, options)
return watch.NewEmptyWatch(), nil
}

func TestWithAllowLabels(t *testing.T) {
tests := []struct {
Desc string
Expand Down Expand Up @@ -258,3 +280,74 @@ func TestWithEnabledResources(t *testing.T) {
}
}
}

func TestWithLabelSelectorFilters(t *testing.T) {
tests := []struct {
Desc string
LabelSelector map[string]string
Want map[string]string
WantError bool
}{
{
Desc: "builtin resource selector",
LabelSelector: map[string]string{"pods": "app=frontend", "nodes": "tenant=team-a"},
Want: map[string]string{"pods": "app=frontend", "nodes": "tenant=team-a"},
},
{
Desc: "unknown resource selector",
LabelSelector: map[string]string{"foos": "app=frontend"},
WantError: true,
},
}

for _, test := range tests {
b := NewBuilder()
err := b.WithLabelSelectorFilters(test.LabelSelector)

if (err != nil) != test.WantError {
t.Fatalf("Test error for Desc: %s. Wanted Error: %v, Got Error: %v", test.Desc, test.WantError, err)
}
if err == nil && !reflect.DeepEqual(b.labelSelectorFilters, test.Want) {
t.Errorf("Test error for Desc: %s\n Want: \n%+v\n Got: \n%#+v", test.Desc, test.Want, b.labelSelectorFilters)
}
}
}

func TestWithLabelSelector(t *testing.T) {
fakeLW := &fakeListerWatcher{}
baseLW := &cache.ListWatch{
ListWithContextFunc: fakeLW.ListWithContext,
WatchFuncWithContext: fakeLW.WatchWithContext,
}
labelSelectorLW := withLabelSelector(baseLW, "tenant in (team-a,team-b)")
listerWatcherWithContext := cache.ToListerWatcherWithContext(labelSelectorLW)

_, err := listerWatcherWithContext.ListWithContext(context.Background(), metav1.ListOptions{FieldSelector: "spec.nodeName=node-a", ResourceVersion: "10"})
if err != nil {
t.Fatalf("unexpected list error: %v", err)
}

_, err = listerWatcherWithContext.WatchWithContext(context.Background(), metav1.ListOptions{FieldSelector: "spec.nodeName=node-a"})
if err != nil {
t.Fatalf("unexpected watch error: %v", err)
}

if len(fakeLW.listOptions) != 1 {
t.Fatalf("expected 1 list call, got %d", len(fakeLW.listOptions))
}
if len(fakeLW.watchOptions) != 1 {
t.Fatalf("expected 1 watch call, got %d", len(fakeLW.watchOptions))
}
if got := fakeLW.listOptions[0].LabelSelector; got != "tenant in (team-a,team-b)" {
t.Fatalf("expected list label selector to be propagated, got %q", got)
}
if got := fakeLW.watchOptions[0].LabelSelector; got != "tenant in (team-a,team-b)" {
t.Fatalf("expected watch label selector to be propagated, got %q", got)
}
if got := fakeLW.listOptions[0].FieldSelector; got != "spec.nodeName=node-a" {
t.Fatalf("expected list field selector to be preserved, got %q", got)
}
if got := fakeLW.listOptions[0].ResourceVersion; got != "10" {
t.Fatalf("expected list resource version to be preserved, got %q", got)
}
}
64 changes: 64 additions & 0 deletions internal/store/listwatch.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
/*
Copyright 2026 The Kubernetes Authors All rights reserved.

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/

package store

import (
"context"

metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/apimachinery/pkg/runtime"
"k8s.io/apimachinery/pkg/watch"
"k8s.io/client-go/tools/cache"
)

type labelSelectorListerWatcher struct {
*cache.ListWatch
lw cache.ListerWatcher
}

func withLabelSelector(lw cache.ListerWatcher, labelSelector string) cache.ListerWatcher {
if labelSelector == "" {
return lw
}

listerWatcherWithContext := cache.ToListerWatcherWithContext(lw)
return &labelSelectorListerWatcher{
ListWatch: &cache.ListWatch{
ListWithContextFunc: func(ctx context.Context, options metav1.ListOptions) (runtime.Object, error) {
options.LabelSelector = labelSelector
return listerWatcherWithContext.ListWithContext(ctx, options)
},
WatchFuncWithContext: func(ctx context.Context, options metav1.ListOptions) (watch.Interface, error) {
options.LabelSelector = labelSelector
return listerWatcherWithContext.WatchWithContext(ctx, options)
},
},
lw: lw,
}
}

func (l *labelSelectorListerWatcher) IsWatchListSemanticsUnSupported() bool {
type unsupported interface {
IsWatchListSemanticsUnSupported() bool
}

if u, ok := l.lw.(unsupported); ok {
return u.IsWatchListSemanticsUnSupported()
}

return false
}
Loading