This is a short blog to let you know that objects represented in unstructured.Unstructured will not be cached by the client returned from manager.Manager by default.
UPDATED on Jan. 9th, 2023
The trick described in this article is no longer needed for controller-runtime version 0.14.0 or later.
ref. github.com
This also means that in-memory indexing and retrieving of unstructured objects will not work by default.
To enable caching for unstructured objects, define a custom function to create a client like this:
func NewCachingClient(cache cache.Cache, config *rest.Config, options client.Options, uncachedObjects ...client.Object) (client.Client, error) { c, err := client.New(config, options) if err != nil { return nil, err } return client.NewDelegatingClient(client.NewDelegatingClientInput{ CacheReader: cache, Client: c, UncachedObjects: uncachedObjects, // THIS IS THE MAGIC CacheUnstructured: true, }) }
and give it when creating manager.Manager as follows:
mgr, err := ctrl.NewManager(ctrl.GetConfigOrDie(), ctrl.Options{
NewClient: NewCachingClient,
...
Read the rest of the article if you'd like to know more about what's happening under the hood.
controller-runtime is the runtime library of kubebuilder. With controller-runtime and kubebuilder, a custom Kubernetes controller can be created easily thanks to their rich functionalities.
One of the functionalities in controller-runtime is to handle any kind of API resources using unstructured.Unstructured.
Unlike types defined in k8s.io/api, unstructured can represent any kind of resources because it internally holds an object as map[string]interface{}.
The following is an example of Service represented in unstructured.Unstructured.
svc := &unstructured.Unstructured{}
svc.SetGroupVersionKind(schema.GroupVersionKind{Version: "v1", Kind: "Service"})
svc.SetNamespace("default")
svc.SetName("test")
svc.UnstructuredContent()["spec"] = map[string]interface{}{
"selector": map[string]interface{}{"app": "sample"},
"ports": []interface{}{
map[string]interface{}{
"name": "https",
"protocol": "TCP",
"port": 443,
"targetPort": 8443,
},
},
}
controller-runtime's client.Client can handle unstructured objects to create/update/delete resources in kube-apiserver.
To reduce the load of kube-apiserver, controller-runtime provides in-memory cache and indexing of objects.
Normally, the cache is used transparently via the client.Client object obtained through Manager.GetClient.
The culprit is the default implementation of creating the caching client.Client. It is defined as cluster.DefaultNewClient as follows:
func DefaultNewClient(cache cache.Cache, config *rest.Config, options client.Options, uncachedObjects ...client.Object) (client.Client, error) { c, err := client.New(config, options) if err != nil { return nil, err } return client.NewDelegatingClient(client.NewDelegatingClientInput{ CacheReader: cache, Client: c, UncachedObjects: uncachedObjects, })
Let's see what is client.NewDelegatingClientInput.
type NewDelegatingClientInput struct { CacheReader Reader Client Client UncachedObjects []Object CacheUnstructured bool }
Voila! CacheUnstructured is not specified, therefore it is off by default.
That's it. Thanks for reading.
