Skip to content

Commit

Permalink
Revamp "Watching Resources" documentation for accuracy and clarity
Browse files Browse the repository at this point in the history
- Fully revamped the "Watching Resources" section for improved clarity and accuracy.
- Ensured terminology aligns with Kubernetes and controller-runtime standards.
- Provided detailed examples for watching owned resources, non-owned resources, and applying predicates to refine watches.
  • Loading branch information
camilamacedo86 committed Sep 17, 2024
1 parent 333170b commit f19cdd9
Show file tree
Hide file tree
Showing 12 changed files with 536 additions and 573 deletions.
5 changes: 3 additions & 2 deletions docs/book/src/SUMMARY.md
Original file line number Diff line number Diff line change
Expand Up @@ -76,8 +76,9 @@
- [Good Practices](./reference/good-practices.md)
- [Raising Events](./reference/raising-events.md)
- [Watching Resources](./reference/watching-resources.md)
- [Resources Managed by the Operator](./reference/watching-resources/operator-managed.md)
- [Externally Managed Resources](./reference/watching-resources/externally-managed.md)
- [Owned Resources](./reference/watching-resources/secondary-owned-resources.md)
- [Not Owned Resources](./reference/watching-resources/secondary-resources-not-owned)
- [Using Predicates](./reference/watching-resources/predicates-with-watch.md)
- [Kind for Dev & CI](reference/kind.md)
- [What's a webhook?](reference/webhook-overview.md)
- [Admission webhook](reference/admission-webhook.md)
Expand Down
6 changes: 3 additions & 3 deletions docs/book/src/reference/reference.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,9 +7,9 @@
Kubernetes cluster.
- [Watching Resources](watching-resources.md)
Watch resources in the Kubernetes cluster to be informed and take actions on changes.
- [Resources Managed by the Operator](watching-resources/operator-managed.md)
- [Externally Managed Resources](watching-resources/externally-managed.md)
Controller Runtime provides the ability to watch additional resources relevant to the controlled ones.
- [Watching Secondary Resources that are `Owned` ](watching-resources/secondary-owned-resources.md)
- [Watching Secondary Resources that are NOT `Owned`](watching-resources/secondary-resources-not-owned)
- [Using Predicates to Refine Watches](watching-resources/predicates-with-watch.md)
- [Kind cluster](kind.md)
- [What's a webhook?](webhook-overview.md)
Webhooks are HTTP callbacks, there are 3
Expand Down
151 changes: 140 additions & 11 deletions docs/book/src/reference/watching-resources.md
Original file line number Diff line number Diff line change
@@ -1,16 +1,145 @@
# Watching Resources

Inside a `Reconcile()` control loop, you are looking to do a collection of operations until it has the desired state on the cluster.
Therefore, it can be necessary to know when a resource that you care about is changed.
In the case that there is an action (create, update, edit, delete, etc.) on a watched resource, `Reconcile()` should be called for the resources watching it.
When extending the Kubernetes API, we aim to ensure that our solutions behave consistently with Kubernetes itself.
For example, consider a `Deployment` resource, which is managed by a controller. This controller is responsible
for responding to changes in the cluster—such as when a `Deployment` is created, updated, or deleted—by triggering
reconciliation to ensure the resource’s state matches the desired state.

[Controller Runtime libraries](https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/builder) provide many ways for resources to be managed and watched.
This ranges from the easy and obvious use cases, such as watching the resources which were created and managed by the controller, to more unique and advanced use cases.
Similarly, when developing our controllers, we want to watch for relevant changes in resources that are crucial
to our solution. These changes—whether creations, updates, or deletions—should trigger the reconciliation
loop to take appropriate actions and maintain consistency across the cluster.

See each subsection for explanations and examples of the different ways in which your controller can _Watch_ the resources it cares about.
The [controller-runtime][controller-runtime] library provides several ways to watch and manage resources.

- [Watching Operator Managed Resources](watching-resources/operator-managed.md) -
These resources are created and managed by the same operator as the resource watching them.
This section covers both if they are managed by the same controller or separate controllers.
- [Watching Externally Managed Resources](watching-resources/externally-managed.md) -
These resources could be manually created, or managed by other operators/controllers or the Kubernetes control plane.
## Primary Resources

The **Primary Resource** is the resource that your controller is responsible
for managing. For example, if you create a custom resource definition (CRD) for `MyApp`,
the corresponding controller is responsible for managing instances of `MyApp`.

In this case, `MyApp` is the **Primary Resource** for that controller, and your controller’s
reconciliation loop focuses on ensuring the desired state of these primary resources is maintained.

When you create a new API using Kubebuilder, the following default code is scaffolded,
ensuring that the controller watches all relevant events—such as creations, updates, and
deletions—for (`For()`) the new API.

This setup guarantees that the reconciliation loop is triggered whenever an instance
of the API is created, updated, or deleted:

```go
// Watches the primary resource (e.g., MyApp) for create, update, delete events
if err := ctrl.NewControllerManagedBy(mgr).
For(&<YourAPISpec>{}). <-- See there that the Controller is For this API
Complete(r); err != nil {
return err
}
```

## Secondary Resources

Your controller will likely also need to manage **Secondary Resources**,
which are the resources required on the cluster to support the **Primary Resource**.

Changes to these **Secondary Resources** can directly impact the **Primary Resource**,
so the controller must watch and reconcile these resources accordingly.

### Which are Owned by the Controller

These **Secondary Resources**, such as `Services`, `ConfigMaps`, or `Deployments`,
when `Owned` by the controllers, are created and managed by the specific controller
and are tied to the **Primary Resource** via [OwnerReferences][owner-ref-k8s-docs].

For example, if we have a controller to manage our CR(s) of the Kind `MyApp`
on the cluster, which represents our application solution, all resources required
to ensure that `MyApp` is up and running with the desired number of instances
will be **Secondary Resources**. The code responsible for creating, deleting,
and updating these resources will be part of the `MyApp` Controller.
We would add the appropriate [OwnerReferences][owner-ref-k8s-docs]
using the [controllerutil.SetControllerReference][cr-owner-ref-doc]
function to indicate that these resources are owned by the same controller
responsible for managing `MyApp` instances, which will be reconciled by the `MyAppReconciler`.

Additionally, if the **Primary Resource** is deleted, Kubernetes' garbage collection mechanism
ensures that all associated **Secondary Resources** are automatically deleted in a
cascading manner.

### Which are NOT `Owned` by the Controller

Note that **Secondary Resources** can either be APIs/CRDs defined in your project or in other projects that are
relevant to the **Primary Resources**, but which the specific controller is not responsible for creating or managing.

For example, if we have a CRD that represents a backup solution (i.e. `MyBackup`) for our `MyApp`,
it might need to watch changes in the `MyApp` resource to trigger reconciliation in `MyBackup`
to ensure the desired state. Similarly, `MyApp`'s behavior might also be impacted by
CRDs/APIs defined in other projects.

In both scenarios, these resources are treated as **Secondary Resources**, even if they are not `Owned`
(i.e., not created or managed) by the `MyAppController`.

In Kubebuilder, resources that are not defined in the project itself and are not
a **Core Type** (those not defined in the Kubernetes API) are called **External Types**.

An **External Type** refers to a resource that is not defined in your
project but one that you need to watch and respond to.
For example, if **Operator A** manages a `MyApp` CRD for application deployment,
and **Operator B** handles backups, **Operator B** can watch the `MyApp` CRD as an external type
to trigger backup operations based on changes in `MyApp`.

In this scenario, **Operator B** could define a `BackupConfig` CRD that relies on the state of `MyApp`.
By treating `MyApp` as a **Secondary Resource**, **Operator B** can watch and reconcile changes in **Operator A**'s `MyApp`,
ensuring that backup processes are initiated whenever `MyApp` is updated or scaled.

## General Concept of Watching Resources

Whether a resource is defined within your project or comes from an external project, the concept of **Primary**
and **Secondary Resources** remains the same:
- The **Primary Resource** is the resource the controller is primarily responsible for managing.
- **Secondary Resources** are those that are required to ensure the primary resource works as desired.

Therefore, regardless of whether the resource was defined by your project or by another project,
your controller can watch, reconcile, and manage changes to these resources as needed.

## Why watch the secondary resources matters?

When building a Kubernetes controller, it’s crucial to not only focus
on **Primary Resources** but also to monitor **Secondary Resources**.
Failing to track these resources can lead to inconsistencies in your
controller's behavior and the overall cluster state.

Secondary resources may not be directly managed by your controller,
but changes to these resources can still significantly
impact the primary resource and your controller's functionality.
Here are the key reasons why it's important to watch them:

- **Ensuring Consistency**:
- Secondary resources (e.g., child objects or external dependencies) may diverge from their expected state.
For instance, a secondary resource may be modified or deleted, causing the system to fall out of sync.
- Watching secondary resources ensures that any changes are detected immediately, allowing the controller to
reconcile and restore the desired state.

- **Avoiding Random Self-Healing**:
- Without watching secondary resources, the controller may "heal" itself only upon restart or when specific events
are triggered. This can cause unpredictable or delayed reactions to issues.
- Monitoring secondary resources ensures that inconsistencies are addressed promptly, rather than waiting for a
controller restart or external event to trigger reconciliation.

- **Effective Lifecycle Management**:
- Secondary resources might not be owned by the controller directly, but their state still impacts the behavior
of primary resources. Without watching these, you risk leaving orphaned or outdated resources.
- Watching non-owned secondary resources lets the controller respond to lifecycle events (create, update, delete)
that might affect the primary resource, ensuring consistent behavior across the system.

## Usage of Predicates

For more complex use cases, [Predicates][cr-predicates] can be used to fine-tune
when your controller should trigger reconciliation. Predicates allow you to filter
events based on specific conditions, such as changes to particular fields, labels, or annotations,
ensuring that your controller only responds to relevant events and operates efficiently.

[controller-runtime]: https://github.com/kubernetes-sigs/controller-runtime
[owner-ref-k8s-docs]: https://kubernetes.io/docs/concepts/overview/working-with-objects/owners-dependents/
[cr-predicates]: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/predicate
[secondary-resources-doc]: watching-resources/secondary-owned-resources
[predicates-with-external-type-doc]: watching-resources/predicates-with-watch
[cr-owner-ref-doc]: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/controller/controllerutil#SetOwnerReference
31 changes: 0 additions & 31 deletions docs/book/src/reference/watching-resources/externally-managed.md

This file was deleted.

25 changes: 0 additions & 25 deletions docs/book/src/reference/watching-resources/operator-managed.md

This file was deleted.

113 changes: 113 additions & 0 deletions docs/book/src/reference/watching-resources/predicates-with-watch.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,113 @@
# Using Predicates to Refine Watches

When working with controllers, it's often beneficial to use **Predicates** to
filter events and control when the reconciliation loop should be triggered.

[Predicates][predicates-doc] allow you to define conditions based on events (such as create, update, or delete)
and resource fields (such as labels, annotations, or status fields). By using **[Predicates][predicates-doc]**,
you can refine your controller’s behavior to respond only to specific changes in the resources
it watches.

This can be especially useful when you want to refine which
changes in resources should trigger a reconciliation. By using predicates,
you avoid unnecessary reconciliations and can ensure that the
controller only reacts to relevant changes.

## When to Use Predicates

**Predicates are useful when:**

- You want to ignore certain changes, such as updates that don't impact the fields your controller is concerned with.
- You want to trigger reconciliation only for resources with specific labels or annotations.
- You want to watch external resources and react only to specific changes.

## Example: Using Predicates to Filter Update Events

Let’s say that we only want our **`BackupBusybox`** controller to reconcile
when certain fields of the **`Busybox`** resource change, for example, when
the `spec.size` field changes, but we want to ignore all other changes (such as status updates).

### Defining a Predicate

In the following example, we define a predicate that only
allows reconciliation when there’s a meaningful update
to the **`Busybox`** resource:

```go
import (
"sigs.k8s.io/controller-runtime/pkg/predicate"
"sigs.k8s.io/controller-runtime/pkg/event"
)

// Predicate to trigger reconciliation only on size changes in the Busybox spec
updatePred := predicate.Funcs{
// Only allow updates when the spec.size of the Busybox resource changes
UpdateFunc: func(e event.UpdateEvent) bool {
oldObj := e.ObjectOld.(*examplecomv1alpha1.Busybox)
newObj := e.ObjectNew.(*examplecomv1alpha1.Busybox)

// Trigger reconciliation only if the spec.size field has changed
return oldObj.Spec.Size != newObj.Spec.Size
},

// Allow create events
CreateFunc: func(e event.CreateEvent) bool {
return true
},

// Allow delete events
DeleteFunc: func(e event.DeleteEvent) bool {
return true
},

// Allow generic events (e.g., external triggers)
GenericFunc: func(e event.GenericEvent) bool {
return true
},
}
```

### Explanation

In this example:
- The **`UpdateFunc`** returns `true` only if the **`spec.size`** field has changed between the old and new objects, meaning that all other changes in the `spec`, like annotations or other fields, will be ignored.
- **`CreateFunc`**, **`DeleteFunc`**, and **`GenericFunc`** return `true`, meaning that create, delete, and generic events are still processed, allowing reconciliation to happen for these event types.

This ensures that the controller reconciles only when the specific field **`spec.size`** is modified, while ignoring any other modifications in the `spec` that are irrelevant to your logic.

### Example: Using Predicates in `Watches`

Now, we apply this predicate in the **`Watches()`** method of
the **`BackupBusyboxReconciler`** to trigger reconciliation only for relevant events:

```go
// SetupWithManager sets up the controller with the Manager.
// The controller will watch both the BackupBusybox primary resource and the Busybox resource, using predicates.
func (r *BackupBusyboxReconciler) SetupWithManager(mgr ctrl.Manager) error {
return ctrl.NewControllerManagedBy(mgr).
For(&examplecomv1alpha1.BackupBusybox{}). // Watch the primary resource (BackupBusybox)
Watches(
&source.Kind{Type: &examplecomv1alpha1.Busybox{}}, // Watch the Busybox CR
handler.EnqueueRequestsFromMapFunc(func(obj client.Object) []reconcile.Request {
return []reconcile.Request{
{
NamespacedName: types.NamespacedName{
Name: "backupbusybox", // Reconcile the associated BackupBusybox resource
Namespace: obj.GetNamespace(), // Use the namespace of the changed Busybox
},
},
}
}),
builder.WithPredicates(updatePred), // Apply the predicate
). // Trigger reconciliation when the Busybox resource changes (if it meets predicate conditions)
Complete(r)
}
```

### Explanation

- **[`builder.WithPredicates(updatePred)`][predicates-doc]**: This method applies the predicate, ensuring that reconciliation only occurs
when the **`spec.size`** field in **`Busybox`** changes.
- **Other Events**: The controller will still trigger reconciliation on `Create`, `Delete`, and `Generic` events.

[predicates-doc]: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/source#WithPredicates
Loading

0 comments on commit f19cdd9

Please sign in to comment.