Skip to content

Commit cef707f

Browse files
(docs): Add doc about Strict Server-Side Field Validation
1 parent f148cd9 commit cef707f

File tree

3 files changed

+232
-0
lines changed

3 files changed

+232
-0
lines changed

docs/book/src/SUMMARY.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -77,6 +77,7 @@
7777

7878
- [Generating CRDs](./reference/generating-crd.md)
7979
- [Using Finalizers](./reference/using-finalizers.md)
80+
- [Strict Field Validation](./reference/strict-field-validation.md)
8081
- [Good Practices](./reference/good-practices.md)
8182
- [Raising Events](./reference/raising-events.md)
8283
- [Watching Resources](./reference/watching-resources.md)

docs/book/src/reference/reference.md

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,10 @@
55
Finalizers are a mechanism to
66
execute any custom logic related to a resource before it gets deleted from
77
Kubernetes cluster.
8+
- [Strict Field Validation](strict-field-validation.md)
9+
Reject unknown fields instead of silently dropping them. Useful for development
10+
and CI. Can be used in production when you control CRD upgrade ordering.
11+
Not scaffolded by default.
812
- [Watching Resources](watching-resources.md)
913
Watch resources in the Kubernetes cluster to be informed and take actions on changes.
1014
- [Watching Secondary Resources that are `Owned` ](watching-resources/secondary-owned-resources.md)
Lines changed: 227 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,227 @@
1+
# Strict Field Validation
2+
3+
## Overview
4+
5+
By default, when your controller writes an object that contains fields not defined in the CRD schema, the API server:
6+
7+
- Accepts the request
8+
- Drops the unknown fields
9+
- May only log a warning
10+
11+
This can hide bugs and version skew between:
12+
- The controller code (Go types) and
13+
- The CRD schema installed in the cluster
14+
15+
`controller-runtime` exposes [client.WithFieldValidation][client-field-validation-docs] to turn on strict server-side field validation for all client writes. When enabled, the API server returns a hard error instead of silently dropping unknown fields.
16+
17+
CRDs should be installed before controllers. However, during upgrades this can be best effort rather than guaranteed. Deployment tools may apply resources without strict ordering, and even tools with ordering features don't guarantee the CRD update succeeds. For controllers using external CRDs from third-party operators, ordering may not be controllable at all.
18+
19+
When controllers and CRDs get out of sync, controller writes may fail with unknown field errors, status updates may not persist, and conversion webhooks can fail if the CRD schema is outdated. Controllers may crash-loop until CRDs are updated.
20+
21+
## What does it solve
22+
23+
Strict validation prevents silent failures when your controller code and CRD schemas get out of sync.
24+
25+
For example, you add a new field `status.newField` to your controller, but the CRD in the cluster hasn't been updated yet. When the controller calls `client.Status().Patch(...)`:
26+
27+
**Without strict validation:**
28+
- API server drops `status.newField` silently
29+
- Controller sees no error
30+
- Field never appears on the object - confusing debugging
31+
32+
**With strict validation:**
33+
- API server returns clear error
34+
- Controller knows CRDs need updating
35+
- Fails fast instead of silent data loss
36+
37+
## When to use it
38+
39+
Strict validation is a good fit when:
40+
41+
- You own both the CRDs and the controllers
42+
- Your upgrade process applies CRDs first or ensures they update together
43+
- You want to fail fast when a controller writes fields not in the schema
44+
- You want to catch bugs in your types or conversions early
45+
- You use typed schemas, or explicitly mark dynamic data with `x-kubernetes-preserve-unknown-fields: true`
46+
47+
## When NOT to use it
48+
49+
Avoid strict validation in production when:
50+
51+
- Controllers and CRDs upgrade independently (i.e., common in Helm)
52+
- You manage third-party CRDs whose schemas evolve independently
53+
- Your CRDs use unstructured/dynamic data without `x-kubernetes-preserve-unknown-fields`
54+
- You need upgrade tolerance when controller and CRD versions are temporarily mismatched
55+
56+
## How Kubebuilder scaffold handles CRD ordering
57+
58+
### Makefile Targets (`make install` + `make deploy`)
59+
60+
```bash
61+
make install # Installs CRDs into the cluster
62+
make deploy # Deploys the controller
63+
```
64+
65+
Two separate commands. CRDs are installed first and established before the controller starts. Order is guaranteed.
66+
67+
### YAML bundle distribution (make build-installer)
68+
69+
```bash
70+
kubectl apply -f dist/install.yaml
71+
```
72+
73+
The bundle positions CRDs early in the file, after Namespace and before Deployment. This works for new installations.
74+
75+
During upgrades, if the CRD already exists and the update fails or is slow, the Deployment may still update. The new controller may start before the CRD update completes.
76+
77+
### Helm chart distribution
78+
79+
By using the [Helm plugin](../plugins/available/helm-v2-alpha.md), you can distribute your solution as a Helm chart package. Users can install or upgrade it with:
80+
81+
```bash
82+
helm install my-operator ./dist/chart
83+
```
84+
85+
Kubebuilder places CRDs in `templates/crd/` to ensure they upgrade with the controller. Helm has a built-in resource order that helps during installation.
86+
87+
During upgrades, if the CRD update fails or is slow to propagate, Helm may still update the Deployment.
88+
89+
Moreover, users can skip CRD updates with `helm upgrade --set crd.enable=false`.
90+
91+
<aside class="note">
92+
93+
Helm recommends a `crds/` directory, but CRDs there **never upgrade**. Kubebuilder uses `templates/crd/` to keep CRDs in sync with controllers but the order is **best effort**. See [Why CRDs are added under templates](../plugins/available/helm-v2-alpha.md#why-crds-are-added-under-templates) for details.
94+
95+
</aside>
96+
97+
<aside class="note">
98+
<h1>Upgrade and Lifecycle Considerations</h1>
99+
100+
During upgrades, CRD updates may not complete before the controller starts. This can happen when deployment tools apply resources without explicit ordering mechanisms.
101+
102+
CRD updates may hit errors or take time to propagate through the API server. If the controller Deployment updates at the same time, the new controller pod may start before the CRD update finishes. With strict validation enabled, controller writes will fail until the CRD is ready.
103+
104+
Ordering tools like ArgoCD sync waves or FluxCD dependsOn can help by controlling when resources are applied. However, these control timing, not whether the CRD update succeeds. If not properly configured or if the CRD apply encounters issues, the controller may still deploy.
105+
106+
For controllers that use CRDs from third-party operators (cert-manager, prometheus-operator, etc.), those CRDs have independent lifecycles. The third-party operator may upgrade its CRDs separately from your controller, which can lead to version mismatches.
107+
108+
</aside>
109+
110+
## Wiring an opt-in flag in cmd/main.go
111+
112+
This feature is **not scaffolded by default**. Follow these steps to add it manually.
113+
114+
### Step 1: Add the strictManager wrapper
115+
116+
In `cmd/main.go`, add this type definition after the `init()` function:
117+
118+
```go
119+
// strictManager wraps the manager to reject unknown fields instead of silently dropping them.
120+
// When the controller writes a field that doesn't exist in the CRD, the write fails immediately.
121+
// This helps catch typos and version mismatches between your code and cluster CRDs.
122+
type strictManager struct {
123+
ctrl.Manager
124+
strictClient client.Client
125+
}
126+
127+
func (m *strictManager) GetClient() client.Client {
128+
return m.strictClient
129+
}
130+
```
131+
132+
### Step 2: Add required imports
133+
134+
Add these imports to `cmd/main.go`:
135+
136+
```go
137+
import (
138+
// ... your existing imports ...
139+
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
140+
"sigs.k8s.io/controller-runtime/pkg/client"
141+
)
142+
```
143+
144+
### Step 3: Add the command-line flag
145+
146+
In the `main()` function, where other flags are defined, add:
147+
148+
```go
149+
func main() {
150+
var metricsAddr string
151+
var enableLeaderElection bool
152+
var probeAddr string
153+
var strictFieldValidation bool // Add this
154+
155+
flag.StringVar(&metricsAddr, "metrics-bind-address", ":8080", "...")
156+
flag.StringVar(&probeAddr, "health-probe-bind-address", ":8081", "...")
157+
flag.BoolVar(&enableLeaderElection, "leader-elect", false, "...")
158+
159+
// Add this flag
160+
flag.BoolVar(&strictFieldValidation, "strict-field-validation", false,
161+
"Reject unknown fields instead of dropping them.")
162+
163+
// ... rest of your code ...
164+
}
165+
```
166+
167+
### Step 4: Wrap the manager conditionally
168+
169+
After creating the manager with `ctrl.NewManager()`, add this wrapper logic:
170+
171+
```go
172+
mgr, err := ctrl.NewManager(ctrl.GetConfigOrDie(), ctrl.Options{
173+
Scheme: scheme,
174+
// ... your other options ...
175+
})
176+
if err != nil {
177+
setupLog.Error(err, "unable to start manager")
178+
os.Exit(1)
179+
}
180+
181+
// When enabled, the controller rejects writes with unknown fields instead of silently dropping them.
182+
var finalMgr ctrl.Manager = mgr
183+
if strictFieldValidation {
184+
finalMgr = &strictManager{
185+
Manager: mgr,
186+
strictClient: client.WithFieldValidation(
187+
mgr.GetClient(),
188+
metav1.FieldValidationStrict,
189+
),
190+
}
191+
}
192+
193+
// Use finalMgr for all subsequent setup
194+
if err := (&controller.MyReconciler{
195+
Client: finalMgr.GetClient(),
196+
Scheme: finalMgr.GetScheme(),
197+
}).SetupWithManager(finalMgr); err != nil {
198+
setupLog.Error(err, "unable to create controller", "controller", "My")
199+
os.Exit(1)
200+
}
201+
202+
// Continue using finalMgr for health checks, starting manager, etc.
203+
if err := finalMgr.AddHealthzCheck("healthz", healthz.Ping); err != nil {
204+
setupLog.Error(err, "unable to set up health check")
205+
os.Exit(1)
206+
}
207+
208+
if err := finalMgr.Start(ctrl.SetupSignalHandler()); err != nil {
209+
setupLog.Error(err, "problem running manager")
210+
os.Exit(1)
211+
}
212+
```
213+
214+
<aside class="note">
215+
<h1>Important: Use finalMgr everywhere</h1>
216+
217+
After wrapping the manager, use `finalMgr` instead of `mgr` for:
218+
- Controller setup
219+
- Webhook setup
220+
- Health checks
221+
- Starting the manager
222+
223+
This ensures all components use the wrapped client with strict validation.
224+
225+
</aside>
226+
227+
[client-field-validation-docs]: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/client#WithFieldValidation

0 commit comments

Comments
 (0)