This isn’t your granddad’s NGINX. We’re talking Kubernetes ingress controllers, a digital plumbing system for your applications. And apparently, not all pipes are created equal. Earlier this month, three production AKS clusters — handling everything from compliance APIs to frantic trading servers — underwent a surgical migration. Off went the aging community <a href="/tag/ingress-nginx/">ingress-nginx</a> controller, on came the F5 NGINX Ingress Controller OSS. The headline? It’s the same controller name, but entirely different insides. Enough to make you want a strong coffee. Or maybe just a stiff drink.
The Case for Change
The practical trigger for this overhaul was less a single catastrophic failure and more a slow creep of annoyance. The community chart had gathered dust, accumulating workarounds for bugs long past their expiration date. Auditing annotations — those little Kubernetes metadata tags that control ingress behavior — had become a chore. And then there’s the persistent itch for consistency: a single, unified ingress stack across all clusters. Why have three different ways of doing the same thing when you can have one? (Spoiler: it’s rarely as simple as it sounds).
The F5 NGINX Ingress Controller OSS is the upstream-maintained version. It aligns with NGINX OSS releases. It offers tighter control over NGINX configuration, bypassing the community’s annotation translation layer. This is the crux of the matter. It’s about having a direct line to the engine, not talking through a translator who might misinterpret your instructions.
What Stayed the Same (Blessedly)
Before we dissect the carnage, let’s acknowledge what didn’t break. This is crucial for anyone contemplating a similar move. Your IngressClass name, thankfully, remains nginx across all clusters. No application-level chaos. The Azure Load Balancer type — internal or public — is also untouched. Cert-manager ClusterIssuers? Mostly fine, with just one field rename to worry about. And Linkerd injection on the controller pods? Still plugging away. These constants are the lifelines in a sea of change.
The Migration Pipeline: A Five-Step Tango
Every cluster followed the same five-step dance. It’s neat, it’s tidy, and it’s supposed to be idempotent. Let’s look at the choreography:
# 1. Pull the F5 chart via OCI — no helm repo add needed
helm pull oci://ghcr.io/nginx/charts/nginx-ingress \
--version 2.5.1 \
--destination /tmp/charts/
# 2. Verify checksum before touching anything
echo "23c866c0531719586570435a4d9a57ac0fb9661fdafd572c8916208cb7b4f225 /tmp/charts/nginx-ingress-2.5.1.tgz" \
| sha256sum --check
# 3. One-time IngressClass migration guard
CONTROLLER=$(kubectl get ingressclass nginx \
-o jsonpath='{.spec.controller}' 2>/dev/null || true)
if [ "${CONTROLLER}" = "k8s.io/ingress-nginx" ]; then
echo "Removing community IngressClass — allowing F5 takeover"
kubectl delete ingressclass nginx
fi
# 4. Helm upgrade
helm upgrade --install nginx-ingress /tmp/charts/nginx-ingress-2.5.1.tgz \
--namespace nginx-ingress \
-f values.yaml \
--wait --timeout 5m
# 5. Verify the right controller is running
kubectl get pods -l app.kubernetes.io/name=nginx-ingress -n nginx-ingress
The IngressClass Conundrum
Step 3 is where the real fun begins. The spec.controller field on an IngressClass resource is immutable. Once set, it’s for life. The community controller stamps it with k8s.io/ingress-nginx. The F5 controller, naturally, expects nginx.org/ingress-controller. Just running helm upgrade without preamble means F5 won’t claim the existing IngressClass. It’ll either create a duplicate, which is a recipe for confusion, or just… ignore everything. Not ideal.
The solution? Delete the IngressClass before the F5 install. But here’s the twist: you can’t just blindly delete it. Run the pipeline again after the migration? Boom. You just deleted the active F5 IngressClass mid-flight. Brief outage, anyone? The guard condition saves the day:
if [ "${CONTROLLER}" = "k8s.io/ingress-nginx" ]; then
kubectl delete ingressclass nginx
fi
After the first successful F5 deployment, the spec.controller field correctly reads nginx.org/ingress-controller. Any subsequent pipeline run gracefully skips the deletion. It’s a one-time, idempotent, safe maneuver. Clever.
Configuration Keys: A Minefield of Renames and Removals
The community chart keeps its configuration keys in a flat controller.config map. F5, bless its heart, nests everything under controller.config.entries. A small change, a colossal gotcha if you’re blindly copying and pasting your old values.yaml.
Community:
controller:
config:
proxy-read-timeout: "600"
load-balance: "ewma"
use-gzip: "true"
F5:
controller:
config:
entries:
proxy-read-timeout: "600s" # note: F5 expects the unit suffix
lb-method: "ewma" # key renamed
# use-gzip has no equivalent — moved to http-snippets
And the hits keep coming. Numerous community config keys simply don’t exist in F5. They’re silently ignored if you leave them in, which is arguably worse than an error. Auditing is paramount. The author painstakingly cross-referenced every key against the F5 config documentation, axing keys like allow-snippet-annotations, enable-vts-status, and use-geoip to name a few. Others, like load-balance, were simply renamed to lb-method. And proxy-read-timeout now requires a unit suffix — 600s instead of just 600. A minor detail that can cause significant downtime.
Full Base Controller Configuration
Here’s a peek at the base controller configuration adopted across all three clusters post-migration:
controller:
kind: deployment
enableCustomResources: false # not using VirtualServer CRDs
enableSnippets: true
telemetryReporting:
enable: false # no outbound access to oss.edge.df.f5.com
ingressClass:
name: nginx
create: true
setAsDefaultIngress: false
service:
annotations:
service.beta.kubernetes.io/
So, Is It Better?
The F5 NGINX Ingress Controller OSS offers a more direct and controlled path for configuring NGINX within Kubernetes. This migration, while fraught with the usual YAML-tweaking and existential dread, seems to have paid off. The move to an upstream-maintained controller simplifies dependency management and ensures you’re running closer to the source of truth for NGINX itself. For operations teams, especially those managing multiple clusters, the promise of a consistent ingress stack is a significant win. It trims the fat from accumulated workarounds and brings a sharper, more auditable configuration surface. It’s not a magic bullet, but it’s a solid step towards sanity in complex Kubernetes deployments.
🧬 Related Insights
- Read more: PyTorch 2.11: 2723 Commits Later, FlashAttention Speeds Up — But TorchScript’s Dead
- Read more: Node.js 25.9.0 Ignites Developer Velocity: New Tools Emerge
Frequently Asked Questions
What’s the difference between community ingress-nginx and F5 NGINX Ingress Controller? The community version is a Kubernetes community project, while the F5 version is the upstream-maintained controller by F5, aligned with NGINX OSS releases, offering tighter configuration control.
Will this migration break my applications? It can. Configuration differences, especially in annotation prefixes and key names, require careful auditing and adjustment. The migration pipeline includes steps to mitigate risks, but thorough testing is essential.
Is it worth switching from the community controller? For environments prioritizing direct NGINX configuration control, a unified ingress stack across clusters, and access to upstream NGINX features, the F5 controller can be a worthwhile migration. However, it demands a careful re-evaluation of your existing ingress configurations.
