MSK multi-region connectivity, IAM, and knowing when to stop

Managed Kafka across regions looks like a checkbox on a roadmap until you hit authentication modes, private connectivity, and the difference between “the cluster accepts IAM” and “something in another VPC can actually use it as a replication source.”

What we needed

Workloads in more than one region, with clusters that already spoke SASL in places and IAM in others. Cross-region tooling — specifically managed replication — wants multi-VPC private connectivity with IAM enabled on the source cluster. That is a cluster setting, not an app setting, and turning it on against a live cluster is slow enough that you plan for it.

What landed in Terraform

The shared Kafka cluster module grew explicit flags for VPC connectivity over IAM (and the matching port exposure). Callers that need the cluster as a cross-region source opt in; everyone else keeps the tighter surface.

Security groups already distinguished IAM, SASL, and TLS listeners. The connectivity work had to stay consistent with that split — enabling IAM connectivity without the listener and CIDR rules is a half-finished cluster that fails in confusing ways.

Alongside that, multi-region app config meant separate nonprod/prod shapes in different regions rather than pretending one workspace could own both. Aurora Serverless IAM auth in the same programme surfaced a different footgun: a module that creates a global IAM artefact will collide if another stack in the region already did.

The POC we removed

We stood up managed MSK Replicator as a cross-region path. It proved the connectivity story and also proved we did not want to operate that path yet — cost, failure modes, and operational ownership did not justify keeping it for the workload that needed it.

Removing the replicator after the POC is the part worth writing down. Shipping infrastructure because it works in a spike is how estates accumulate haunted units. The connectivity flags stayed; the replication topology did not.

What I’d check first next time

1. Whether the consumer of the cluster needs IAM multi-VPC connectivity at all, or only local IAM/SASL auth.
2. How long enabling connectivity takes on an existing cluster before you schedule it in a change window.
3. Whether “multi-region” means active-active replication or just the ability to stand the same stack up elsewhere — those are different Terraform problems.