Kafka ops: Migrating from ec2 classic to VPC

A few strategies for migrating Kafka to new cluster from classic to VPC:

Strategy 1:

Launch a new cluster (Zookeeper+Kafka brokers),
point producers to new cluster and drain the older cluster once consumers are also pointed to new cluster.

pros:

• Existing cluster isn't impacted, and hence helps in easy revert.
• Smooth transition on kafka cluster end. No config changes or zookeeper changes.

cons:

• In case of revert, potential loss of messages.
• Consumers may have to take some downtime.
• Few consumers consume and produce to the cluster. They've to engineer to adapt both kafka clusters in their application.

Strategy 2:

Launch a new cluster (Zookeeper + Kafka brokers),
point producers to both the clusters. Let data in both clusters be eventually consistent. Once both the clusters are identical, drain the older cluster.

pros:

• Minimal engineering required is on producers (also, those who produce by consumption).
• Dummy consumers can be put up, and testing can be done somewhat smoothly.

cons:

• Potential risk of duplicate consumption of messages.
• Application need changes.

Strategy 3:

Add new brokers one by one, drain one by one.
Once the brokers are migrated, move zookeeper similarly.

pros:

• applications are agnostic to these changes. No engineering changes required.

cons:

• Draining a broker would turn out expensive. Cluster would take significant bandwidth and throttle upon network, causing distress to applications.
  There's a way to put throttling limits on bandwidth and let migration happen in controlled manner.
• Operationally, lengthy procedure. Overall transition time would be in weeks.