--- title: "Save the Address, Save the Cloud (KubeVirt VM Migration Story)" source: "https://www.tigera.io/blog/save-the-address-save-the-cloud-kubevirt-vm-migration-story/" --- [Technical Blog](https://www.tigera.io/category/technical-blog/) # Save the Address, Save the Cloud (KubeVirt VM Migration Story) By [Reza Ramezanpour](https://www.tigera.io/blog/author/rezar/) on Jul 08, 2026 • 9 min read Kubernetes is built for containers, and it’s been doing that since it used to run docker as an engine for its containers. But what if you want to add VMs to the mix? After all, containers are ephemeral and don’t require fixed IPs as they shift the identity toward labels, but VMs on the other hand are tied to IP addresses and in some cases MAC addresses. This brings us to this blog about [VM migration](https://www.tigera.io/learn/guides/vmware-migration/) and IP preservation. Unlike a pod that can be part of a deployment and run in a swarm of stateless endpoints, a VM is a stateful machine run by hypervisor like QEMU and extended to Kubernetes via KubeVirt Custom Resource Definitions (CRDs). ## There Is Something About KubeVirt [KubeVirt](https://www.tigera.io/learn/guides/kubevirt/) is an abstraction layer between the underlying hypervisor (QEMU) on your machine and Kubernetes. Its job is to manage a VM’s lifecycle and provide the necessary requirements for a VM to be a native resident in Kubernetes. These requirements are CPU, Memory, Networking, etc. KubeVirt does this by wrapping each VM in an ordinary Kubernetes pod called `virt-launcher`. Inside that pod, KubeVirt runs `libvirt` and QEMU, and the “VM” is really just a process scheduled, networked, and accounted for like any other pod. That detail matters a lot once we get to migration: when a VM moves to another node, what Kubernetes actually does is create a brand-new `virt-launcher` pod on the destination and tear down the old one. Everything hard about [live migration](https://www.tigera.io/learn/guides/vmware-migration/vmware-live-migration/) comes from making that pod swap invisible to the workload running inside. ### CPU CPU is the part that does the actual work, every instruction the guest operating system and its applications execute runs on a virtual CPU that KubeVirt maps onto real cores of the host node. You can pin the VM to dedicated cores, expose host CPU features, or let it float over shared cores. For migration, the CPU matters for a subtle reason: while a VM is being moved, its CPU keeps running and keeps changing memory. The faster the guest dirties memory, the harder it is to copy that memory to the other node before it changes again. We’ll come back to this race in a moment. ### Memory Other than being expensive these days, RAM or Memory has a crucial role in VM migration, since it is the place where everything that the CPU is working on is stored and referenced. In a physical computer, memory is the expensive stick that you buy and install in your computer. However, in a VM, memory is a region of your computer’s RAM allocated for the tasks that the VM is actively working on. Memory is the thing migration is really about. When [KubeVirt live-migrates](https://www.tigera.io/learn/guides/kubevirt/kubevirt-live-migration/) a VM, the bulk of what it ships from the source node to the destination node is the VM’s RAM, gigabytes of it, while the guest keeps running and keeps writing to it. ### Networking Another important part of a VM is networking, and KubeVirt supports multiple networking modes. Our focus is going to be on bridge, since that is required for VM migration with Calico, but if you’d like to learn more about other modes feel free to check out the official [KubeVirt documentation](https://kubevirt.io/user-guide/network/interfaces_and_networks/). #### What is a bridge? A bridge is similar to a playground where all the resources connected to it are able to communicate with each other. In Linux, a software bridge is a virtual switch: you plug interfaces into it and it forwards Ethernet frames between them just like a physical switch would. In KubeVirt’s `bridge` mode, the VM is connected to the pod network through a Linux bridge, and the pod’s IP address is handed down to the VM itself (via DHCP). The guest doesn’t get some separate, NAT’d address, it uses the pod IP directly as its own. If the relevant pod interface has a MAC address and the VM doesn’t override it, the VM inherits that MAC too. That “VM uses the pod IP directly” property is exactly why bridge mode is the only mode that works for live migration with Calico, and it’s the hinge the rest of this post turns on. ## How local live migration actually works Local live migration is the process of moving a running VM from one node to another within a cluster while the guest keeps running and stays reachable. No reboot, no shutdown, ideally the application inside never even notices. You start the process by posting a `VirtualMachineInstanceMigration` (VMIM) object, or just running `virtctl migrate vm1`, and KubeVirt does the rest. The default strategy is pre-copy, and it works roughly like this: - A new target VM (a fresh `virt-launcher` pod) is created on the destination node, while the source machine is still running. - The source starts streaming chunks of VM state, mostly RAM, to the target. This repeats: pages that the guest dirties while the copy is in flight get re-sent. - Once enough state has transferred that only a tiny delta remains, the guest is briefly paused, the last pages are shipped, and the guest resumes on the target. - The source VM is removed. Keep in mind that the VM migration is a race between the source and your network speed. If the source is copying memory to the target and the guest is simultaneously adding new things to that memory and this rate is faster than the network can copy them, the migration may never converge. KubeVirt has knobs for that (auto-converge throttles the guest CPU; post-copy runs the guest on the target immediately and faults memory across on demand), but for most workloads pre-copy just works, and that’s what we’ll see in the real report below. KubeVirt is deliberately conservative here. Out of the box it runs at most 5 migrations in parallel cluster-wide, no more than 2 outbound per node, and caps each migration at 64 MiB/s of bandwidth, so a busy cluster doesn’t saturate its own network moving VMs around. ## The hard part: the VM has to keep its IP Here’s the thing the KubeVirt docs gloss over and where Calico Unified Platform does the heavy lifting. Remember that a migration is really a pod swap, old `virt-launcher` pod on node A dies, and a new one on node B is born. Normally, a brand-new pod means a brand-new IP. For a stateless web pod behind a Service, who cares. For a VM, the IP is its identity, every open TCP connection, every client that resolves it, every firewall rule references that address. Change the IP and you’ve effectively killed and rebooted the machine, which is the exact opposite of “live.” So the job is: the new pod on node B must come up with the same IP the old pod had on node A, traffic must switch over to node B at exactly the right moment, and network policy has to already be in place on node B before that switch, otherwise the VM lands on the new node and gets firewalled off from its own connections. Calico coordinates all of this. Here’s how the pieces fit. ## Bridge mode is non-negotiable Because Calico’s IP persistence depends on the VM IP matching the pod IP, only bridge mode qualifies. Modes like `masquerade` give the VM a different internal IP than the pod and NAT between them, which breaks both IP persistence and policy enforcement during migration. KubeVirt actually refuses to migrate a bridge-mode VM unless you opt in with an annotation: ``` `apiVersion: kubevirt.io/v1 kind: VirtualMachine metadata: name: vm spec: template: metadata: annotations: kubevirt.io/allow-pod-bridge-network-live-migration: "" ...` ``` ## VM IP address persistence Calico keys its IP allocation off the VM’s identity rather than the pod’s. Internally, instead of the usual per-pod IPAM handle, the address is held under a VM-scoped handle like `k8s-pod-network.vmi.default.vm1`. When the target pod is created during migration, Calico’s CNI plugin recognizes it as a KubeVirt `virt-launcher` pod, looks up that VM handle, finds the existing IP, and reuses it instead of allocating a fresh one. This is a cluster-wide setting added in Calico v3.32 (`kubeVirtVMAddressPersistence`, enabled by default), and it’s mandatory for this type of migration. With persistence off, Calico rejects the migration target outright. ## Don’t NAT the VM on the way out If a VM talks to something outside the cluster and `natOutgoing` is enabled on its IP pool, the server on the other end sees the VM’s traffic as coming from the node’s IP. Migrate the VM and that source IP changes from node A’s address to node B’s, and any in-flight connection to that external server breaks. So for migratable VMs you disable `natOutgoing` on their pool, keeping the VM’s own IP on the wire end-to-end. ## Switching the traffic with BGP route priority Both nodes briefly believe they host the VM’s `/32` route. Calico breaks the tie with route priority: the target host programs the route with an elevated priority (a lower kernel metric, 512 by default) than the source’s normal priority (1024). Lower metric wins in the Linux kernel, so traffic steers to the target. Within a single AS this propagates automatically (Calico maps the kernel metric onto BGP `local_pref`); across eBGP rack boundaries you carry the signal with BGP communities via a `BGPFilter`. After a convergence window (~30s by default), priorities return to normal. ## Policy must land before the switch This is the subtle one. If the VM activated on the target node before its network policy was programmed there, it would arrive into a node that doesn’t yet know its firewall rules, and get cut off. Calico prevents this with an interlock: when the CNI plugin sets up the migration target pod, it returns the IP with empty routes, deliberately not programming the host-side routes that would pull traffic over. Felix only completes that switch once policy is in place on the destination (governed by `policy_setup_timeout_seconds`). Policy first, traffic second. ## Wrapping up Live migration looks like magic, a running machine teleports between hosts and nobody notices, but it’s really two systems cooperating very carefully. KubeVirt handles the compute side: racing the guest’s memory across the wire with pre-copy until it converges, then cutting the CPU over in under a second. Calico handles the network side: pinning the VM’s IP to the VM’s identity so the new pod reclaims it, withholding routes until policy is programmed on the destination, switching traffic with BGP route priority, and cleaning up dual ownership after a convergence window, all without the VM’s address ever changing. Get the networking mode wrong (anything but bridge), forget to disable `natOutgoing`, or skip the policy interlock, and “live” migration becomes a reboot with extra steps. Get them right, as the report above shows, and a stateful VM moves between physical machines while its TCP connections stay open and its identity stays put. Watch an interactive demo: [Live Migration of VMs Running on Kubernetes](https://app.arcade.software/share/YBtRqEnDgzll8GyFUDrh) [VM Migration](https://www.tigera.io/tags/vm-migration/)[Products](https://www.tigera.io/tags/products/) ## Related posts [![Meet Mylo: An AI-native way to work with Calico](https://www.tigera.io/app/uploads/2026/09/Meet-Mylo-An-AI-native-way-to-work-with-Calico.png)](https://www.tigera.io/blog/meet-mylo-an-ai-native-way-to-work-with-calico/) #### [Meet Mylo: An AI-native way to work with Calico](https://www.tigera.io/blog/meet-mylo-an-ai-native-way-to-work-with-calico/) By [Phil DiCorpo](https://www.tigera.io/blog/author/phil-dicorpo/) on Sep 3, 2026 A library of Calico tools and skills — delivered through the Calico MCP Server What if your hardest network question took ten minutes instead of ten days? Anyone who has operated Kubernetes networking at scale... [Read more](https://www.tigera.io/blog/meet-mylo-an-ai-native-way-to-work-with-calico/) [![The Safest Place to Run an AI Agent Is On a Cluster That Doesn’t Trust It](https://www.tigera.io/app/uploads/2026/08/The-Safest-Place-to-Run-an-AI-Agent-Is-On-a-Cluster-That-Doesnt-Trust-It.png)](https://www.tigera.io/blog/the-safest-place-to-run-an-ai-agent-is-on-a-cluster-that-doesnt-trust-it/) #### [The Safest Place to Run an AI Agent Is On a Cluster That Doesn’t Trust It](https://www.tigera.io/blog/the-safest-place-to-run-an-ai-agent-is-on-a-cluster-that-doesnt-trust-it/) By [Alister Baroi](https://www.tigera.io/blog/author/alister-baroi/) on Aug 27, 2026 Every organization running AI agents has already made a hosting decision. Most made it by accident. The sales team switched on the agent built into their CRM. Engineering is piloting a coding agent in a... [Read more](https://www.tigera.io/blog/the-safest-place-to-run-an-ai-agent-is-on-a-cluster-that-doesnt-trust-it/) [![AI Red Team Agents Automate Attacks on your AI Agents. Runtime Policies Automate their Defense.](https://www.tigera.io/app/uploads/2026/08/AI-Red-Team-Agents-Automate-Attacks-on-your-AI-Agents.-Runtime-Policies-Automate-their-Defense.png)](https://www.tigera.io/blog/ai-red-team-agents-automate-attacks-on-your-ai-agents-runtime-policies-automate-their-defense/) #### [AI Red Team Agents Automate Attacks on your AI Agents. Runtime Policies Automate their Defense.](https://www.tigera.io/blog/ai-red-team-agents-automate-attacks-on-your-ai-agents-runtime-policies-automate-their-defense/) By [Alister Baroi](https://www.tigera.io/blog/author/alister-baroi/) on Aug 24, 2026 The AI red teaming market grew up fast this year. OpenAI bought Promptfoo, Cisco and Microsoft shipped automated attack suites, and a seed-stage startup publicly compromised 50 of 55 live customer service bots. These platforms... [Read more](https://www.tigera.io/blog/ai-red-team-agents-automate-attacks-on-your-ai-agents-runtime-policies-automate-their-defense/)