Cross-vCenter migration of vSphere Encrypted VMs

During a recent consolidation prospect I was faced with the challenge of migrated encrypted VMs from one vCenter to another.

For the non-encrypted VMs this is not a problem- the excellent Cross vCenter vMotion tool worked very well live migrating from vSphere 6.0/6.7 environments to the new 6.7 this is now built in to the latest vSphere). However VMs using vSphere encryption for encryption at rest can’t use this method.

The reason behind this is that the encryption key is in essence formed by the pairing of the vCenter and the Key Management System (KMS). Even though in my case we’re using the same KMS for both the old and new vCenters, that paring can’t be assured to exist in the new environment. The documentation (dated May 31st 2019) says this isn’t currently supported, so this may change in the future.

Decrypting the VM (and then vMotion-ing) is technically possible, but requires potentially lengthy downtime for the decrypt/encrypt operations and also is not acceptable from a security point of view. We’re encrypting these workloads for a reason, so leaving them unencrypted at rest whilst we do this shuffle is not an option.

Solution

So a live vMotion was out of the question in my scenario, but the solution I put together required only a small bit of downtime and kept the encrypted status of the VM. The answer was to leverage the existing backup technology.

This environment is using Commvault for virtual machine backup (other providers are available and the method would still apply), and these VMs were getting encrypted backup. This meant we could use this technology to keep that encrypted status, but also reduce the downtime because we only needed to take an incremental copy, not copy off the entire virtual disk.

These are the steps I followed:

  1. Make sure the VM is backing up as normal. Plan in the downtime window (you can get an idea of the length of time required in advance by running steps 5-6 without powering on the target)
  2. Set the default storage policy on the target datastore to an encrypted one.
  3. Create a new (encrypted) target VM on the target vCenter. Create it with the same specification (CPU/ memory) as the source, but without any hard disks.
  4. Shutdown the source VM
  5. Take the final (incremental) backup of the source VM.
  6. Restore the virtual disk(s) from the backup to the new target VM.
  7. Power on and test the target VM
  8. Mark up the source VM so it isn’t accidentally powered on, or remove it altogether.