{"id":1698,"date":"2013-05-24T03:51:14","date_gmt":"2013-05-24T03:51:14","guid":{"rendered":"http:\/\/www.obieta.com\/?p=1698"},"modified":"2013-05-24T03:51:14","modified_gmt":"2013-05-24T03:51:14","slug":"resynchronization-of-virtual-machines-in-hyper-v-replica","status":"publish","type":"post","link":"https:\/\/obieta.com\/?p=1698","title":{"rendered":"Resynchronization of virtual machines in Hyper-V Replica"},"content":{"rendered":"

What is resynchronization and why is it needed?<\/h2>\n

Hyper-V Replica provides protection to VMs by tracking and replicating changes to the virtual hard disks (VHDs) of the VM. Hyper-V Replica runs 24 hours, 365 days in a year; for any VM that has been enabled for replication it ensures that the data on the primary site and the Replica site are kept as closely in sync as supported.<\/p>\n

To begin with, Hyper-V Replica (HVR) requires that the data on the virtual hard disks (VHDs) of the primary and replica VMs be the same. This is achieved through the process of initial replication, and establishes a baseline on which replicated changes can be applied. However, due to factors beyond the control of the administrator \u2013 such as faulty hardware and OS bugchecks \u2013 it is possible that the primary and Replica VMs are not in sync.<\/p>\n

Thus in a rainy day scenario (details in following section), when HVR determines that the replica VM can no longer be kept in sync with the primary by applying the replicated changes then resynchronization is required. Resynchronization (or Resync) is the process of re-establishing the baseline \u2013 by ensuring that the primary and replica VHDs have exactly the same data stored.<\/p>\n

(NOTE: In this post we will use a VM named \u201cRESYNC VM\u201d in all examples and screenshots.)<\/em><\/p>\n

 <\/p>\n

 <\/p>\n

When does resynchronization happen?<\/h2>\n

It would become quite obvious after going through this table below that Resync is not expected to occur regularly. In fact, in the normal course of replication this is quite a rare event. The VM enters the\u00a0\u201cResynchronization Required\u201d<\/em> state when any one of the conditions are encountered:<\/p>\n\n\n\n\n\n\n\n\n\n
\n

Site<\/h4>\n<\/td>\n

\n

Condition<\/h4>\n<\/td>\n

\n

Scenario example<\/h4>\n<\/td>\n<\/tr>\n

\n

Primary<\/p>\n<\/td>\n

\n

Modify VHD when VM is turned off<\/p>\n<\/td>\n

\n

Mount\/modify VHD outside the VM, Edit disk, Offline patching<\/p>\n<\/td>\n<\/tr>\n

\n

Primary<\/p>\n<\/td>\n

\n

Size of tracking log files > 50% of total VHD size for a VM<\/p>\n<\/td>\n

\n

Network outage causes logs to accumulate<\/p>\n<\/td>\n<\/tr>\n

\n

Primary<\/p>\n<\/td>\n

\n

Write failure to tracking log file<\/p>\n<\/td>\n

\n

VHD and logs are on SMB and connectivity to the SMB storage is flaky.<\/p>\n<\/td>\n<\/tr>\n

\n

Primary<\/p>\n<\/td>\n

\n

Tracking log file is not closed gracefully<\/p>\n<\/td>\n

\n

Host crash with primary VM running. Applicable to VMs in a cluster also.<\/p>\n<\/td>\n<\/tr>\n

\n

Primary<\/p>\n<\/td>\n

\n

Reverting the volume to an older point in time<\/p>\n

Reverting the VM to an older snapshot<\/p>\n<\/td>\n

\n

Volume\/snapshot backup and restore<\/p>\n<\/td>\n<\/tr>\n

\n

Secondary<\/p>\n<\/td>\n

\n

Out-of-sequence or Invalid log file is applied<\/p>\n<\/td>\n

\n

Restoring a backed-up copy of the Replica VM<\/p>\n

Importing an older VM copy, when migration by using export-import<\/p>\n

Reverting volume to an older point in time using Volume backup and restore.<\/p>\n

Reverting the VM to an older snapshot<\/p>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

When the VM enters the\u00a0\u201cResynchronization Required\u201d<\/em> state, the replication health becomes \u201cCritical\u201d and the VM is scheduled for resynchronization. At the same time, HVR stops tracking the guest writes for the VM and nothing is replicated.<\/p>\n

The replication health will also show this message:<\/p>\n

\"resync<\/a><\/p>\n

 <\/p>\n

 <\/p>\n

 <\/p>\n

Initiating and scheduling resynchronization<\/h2>\n

Depending on the VM setting, the user might have to trigger the resynchronization operation explicitly. When that is required, follow the instructions as given in the replication health screen:<\/p>\n

    \n
  1. Right-click on the VM for the options<\/li>\n
  2. Under\u00a0Replication<\/strong>, select the\u00a0Resume Replication<\/strong> option<\/li>\n<\/ol>\n

    You will be presented with the screen to schedule the resynchronization operation:<\/p>\n

    \"resync<\/a><\/p>\n

    To start the resync operation from PowerShell, use the\u00a0Resume-VMReplication<\/strong><\/a> commandlet:<\/p>\n

    \n
    \n
    Resume-VMReplication \u2013VMName \u201cRESYNC VM\u201d -Resynchronize \u2013ResynchronizeStartTime \u201c04\/15\/2013 12:00:00\u201d<\/pre>\n<\/div>\n<\/div>\n
     <\/p>\n
    User-initiated resynchronization is also possible, but unless absolutely necessary it should be avoided. In order to explicitly force resynchronization on a VM that is not in the\u00a0\u201cResynchronization Required\u201d<\/em> state, first suspend the replication and then initiate resync:<\/p>\n
    \n
    \n
    Suspend-VMReplication -VMName \"RESYNC VM\"<\/pre>\n
    Resume-VMReplication -VMName \"RESYNC VM\" -Resynchronize<\/pre>\n<\/div>\n<\/div>\n
     <\/p>\n
    The scheduling of the resynchronization operation can be configured for each VM:<\/p>\n
      \n
    1. On the primary site, open the Hyper-V Manager<\/li>\n
    2. Right-click on the desired VM, and select the\u00a0Settings\u2026<\/strong> option<\/li>\n
    3. In the left hand pane under\u00a0Replication<\/strong>, select the\u00a0Resynchronization<\/strong> option<\/li>\n<\/ol>\n
      \"resync<\/a><\/p>\n
      The default option is to schedule the resynchronization operation during off-peak hours. The resource intensive nature of the operation makes such scheduling useful, and aims to reduce the impact on running VMs.<\/p>\n
      The same can be configured in PowerShell using the\u00a0Set-VMReplication<\/strong><\/a> commandlet:<\/p>\n
      \n
      \n
      # Manual resync<\/pre>\n
      Set-VMReplication -VMName \"RESYNC VM\" -AutoResynchronizeEnabled 0<\/pre>\n
      \u00a0<\/pre>\n
      # Automatic resync<\/pre>\n
      Set-VMReplication \u2013VMName \"RESYNC VM\" -AutoResynchronizeEnabled 1 -AutoResynchronizeIntervalStart 00:00:00 -AutoResynchronizeIntervalEnd 23:59:59<\/pre>\n
      \u00a0<\/pre>\n
      # Scheduled resync<\/pre>\n
      Set-VMReplication \u2013VMName \"RESYNC VM\" -AutoResynchronizeEnabled 1 -AutoResynchronizeIntervalStart 00:00:00 -AutoResynchronizeIntervalEnd 06:00:00<\/pre>\n<\/div>\n<\/div>\n
       <\/p>\n
      To see the resynchronization settings in PowerShell, use the\u00a0Get-VMReplication<\/strong><\/a> commandlet and look for theAutoResynchronizeEnabled<\/em>,\u00a0AutoResynchronizeIntervalStart<\/em>, and\u00a0AutoResynchronizeIntervalEnd <\/em>fields:<\/p>\n
      \n
      \n
      Get-VMReplication -VMname \"RESYNC VM\" | fl *<\/pre>\n<\/div>\n<\/div>\n
       <\/p>\n
       <\/p>\n
       <\/p>\n
       <\/span><\/p>\n
       <\/span><\/p>\n
      The process of resynchronization<\/h2>\n
      When the resync operation is triggered \u2013 either automatically or by the user \u2013 the following high-level sub-operations are executed in sequence:<\/p>\n
        \n
      1. Check the VHD characteristics of primary and replica VMs:<\/strong> before resync can be done, these have to match. Hyper-V Replica checks the geometry and size of the disk before starting resync. Top on the list of exceptions to watch out for are size mismatches \u2013 caused by resizing either a primary or replica VHD without appropriately resizing the other one.<\/li>\n
      2. Start tracking the VHDs:<\/strong>\n
          \n
        1. The guest writes are tracked into the log file, but these changes are not replicated until resync is completed.<\/li>\n
        2. It is important to note that if resync takes too long then you might hit the \u201c50% of total VHD size for a VM\u201d condition and end up sending the VM into the\u00a0\u201cResynchronization Required\u201d<\/em> state again.<\/li>\n
        3. Event number 29242 is logged that specifies the VM, VHDs, start block, and end block.<\/li>\n<\/ol>\n<\/li>\n
        4. Create a diff disks for the replica VHDs:<\/strong> this allows the resync operation to be cancelled without leaving the underlying VHD in an inconsistent state. The diff disk with all the resync-ed changes is then merged back into the VHD at the end of the resync operation.<\/li>\n
        5. Compare and sync the VHDs:<\/strong> the comparison of the VHDs is done block-by-block and only the blocks that differ are sent across the network. This can reduce the data sent over the network, depending on how different the two VHDs are. While this operation is going on:\n
            \n
          1.  <\/em>Pause Replication<\/em> <\/strong>will stop the current resync operation. Doing\u00a0Resume Replication<\/em> later will continue the resync comparisons from where it left off.<\/li>\n
          2. Planned failover or Test failover will not be possible.<\/li>\n
          3. At any point the user can always do Unplanned Failover, but this will cancel the resync operation.<\/li>\n
          4. Resync can be cancelled at any point. This will keep the VM in the\u00a0\u201cResynchronization Required\u201d<\/em> state, and the next time replication is resumed, it will start from the beginning.<\/li>\n<\/ol>\n<\/li>\n
          5. Completion of compare and sync:<\/strong> HVR logs event number 29244 once the compare and sync operation is done, and it specifies the VHD, VM, blocks sent, time taken, and result of the operation.<\/li>\n
          6. Merge the resync changes to the VHD: <\/strong>after this operation completes, the resync operation cannot be cancelled or undone.<\/li>\n
          7. Delete the recovery points:<\/strong> this is a significant side-effect of resync. The recovery points are built upon the VHD as a baseline. However, resync effectively changes that baseline and makes the data stored in those recovery points invalid. After resync completes, the recovery points are built again over a period of time.<\/li>\n<\/ol>\n
             <\/p>\n
             <\/p>\n
            Resynchronization performance<\/h2>\n
            Resynchronization performance was tested and compared against the performance of Online Initial Replication (IR). The setup consisted of a standalone server with 4 running VMs \u2013 2 File Servers and 2 SQL servers running typical workloads. Two VMs were replicated to a standalone Replica server. The network bandwidth was varied to see the impact. Data size that was replicated during Online IR was approximately 80GB.<\/p>\n
             <\/strong> <\/strong><\/strong><\/strong><\/p>\n\n\n\n\n\n\n\n\n
            <\/td>\nNetwork speed<\/strong><\/td>\nOnline IR size<\/strong><\/td>\nOnline IR time<\/strong><\/td>\nResync size<\/strong><\/td>\nResync time<\/strong><\/td>\n<\/tr>\n
            Resync \u2013 offline scheduling<\/td>\n1 Gbps<\/td>\n~80 GB<\/td>\n~1.5 hrs<\/td>\n~5.5 GB<\/td>\n~2 hrs<\/td>\n<\/tr>\n
            Resync \u2013 immediate<\/td>\n1 Gbps<\/td>\n~80 GB<\/td>\n~1 hr<\/td>\n~100 MB<\/td>\n~1 hr<\/td>\n<\/tr>\n
            <\/td>\n<\/td>\n<\/td>\n<\/td>\n<\/td>\n<\/td>\n<\/tr>\n
            Resync \u2013 offline scheduling<\/td>\n1.5 Mbps<\/td>\n~80 GB<\/td>\n4 days<\/td>\n~10 GB<\/td>\n~1 day<\/td>\n<\/tr>\n
            Resync \u2013 immediate<\/td>\n1.5 Mbps<\/td>\n~80 GB<\/td>\n4 days<\/td>\n~ 78 MB<\/td>\n~1 hour<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n
            The tests indicate that resync is preferable to Online IR in low speed networks. When the two sites are connected by a high speed network, resync works well for low churn workloads.<\/p>\n
            There is also a perfmon counter for measuring the resynchronized bytes:\u00a0\u00a0Hyper-V Replica VMResynchronized Bytes<\/strong>.<\/p>\n
             <\/p>\n
            Conclusion<\/h2>\n
            The disks going out of sync is a rainy-day event in Hyper-V Replica. However with the Resynchronization operation, this is handled gracefully within the product to optimize the administrative overhead and the resources used in bringing the disks back into sync.<\/p>\n","protected":false},"excerpt":{"rendered":"
            What is resynchronization and why is it needed? Hyper-V Replica provides protection to VMs by tracking and replicating changes to the virtual hard disks (VHDs) of the VM. Hyper-V Replica runs 24 hours, 365 days in a year; for any VM that has been enabled for replication it ensures that the data on the primary […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":[],"categories":[1,8],"tags":[],"_links":{"self":[{"href":"https:\/\/obieta.com\/index.php?rest_route=\/wp\/v2\/posts\/1698"}],"collection":[{"href":"https:\/\/obieta.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/obieta.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/obieta.com\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/obieta.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=1698"}],"version-history":[{"count":0,"href":"https:\/\/obieta.com\/index.php?rest_route=\/wp\/v2\/posts\/1698\/revisions"}],"wp:attachment":[{"href":"https:\/\/obieta.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=1698"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/obieta.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=1698"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/obieta.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=1698"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}