Introduction
As a Nasuni customer, your data is stored and protected via the cloud. However, some customers might wish to supplement or leverage this cloud protection through services such as Azure GZRS, Azure Vaulted Backup, or other tooling. This guide covers a few of those products, but there are similarities between them, as there are between them and any other methods of asynchronous replication of cloud buckets.
Note: With the exception of Azure GRS/GZRS controlled failover, Nasuni recommends using these processes only in a complete disaster recovery scenario, as they require Nasuni’s direct involvement.
Using Azure GRS/GZRS
The following procedures are applicable to customers using Azure GRS or Azure GZRS. These storage classes offer asynchronous geo-replication. Customers do not need to account for separate credentials or an endpoint for the secondary copy. Azure manages that through the GRS and GZRS storage class service. This means that there is no need for Nasuni Support to change configuration settings.
Failover is triggered through the Azure portal.
Controlled Failover
Note: This procedure does not require Nasuni’s involvement.
A controlled failover ensures that unprotected data in the cache is protected.
The customer performs all parts of this procedure.
To perform an Azure controlled failover, follow these steps:
Log into the NMC.
On all Nasuni Edge Appliances connected to the volume, set all shares associated with the volume to read only.
Click the Volumes tab at the top.
Click the Shares option on the left side.
Under the Actions column, click the edit icon
.Check the Read Only option.
Click Update Share.
Note: If the NMC is unavailable or the NEA cannot connect to it, use the Edge Appliance UI. On the Volumes page, select the volume, click Total Shares, and then select Read Only Share.
Disconnect all users from the volume’s shares.
For CIFS shares, this can be done in the NMC UI by navigating to Filers > CIFS Clients, then clicking the Disconnect icon for the appropriate shares.
Perform a final snapshot on the volume.
In the Nasuni Management Console (NMC), on the Volumes page, locate the volume you want to snapshot. Under the Actions tab, click Take snapshot now.
Using the Azure portal, navigate to the storage account, select Redundancy, and click Prepare for failover.
After the GRS or GZRS service replication completes successfully, obtain the last successfully completed sync time and ensure it is later than the last Nasuni snapshot from step 1.c.
Click Failover and type yes to confirm.
When failover completes on all Nasuni Edge Appliances connected to the volume, perform these steps:
Convert all shares to the appropriate Read/Write access permission.
Test the snapshot process.
Uncontrolled Failover
Caution: An uncontrolled failover DOES NOT ensure that unprotected data in the cache is protected. Data loss is possible.
Caution: If the NEA that owns the volume associated with the uncontrolled failover also owns other volumes, those other owned volumes could also experience data loss.
The customer performs some parts of this procedure, while Nasuni Support performs others.
To perform an Azure uncontrolled failover, follow these steps:
Log into the NMC.
Click the Volumes tab, and disconnect each Edge Appliance from the shared volume.
On the Volumes tab, click Remote Access. On the Edge Appliance that owns the volume, disable remote access to the volume.
Obtain the most recent successfully completed GRS or GZRS sync time from your Azure account and provide it to Nasuni Support.
Note: The last sync time must be within the past 30 days.
Using the Azure portal, navigate to the storage account, select Geo-Replication, and click Prepare for failover.
Nasuni Support provides the Nasuni NOC team with the timestamp the customer provided.
When the failover process in step 4 completes, the customer performs a recovery on the Nasuni Edge Appliance that owns the volume, with assistance from Nasuni Support if necessary.
Re-share the volume and connect to other Nasuni Edge Appliances with appropriate share read/write access.
Test the snapshot process.
Nasuni Support tests the snapshot process on all Nasuni Edge Appliances.
Azure Vaulted Backup
Azure Vaulted Backup can be used to protect your storage accounts in the event of an extreme compromise. Customers need to create a separate credential and endpoint for the recovery copy.
Failover is triggered via the Azure Portal.
Important: Though Azure Vaulted Backup can go out much longer, and defaults to a period of 90 days, Nasuni can only recover to a period of 30 days.
Controlled Failover
Controlled failover via Azure Vaulted Backup is not currently recommended by Nasuni. If you have a use-case that requires it outside of a test environment, contact Nasuni.
Uncontrolled Failover
Caution: An uncontrolled failover DOES NOT ensure that unprotected data in the cache is protected. Data loss is possible.
Caution: If the NEA that owns the volume associated with the uncontrolled failover also owns other volumes, those other owned volumes could also experience data loss.
Note: If, due to an outage, the Primary NEA or any of the remote NEAs is not accessible, perform a recovery operation on the affected NEAs. After the affected NEAs are recovered, Nasuni Support can change the cloud endpoints as required. See NEA Recovery.
To perform the uncontrolled failover, follow these steps:
Log into the NMC.
Disconnect the volume from remote Edge Appliances.
On the Volumes tab, click Connect Volume.
Click Edit Connections for the appropriate volume.
Remove the checkmark to disconnect the Edge Appliance from the volume.
Click Save Connections to save and disconnect the Edge Appliances from the volume.
Disable remote access to the volume on the Nasuni Edge Appliance that owns the volume.
On the Volumes tab, click Remote Access.
Select the appropriate volumes from the list, then click Edit Volumes.
Toggle Enabled from On to Off.
Click Save Remote Access Settings to save the change to disable volume sharing.
Shut down the Edge Appliance that owns the volume.
Contact Nasuni Support immediately.
Obtain the time the last successfully completed backup was initiated and provide it to Nasuni Support.
Nasuni Support provides the Nasuni NOC team with the timestamp the customer provided.
Complete the restore of the Azure Backup vault to a new storage account via the steps mentioned in the link. Nasuni recommends creating a new storage account for this purpose and restoring all containers in it. Note the name and primary access key for the storage account.
Click the Account tab.
Click Cloud Credentials, and create a new credential for the storage account that you restored in the previous step.
Perform a disaster recovery for the Edge Appliance, with or without the help of support.
Turn on remote support for the new Edge Appliance.
From the NMC, click the Filers tab.
Under Filer Services on the left side, click Remote Support.
Check the Edge Appliances on the list, then click Edit Filers.
Click Enable Remote Support, enter the Timeout minutes, and click Save Settings.
Nasuni support asks you for the name of the new credential. They might also ask you for the access key and the credential UUID.
Nasuni notifies you when their side of the restore is complete. Following the notification, test the snapshot process on the newly recovered Nasuni Edge Appliance that owns the volume. Nasuni support can help you with this and the remaining processes.
If snapshots work on the owning Nasuni Edge Appliance, re-share the volume to the necessary Edges.
Re-enable Remote Access.
Click Volumes, then Remote Access.
Click the name of the Volume in question, then Edit Volumes.
Ensure the volume is set to Enabled.
Adjust Read/Write configuration if desired, then click Save Remote Access Settings.
Reconnect the volume to remote Edges.
Click Volumes, then Connect Volume.
On the relevant volume menu, click Edit Connections.
Click on the Edges the volume should be shared with.
Test the snapshot process on the reconnected Edge Appliances.
Third-Party Replication Tools
The following procedure applies to customers who use a third-party replication tool (rather than Azure's native GRS/GZRS) to replicate data between Azure Blob Storage accounts. Because Azure does not manage the replication in this scenario, the failover process might require Nasuni Support to manually update the cloud endpoint and, if applicable, the credentials on each Edge Appliance.
Failover is via a combination of the Azure portal and the 3rd-party tool, depending on the tool.
Controlled Failover
Important: This procedure is supported only if all Edge Appliances are running version 9.8 or later.
This process can be executed if the company's governing policy requires failover from the primary location to the recovery location at a specified frequency.
Tip: Nasuni recommends trying this process on a test system before performing the procedure on a production system.
The customer is responsible for ensuring that the third-party replication tool has successfully completed the sync between the primary and recovery locations before failing over. This process assumes that the user guarantees that replication has completed entirely successfully.
To perform the controlled failover, follow these steps:
Contact Nasuni Support and reference this procedure.
Make sure the replication destination storage account name and container are available for Nasuni Support.
If the cloud credentials differ from those of the destination storage account, add them to Nasuni via the NMC for all appliances connecting to the volume.
Enable Remote Support on the Owner NEA.
From the NMC, click the Filers tab.
Click Remote Support on the left side.
Check the Edge Appliances on the list, then click Edit X Filers.
Click Enable Remote Support, enter a Timeout, and click Save Settings.
Note: If, due to an outage, the Owner NEA or any of the remote NEAs is not accessible, perform a recovery operation on the affected NEAs. After the affected NEAs are recovered, Nasuni Support can change the cloud endpoints as required. See NEA Recovery.
On all remote Edge Appliances connected to a volume, perform these steps:
Convert all shares to read-only.
Click the Volumes tab.
Click Remote Access from the left side.
Check the Edge Appliances on the list, then click Edit X Volumes.
Select Read Only from the drop-down.
Click Save Remote Access Settings.
Disconnect all users from the affected volume across all Edge Appliances that share it. This ensures that any user who reconnects can no longer write new data.
From the NMC UI, click Filers, then CIFS Clients.
Take a final snapshot of all Edge Appliances that share the impacted volumes.
On the Volumes tab, locate the Volume List.
Under the Actions column, click the snapshot icon
.
Disconnect the volume from the remote Edge Appliances.
On the Volumes tab, click Connect Volume.
Click Edit Connections for the appropriate volume.
Remove the checkmark to disconnect the Edge Appliance from the volume.
Click Save Connections to save and disconnect the Edge Appliances from the volume.
On the Nasuni Edge Appliance that owns the volume, perform these steps:
Convert all shares to read-only.
Disconnect all users from the affected volume across all Edge Appliances that share it. You can use the Edge Appliance UI or the NMC for this. This prevents any user who reconnects from writing new data.
Perform final snapshots.
Un-share the volume (disable Remote Access).
This is done in the NMC:On the Volumes tab, click Remote Access.
Check the volumes on the list, then click Edit X Volumes.
Toggle the Enabled button from On to Off.
Click Save Remote Access Settings to save the change to disable volume sharing.
At this point, Nasuni Support must switch the storage account endpoint and region via Remote Support.
After step 4 is completed, on the owning appliance, make all shares read/write where appropriate.
Uncontrolled Failover
Important: Though your retention tool might be able to recover from longer time frame, Nasuni can only recover to a period of 30 days.
The following procedure applies to customers who use a third-party replication tool to replicate data between Azure Blob Storage accounts. This process is executed when failover is unplanned.
Caution: An uncontrolled failover DOES NOT ensure that unprotected data in the cache is protected. Data loss is possible.
Caution: If the NEA that owns the volume associated with the uncontrolled failover also owns other volumes, those other owned volumes could also experience data loss.
Note: If, due to an outage, the Primary NEA or any of the remote NEAs is not accessible, perform a recovery operation on the affected NEAs. After the affected NEAs are recovered, Nasuni Support can change the cloud endpoints as required. See NEA Recovery.
To perform the uncontrolled failover, follow these steps:
Log into the NMC.
Disconnect the volume from remote Edge Appliances.
On the Volumes page, click Connect Volume.
Click Edit Connections for the appropriate volume.
Remove the checkmark to disconnect the Edge Appliance from the volume.
Click Save Connections to save and disconnect the Edge Appliances from the volume.
On the Nasuni Edge Appliance that owns the volume, disable remote access to the volume.
On the Volumes page, click Remote Access.
Check the Edge Appliances on the list, then click Edit X Volumes.
Toggle ‘Enabled’ from On to Off.
Click Save Remote Access Settings to save the change to disable volume sharing.
Contact Nasuni Support immediately.
Obtain the last successfully completed sync time from your Azure account and provide that time to Nasuni Support. Verify that it matches what the third-party tool states.
Nasuni Support provides the Nasuni NOC team with the timestamp the customer provided.
Through the third-party restore process, restore the contents of the original storage account to the same or another Azure storage account.
When the restore process completes, the customer performs the Disaster Recovery process on the Nasuni Edge Appliance that owns the volume, with assistance from Nasuni Support if necessary.
Test the snapshot process on the newly recovered Nasuni Edge Appliance that owns the volume.
If snapshots work on the owning Nasuni Edge Appliance, re-share the Volume and connect to other Nasuni Edge Appliances with appropriate share read/write access.
Test the snapshot process on the reconnected Edge Appliances.