Introduction
This document is intended to guide customers and Nasuni Service personnel in adding space and performance to the Nasuni Edge Appliance cache by configuring additional virtual cache disks that are striped together.
Note: Parts of this procedure can only be performed by Nasuni personnel.
Important: If this Nasuni Edge Appliance was originally deployed with a revision before 10.0, and if this Nasuni Edge Appliance has since updated to revision 10.0 or later, then you must reboot the Nasuni Edge Appliance before performing the manual striping procedure. For more information, see KB 0013612.
Striped Caches
With striped caches, the drives are “striped” together, instead of being concatenated together. Striping means that individual disk chunks (the "stripe size") are arranged to spread out data across all the disks, such that the first block lives on the first disk, the second block lives on the second disk, and so on.
As with scalable caches, this allows us to exceed the size limit of a single-disk cache. Striping gives a significant performance improvement because we can interact with many disks at once.
For example, if we are reading blocks 0-3 of a 4-disk stripe set, we can read all 4 blocks concurrently. The operating system kernel knows the dimensions (sometimes called the "shape") of the stripe set and automatically schedules the reads across the various devices to maximize performance.
The filesystem is also aware of the cache shape and arranges its internal data structures to spread out across the stripe set and not concentrate "hot" data structures on a single device. Performance is ideally that of all disks taken together, as all disks contribute to the overall IO.
Important: If the backend storage (datastore) is already striped, do not stripe the cache of the Nasuni appliance.
Important: If multiple cache disks are provided before the first boot, Nasuni automatically provisions them as a striped set for best performance.
EC2-based, GCP-based, and Nutanix-based Edge Appliances do not support automatic striping at first boot.
Multiple cache disks must all be the same size, or the size of the smallest cache disk determines how much of each disk can be used.
If you want to add an additional cache disk after the first boot, you have two alternatives:
First, create a new virtual machine with the required cache size, and then recover the existing Edge Appliance to this new Edge Appliance. This alternative is faster; however, all data in the original cache must be repopulated.
Instead, request Nasuni Support to add the new disk to the cache. The data in the original cache is unaffected.
This alternative can take a long time if performed at a low priority in order to avoid impacting users. If impacting users is not an issue, or if the procedure can be performed when users are not accessing the data, the process can be made faster.
Advantages to striped caches include the following:
New devices can be added so that customers can grow the cache as their needs grow. There is a per-hypervisor disk count limit, but it is large.
Existing devices can be expanded at any time, up to the per-hypervisor limit (but see the Drawbacks below).
Better performance can be achieved than with a single cache disk or a set of scalable cache disks.
Drawbacks include the following:
While existing devices can be expanded at any time, up to the hypervisor limit, all disks of the stripe set must be expanded equally to be useful. For example, a three-disk stripe set where two of the disks are 4 TB and one of the disks is 6 TB results in a 12 TB cache (4 TB x 3), and not a 14 TB cache, because 2 TB of the 6 TB disk cannot be used.
All new devices must be added in multiples of the stripe width. For example, if we start with a 3-disk stripe set, new devices must be added in multiples of 3.
Because the filesystem needs to be aware of the cache shape to lay out the internal data structures, the initial striping step is to format the cache. This means that the only way to stripe an existing Edge Appliance is to either perform a recovery procedure or request Nasuni Support to add the new disk to the cache.
The Edge Appliance must be offline during the striping operation.
Step 1: Communicate and plan
Before setting up a striped cache, the customer must decide how many stripes to use. This is referred to as an “n-way” stripe, where “n” is the number of stripes.
The customer should add enough disks to the Nasuni Edge Appliance so that there is one disk per desired stripe. The resulting cache size is that of all the disks added together. For example, if a customer wants a 4-TB cache made up of 4 stripes, they need to arrange for 4 1-TB disks on the Nasuni Edge Appliance.
Note that Nasuni Edge Appliances come with a single cache disk that can be resized. Therefore, customers do not need to add all the disks initially, because they can resize the existing single disk and then add more disks to fill the stripe set. Using the above example, the customer should resize the existing cache disk to 1 TB, and then add 3 more 1-TB disks to reach the total of 4 1-TB disks.
Step 2: Customer adds virtual cache disks
The customer adds a set of virtual disks to the virtual machine for the Nasuni Edge Appliance. The additional virtual disks should all be the same size. The customer can make the additional virtual disks small with the intention of growing them later or start with them already large.
Nasuni can provide the customer with the document “Adding Cache Disks” to
use as instructions for this procedure.
When the customer has added more virtual disks to the virtual machine, the customer should notify Nasuni Support or Professional Services that there is now more than one cache disk for the Nasuni Edge Appliance.
The customer should also enable Remote Support for the Nasuni Edge Appliance.
Important: Unlike the Scalable Cache procedure, which can be done at any time without data loss, this Striped Cache procedure involves reformatting the cache. The customer must ensure that all data snapshots to the cloud have been completed before proceeding.
Step 3: Shut down critical Nasuni Edge Appliance processes and reformat existing cache (Nasuni personnel)
Nasuni personnel shut down critical Nasuni Edge Appliance processes and reformat the existing cache.
This removes the existing cache from the Edge Appliance.
Step 4: Discover the added virtual cache disks (Nasuni personnel)
After the customer has added the virtual cache disks to the Nasuni Edge Appliance, Nasuni personnel log in to the Nasuni Edge Appliance using the support bridge. Nasuni personnel then identify the newly added virtual cache disks.
Step 5: Join the newly added virtual cache disks to the existing cache disks (Nasuni personnel)
After verifying the newly added device, Nasuni personnel join all the newly added devices to the existing cache disks.
Step 6: Create the new striped cache (Nasuni personnel)
Nasuni personnel create the new striped cache.
Step 7: Finish procedure
To finish the procedure, perform the following steps:
Reboot the Nasuni Edge Appliance.
If the previous Nasuni Edge Appliance was in Active Directory mode, you must re-join Active Directory. Similarly, if the previous Nasuni Edge Appliance was in LDAP mode, you must re-join LDAP.
At this point, the procedure is complete. Customers can continue to use the Nasuni Edge Appliance with the expanded cache.
Note: The usual rule for cache growth also applies here: you can only make cache disks larger and can never make cache disks smaller.