Introduction
Nasuni enables enterprises to store and synchronize files across all locations at any scale. Powered by the Nasuni UniFS® global file system, the Nasuni® file services platform stores unstructured data in private or public cloud object storage from providers such as Amazon, Dell EMC, IBM, WD ActiveScale, Hitachi Vantara, and Microsoft, while intelligently caching actively-used data on virtual or hardware Nasuni Edge Appliances for high performance access. Nasuni serves a variety of use cases, including NAS/file server consolidation, multi-site collaboration, business continuity, digital transformation, and active archiving.
External auditing solutions can help to:
Protect enterprise data such as: sensitive files and emails; confidential customer, patient, and employee data; financial records; strategic and product plans; and other intellectual property.
Detect threats and cyberattacks by analyzing data, account activity, and user behavior.
Support data governance, compliance, classification, and threat analytics.
This document discusses the deployment considerations and the best practices around them as it relates to external auditing solutions, when paired with the Nasuni cloud-based file architecture.
Deployment
Typically, Nasuni Edge Appliances are deployed at the edge to provide end users with high-performance access for their actively-used data. Since the Nasuni UniFS® global file system stores the authoritative copy of all files and metadata in private or public cloud object storage platforms, special considerations are necessary for use cases that require scanning that content.
Nasuni strongly recommends deploying a dedicated Nasuni Edge Appliance with which the external auditing solution interacts. By deploying a dedicated Nasuni Edge Appliance along with an external auditing solution local to the region where the authoritative copy resides, the need for transferring the entire data set across a wide area network to the edge is eliminated. This reduces the duration of scans as needed by the external auditing solution use case. This architecture also minimizes any egress fees charged by the public cloud provider, because the data transfer between the Nasuni Edge Appliance and the external auditing solution occurs within the same region. Using a dedicated Nasuni Edge Appliance also allows the administrator of the external auditing solution to run scans during normal business hours without fear of impacting end-user performance.
Figure 1: Nasuni and External Solution Architecture.
Considerations
Nasuni Filer and External Auditing Solution ratio
Traditionally, siloed NAS architectures required a one-to-one relationship between the NAS device and an external auditing solution in order to avoid inefficient scans across wide area networks.
The Nasuni UniFS® global file system eliminates the need to deploy an external auditing solution along with every Nasuni Edge Appliance, since changes to data on volumes that are shared across Nasuni Edge Appliances are propagated to all connected appliances.
In addition, every Nasuni Edge Appliance is capable of sending file system audit events via the Nasuni Auditing API an API back to the dedicated external auditing solution. The external auditing solution then analyzes these audit events. If a threat is detected, the external auditing solution can then generate administrative alerts and take proactive actions to lock down data or users. These audit events can also be used by the external auditing solution to identify which files and folders have been modified and perform incremental scans of just those items. See the Nasuni Management Console Guide for details about configuring file system auditing.
Public Cloud Networking
The Nasuni Edge Appliance and external auditing solution virtual machines require connectivity to infrastructure resources. Both the Edge Appliance and external auditing solution require access to Active Directory domain controllers. The external auditing solution requires access to the external auditing solution server. If these infrastructure resources are deployed solely on-premises, then a trusted network path, in the form of a VPN or direct connection, must be created between the public cloud and your datacenters. Alternatively, infrastructure resources that already exist in the public cloud can be used by the Edge Appliance and the external auditing solution.
Multi-Region Deployments
When data is hosted across multiple cloud service provider regions, it is recommended that a dedicated Nasuni Edge Appliance and external auditing solution be deployed in each region. This ensures the best performance, because the Nasuni and external auditing solution VMs are close to the data. It also minimizes egress fees. In addition, this arrangement addresses data sovereignty concerns, such as those of the European Union’s General Data Protection Regulation (GDPR), by ensuring that content scanning happens within a particular region.
In a multi-region scenario that does not involve data sovereignty concerns, and where cost takes precedence over scan duration, it can be more cost-effective to use a single Nasuni Edge Appliance and external auditing solution pair to scan multiple volumes/regions. Each cloud service provider has different costs associated with data transfers. Consideration should be given to how much data is scanned by the external auditing solution and, thus, how much data traverses the cloud service provider’s network. In the case of external auditing solution, which only involves the scanning of metadata, egress fees are almost always less than the cost of deploying multiple VMs in each region. However, when certain external auditing solutions are used that involve scanning file contents, significantly more data can be involved in each scan. The major cloud service providers offer cost calculators that can be used to determine the break-even point for data being scanned across regions vs. the cost of deploying additional VMs.
Network Architecture
In a multi-region deployment, the Edge Appliances and external auditing solutions in each region need to be able to communicate with Active Directory domain controllers and the external auditing solution server. This communication can be routed across region-specific secure connections between a regional office and the cloud service provider, or across the cloud service provider’s backbone to a shared secure connection at a single customer location.
The specific network configurations for each scenario vary depending on the cloud service provider. Please consult your provider’s documentation for the latest deployment guidance.
Run External Auditing Solution After Metadata Is Pulled into Cache
Before the initial scan of a Nasuni volume by the external auditing solution, ensure that metadata for the volume has been pulled into the cache completely by using the Nasuni File Browser. After the initial scan, metadata changes are minimal, and follow-on scans can automatically trigger the download of the incremental changes.
To bring metadata into the cache, follow these steps:
Log into the Nasuni Management Console (NMC).
Click Volumes.
Click File Browser in the left-hand column.
From the Volume drop-down list, select the volume to scan.
From the Filer drop-down list, select the Nasuni Edge Appliance closest to the external auditing solution.
In the Version drop-down list, ensure that “Current Version” is selected.
In the Volume Actions area, click “Bring into Cache”. The “Bring Volume Into Cache” dialog box appears.
Select “Bring Metadata Only”.
Important: If you do not select “Bring Metadata Only”, the Nasuni Edge Appliance starts downloading all of the data on the volume into the cache.
Click “Start Transfer”. This begins the process of copying metadata into the local cache of the Nasuni Edge Appliance.
Monitor the Notifications on the NMC for messages indicating that metadata is being brought into cache and that the job is complete. The message is of the form, “Metadata for entire volume <volume_name> has been successfully brought into cache.”
Important: This message indicates that the Nasuni Edge Appliance has finished downloading the metadata associated with the volume. However, it is possible that some directories might have been skipped. Nasuni Support can review system logs to determine whether any directories have been skipped.
Using the NMC API
You can bring metadata into the cache using the NMC API. By default, both the metadata and data for the specified path are brought into the cache. Bringing only the metadata into cache is an option if $MetadataOnly is set to "true".
Required Inputs: NMC hostname, username, password, volume_guid, filer_serial, path, metadata only, force
Compatibility: Nasuni 8.5 or higher required
Script Name: BringPathIntoCache.ps1
#Bring the specified path into cache
#populate NMC hostname and credentials
$hostname = "insertNMChostnameHere"
#username for AD accounts supports both UPN (user@domain.com) and DOMAIN\\samaccountname formats (two backslashes required ).
#Nasuni Native user accounts are also supported.
$username = "username"
$password = "password"
$credentials = '{"username":"' + $username + '","password":"' + $password + '"}'
#specify Edge Appliance and Volume
$volume_guid = "InsertVolumeGuid"
$filer_serial = "InsertFilerSerial"
#Set the path to bring into cache. The path should start with a "/" and is the path as displayed in the file browser
# and is not related to the share path.
$FolderPath = "/Insert/path/here"
#Specify “true” if only metadata should be brought into cache
$MetadataOnly = "true"
#Specify if available cache space should be ignored for the request, usually a bad idea
$Force = "false"
# Allow untrusted SSL certs
if ("TrustAllCertsPolicy" -as [type]) {} else {
Add-Type -TypeDefinition @"
using System.Net;
using System.Security.Cryptography.X509Certificates;
public class TrustAllCertsPolicy : ICertificatePolicy {
public bool CheckValidationResult(
ServicePoint srvPoint, X509Certificate certificate,
WebRequest request, int certificateProblem) {
return true;
}
}
"@
[System.Net.ServicePointManager]::CertificatePolicy = New-Object -TypeName TrustAllCertsPolicy
#set the correct TLS Type
[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12
}
#build JSON headers
$headers = New-Object "System.Collections.Generic.Dictionary[[String],[String]]"
$headers.Add("Accept", 'application/json')
$headers.Add("Content-Type", 'application/json')
#construct Uri
$url="https://"+$hostname+"/api/v1.1/auth/login/"
#Use credentials to request and store a session token from NMC for later use
$result = Invoke-RestMethod -Uri $url -Method Post -Headers $headers -Body $credentials
$token = $result.token
$headers.Add("Authorization","Token " + $token)
#Set the URL for the folder update NMC API endpoint
$CacheUrl="https://"+$hostname+"/api/v1.1/volumes/" + $volume_guid + "/filers/" + $filer_serial + "/cache-path" + $FolderPath
#build the body for the folder update
$body = @{
metadata_only = $MetadataOnly
force = $force
}
#set folder properties
$response=Invoke-RestMethod -Uri $CacheUrl -Method Post -Headers $headers -Body (ConvertTo-Json -InputObject $body)
write-output $response | ConvertTo-Json
Subsequent External Auditing Solution Scans
After the initial scan of the Nasuni volume, it is recommended to configure the external auditing solution to perform incremental scans, rather than use, for example, a 24-hour full-scan frequency. The incremental scans rely on the stream of audit events from the Nasuni Edge Appliances to identify new data to scan.
While the bulk of the metadata for the volume is already resident in the cache due to the initial “Bring into Cache” procedure, any new metadata must be downloaded to the cache by the Nasuni Edge Appliance as the external auditing solution requires it. The amount of new metadata to download to the cache depends on how much data changes and how frequently the data changes in the volume being scanned.
External Auditing Solution File Content Scans
Unlike external auditing solution processes that only scan metadata, other external auditing solution processes can scan the contents of files in order to identify sensitive information. They might copy the relevant files from the Nasuni Edge Appliance to the external auditing solution, and then analyze them, based on a set of discovery rules configured by the administrator of the external auditing solution. If a change is made to the rules that define the sensitive information, the external auditing solution might rescan the data. This can happen due to an update to the ruleset provided by the external auditing solution, or due to a configuration change made to the ruleset by the administrator of the external auditing solution.
To accommodate the scanning of data, careful consideration must be given to the amount of data to scan. Ideally, the cache of the Nasuni Edge Appliance should be large enough to contain the dataset that the external auditing solution is scanning, in addition to any space necessary for the cache to perform its other tasks. As an example, a 40 TB volume might only have 2 TB of data to scan. In this case, the cache of the Nasuni Edge Appliance should include 3 TB for this scanning task, in addition to any space necessary for the cache to perform its other tasks. This would allow the external auditing solution dataset to remain in the cache, with some allowance for future growth.
Increasing the default number of threads used by the external auditing solution to perform scans improves the speed of the scans. Contact the Support for the external auditing solution for assistance with increasing the thread count for the external auditing solution.
Regular scans performed by the external auditing solution help ensure that the data remains resident in the cache, so it is not necessary to pin specific data to the cache.
Enable “Auto Cache” for the volume, in order to proactively load as much new data created by other Nasuni Edge Appliances as possible.
It might not be possible or practical to specify a cache large enough to contain the entire dataset to be scanned. For example, policy requirements might specify that the entire volume must be scanned by the external auditing solution. In such cases, it is critical that the Nasuni Edge Appliance be located as close as possible to the cloud storage provider. This helps to ensure that adequate bandwidth is available for downloading large amounts of data from the object store into the Nasuni Edge Appliance’s cache, as needed. In this scenario, a scan takes additional time to complete, because the Nasuni Edge Appliance must bring required data into the cache, and also evict already-scanned data from the cache to make room for more data, before the external auditing solution can perform the specified scans. This frequent rolling-over of the contents of the Nasuni Edge Appliance’s cache would have a negative impact on the end-user experience, further emphasizing the need for a dedicated Nasuni Edge Appliance.
Reserved Instances
Public cloud providers might offer special pricing for reserved instances of virtual machines. This special pricing can provide considerable cost savings over the life of a virtual machine. Consult your cloud provider’s product offering for information about purchasing reserved instances for the dedicated Nasuni Edge Appliance and external auditing solution.
Connecting a Volume to a New Edge Appliance
After initially configuring the external auditing solution to monitor your volumes, if you connect a monitored volume to a new Edge Appliance, you must configure the new Edge Appliance to send audit events to the external auditing solution. For assistance with updating the configuration, contact Support for the external auditing solution.
Nasuni Virtual Resource Recommendations
The below specifications are recommendations for optimizing the performance of the virtual Nasuni Edge Appliance for use by an external auditing solution. Customers may choose to start with much lower specifications, and only increase the resources if they wish to decrease the scanning times.
Nasuni Virtual Filer
When scanning only metadata:
8 vCPUs
32 GiB Memory
1 TiB Cache (SSD) providing at least 5000 IOPs
256 GiB COW (SSD)
When scanning full file content:
16 vCPUs
64 GiB Memory
1 TiB Cache (SSD) providing at least 5000 IOPs
256 GiB COW (SSD)
External Auditing Solution
Refer to the configuration documentation of the external auditing solution for sizing guidance.
Technical Support
Online self-help resources and Technical Support are available at www.nasuni.com/support.