Troubleshooting
The purpose of this section is to present information that can help, should problems occur.
A. Gathering information
The Nasuni Edge Appliance provides many sources of information that are useful for diagnosing and solving problems.
Pop-up notifications are active messages that appear, based on the state of your system
Pop-up notifications include messages about recommended hardware resources and number of notifications. You can check any or all messages on the Notifications page.
The Notifications page lets you view and acknowledge Nasuni Edge Appliance messages
Notifications are searchable and sortable. Informational messages describe system actions or states that do not require action or attention. Warnings describe unusual events that do not require action. Errors describe unusual or incorrect situations that the user should notice and act to resolve. Alerts describe unusual or incorrect situations that the user must act to resolve.
On Nasuni Edge Appliance: Click Notifications, then click All Messages.
On NMC: Click Notifications, then enter the name of the Nasuni Edge Appliance in the Filter text box and click Apply Filter.
If the user has set up email, alerts generate emails to the user. See “For each Nasuni Edge Appliance, configure Email settings” on page 32.
Information on the health of Edge Appliances is available on the NMC Filers page and on the NMC Filer Details pages. Information available includes the following:
CPU: “Unhealthy”, if the CPU utilization is high. Otherwise, “Healthy”.
Directory Services: “Unhealthy”, if this Edge Appliance has failed connecting to Active Directory several times in a row. Otherwise, “Healthy”.
Disk (for Nasuni Edge Appliance hardware appliances only): “Unhealthy”, if any of the following conditions is true for this Edge Appliance:
Filesystem: “Unhealthy”, if any of the following conditions is true for this Edge Appliance:
The local filesystem (
/, /var, /var/nasuni, or/bootmount points) for the Nasuni OS is getting full.Otherwise, “Healthy”.
Memory: “Unhealthy”, if any of the following conditions is true for this Edge Appliance:
NFS: “Unhealthy”, if one or more NFS exports might not be available for this Edge Appliance. Otherwise, “Healthy”.
Network: “Unhealthy”, if any of the following conditions is true for this Edge Appliance:
This Edge Appliance is having trouble connecting to cloud object storage.
This Edge Appliance is having connectivity problems reaching the Nasuni Operations Center (NOC).
This Edge Appliance is unable to connect to the Global File Lock servers.
This Edge Appliance is unable to connect to the NMC message queuing service. NMC management of this Edge Appliance might be impacted.
Otherwise, “Healthy”.
SMB: “Unhealthy”, if one or more SMB (CIFS) shares might not be available for this Edge Appliance. Otherwise, “Healthy”.
Services: “Unhealthy”, if any of the following conditions is true for this Edge Appliance:
The UniFS filesystem process is not available.
One or more Nasuni services are not responding. Otherwise, “Healthy”.
Tip: If any Health condition is displayed as “Unhealthy”, you can view detailed information and any recommendations by hovering over the “Unhealthy” indicator. Alternatively, clicking “View Recommendations” opens the Health Monitor Current Status dialog box, which displays detailed information and any recommendations.
.png)
Figure 6-1: Health Monit or Current Status dialog box.
Information is available on Network status, CIFS share status, NFS export status, FTP/SFTP directory status, platform, subscription status, and software update status
On Nasuni Edge Appliance: Click Status, then click Subscription Status, Updates, Network Status, Platform Status, CIFS Status, NFS Status, or FTP Directory Status to view information.
On NMC: Click Volumes, then click Exports, FTP Directories, or Shares to view comparable information. Click Filers, then click Network, Platform, or Software Updates to view comparable information. To view subscription status, click Account Status.
File System Auditing information available
You can configure extensive file system auditing and logging of operations for volumes.
Tip: Auditing volume events such as Create, Delete, Rename, and Security can aid in recovering from ransomware attacks.
SNMP information available
Extensive metrics are available through SNMP monitoring, including inbound and outbound network traffic, volume size, time of last snapshot, local cache space used and free, amount of unprotected data, cache hit/miss rate, CPU percent utilization and load averages, memory utilization and swap utilization, disk performance information, number of connected CIFS clients, number of merge conflicts, snapshot success count, and times for snapshots. Both 32- bit and 64-bit SNMP network counters are supported.
Nasuni provides two ways to configure SNMP monitoring:
You can enable SNMP traps, which send information to destinations that you provide.
You can use apps that can pull SNMP information, using the definitions in the NASUNI- FILER-MIB.
You can configure either or both.
As the SNMP agent, Nasuni receives requests on UDP port 161 from the third-party SNMP manager that is used for system monitoring. Nasuni sends agent responses back to the source port on the third-party SNMP manager. The third-party SNMP manager receives notifications (including Traps and InformRequests) on SNMP destination port 162. You cannot change port 161 or port 162.
You can also monitor hardware conditions using iDRAC. See iDRAC Configuration.
Information on volumes, including remote access, snapshots, data in cache, and synchronizations, is available.
On Nasuni Edge Appliance: Click Volumes, then select the specific volume
On NMC: Click Volumes, then click Remote Access, Snapshot Schedule, or Sync Schedule to view comparable information.
Information on files and folders, including whether they are present or pinned in the cache and whether they are protected in the cloud, is available
On Nasuni Edge Appliance: Click File Browser, then select the specific file or folder
On NMC: Click Volumes, then click File Browser, then navigate to, or search for, the data you want.
Caution: The maximum length of a file name is 255 bytes.
In addition, the length of a path, including the file name, must be less than 4,000 bytes.
Since the UTF-8 representation of characters from some character sets can occupy several bytes, the maximum number of characters that a file path or a file name might contain can vary.
If a particular client has other limits, the smaller of the two limits applies.
B. Performance issues
Performance issues are related to many conditions.
Observe how often performance issues occur
The frequency — daily, hourly, weekly — and specific times of performance issues — when users begin work, when backups begin — might suggest causes.
Are the performance issues reproducible at will?
If specific activity, such as user access, causes performance issues, that suggests possible causes.
Observe any symptoms associated with performance issues
Specific symptoms — slow application performance, delays opening files — might suggest possible causes.
Do applications access the Nasuni Edge Appliance directly?
Because applications can access large amounts of data rapidly and frequently, application access can be more resource-intensive than user access. In particular, applications that poll the Nasuni Edge Appliance continually for metadata, such as document management systems, can cause large amounts of cloud access.
Do you access the Nasuni Edge Appliance remotely?
VPNs can slow down or block access to the Nasuni Edge Appliance.
Monitor actual resource utilization
If using Nasuni Edge Appliance hardware appliance, check status of hard drives
Verify supported virtual platforms, including the following:
Verify thin or thick provisioning (virtual machine).
Tip: Nasuni recommends using Thick provisioning for all virtual disks for performance reasons. On the VMware virtual platform, Nasuni recommends Thick Provision Eager Zeroed. Thin provisioning is not supported, because acceptable performance is unlikely.
Warning: Do not attempt to restore from a virtual machine snapshot or backup.
Verify minimum memory: 16 GiB (16 GiB recommended, more for heavy loads) (virtual machine)
Verify minimum number of CPU processors: 8 or more recommended for optimal performance (virtual machine)
Verify that server is running properly (virtual machine).
With a virtual machine, are other virtual machines running and competing for resources? If other virtual machines are contending for resources, this can affect performance.
Verify the type of data store.
Different data stores — local, various RAID types —offer different performance.
Verify write-back cache enabled.
A write-back cache can improve performance. A write-back cache is a feature of some RAID controllers.
Does the performance issue occur during Nasuni snapshots?
There might be a system load issue related to processing Nasuni snapshots. You can see if a Nasuni snapshot is in progress by examining the Snapshot Status on the Home page.
Does the performance issue occur while data is being obtained from cloud object storage (faulting)?
If data must be obtained from cloud object storage, there might be an issue processing access to cloud object storage.
On Nasuni Edge Appliance: Click File Browser, then navigate to a specific file or folder and examine its Cache Resident status. To bring a specific file or folder into the cache, click Bring into Cache.
On NMC: Click Volumes, then click File Browser, then navigate to, or search for, the data whose Cache Resident status you want to examine. To bring a specific file or folder into the cache, click Bring into Cache.
Caution: The maximum length of a file name is 255 bytes.
In addition, the length of a path, including the file name, must be less than 4,000 bytes.
Since the UTF-8 representation of characters from some character sets can occupy several bytes, the maximum number of characters that a file path or a file name might contain can vary.
If a particular client has other limits, the smaller of the two limits applies.
If the outbound Quality of Service is large or unlimited, and the inbound Quality of Service is small, the limited inbound bandwidth for return packets (such as acknowledgments) might affect the outbound bandwidth.
If the inbound Quality of Service is too low, and data must be obtained from cloud object storage, data access might be affected.
Tip: Set the outbound Quality of Service to the highest value possible. This helps snapshots complete rapidly.
On Nasuni Edge Appliance: Click Configuration, then select Quality of Service.
On NMC: Click Filers, then click Quality of Service. Select Nasuni Edge Appliances, then click Edit Filers.
When and how frequently does syncing occur for a volume?
Syncing merges data from other Nasuni Edge Appliances to this volume. While this helps to ensure that everyone at all locations is using the most current data, syncs increase the processing load.
On Nasuni Edge Appliance: Click Volumes, select the volume, then examine the Sync Schedule.
Is Auto Cache enabled for a folder?
If Auto Cache is enabled for a folder, there might be an issue processing syncs.
On Nasuni Edge Appliance: Click Volumes, select the volume, then examine the Auto Caching Folders status.
On NMC: Click Volumes, then click Auto Cached Folders.
Is file system auditing enabled?
File system auditing has an impact on performance. Ensure that settings are at the minimum necessary level.
For NFS exports, check NFS mount options Confirm that NFS is using recommended settings.
Although rsize=16384 and wsize=16384 are recommended, tune rsize and wsize for your environment.
Is Previous Versions enabled for a CIFS share?
When Previous Versions is enabled on a share, certain operations on that share can take longer than expected. Specifically, these operations include file upload to web-based applications, such as Salesforce, Microsoft Office 365, or Microsoft mail. Processing for these operations includes the following features:
When the Previous Versions dialog is selected, it enumerates snapshots.
If the required metadata is not present in the cache, it must be brought in from cloud object storage. This can make simple file access take longer than expected. Microsoft is currently investigating the behavior of their Previous Versions dialog.
To mitigate such performance issues, current best practice is to use separate shares for operations involving previous versions. This can be configured by following these steps:
On the Nasuni Edge Appliance or the NMC, disable Previous Versions for the original share.
Create a second share identical to the first (perhaps named “<ShareName>_Restore”).
If a user must use Windows Previous Versions, instruct them to use the second share.
Are users searching snapshots?
Searching snapshots requires accessing large amounts of data from cloud object storage.
Is the Antivirus Service enabled for volumes?
The Antivirus Service adds a small processing load.
On NMC: Click Volumes, then click Antivirus Service. For the selected volumes, click Edit Volumes.
Is snapshot retention not equal to “all snapshots” for a volume?
If snapshot retention is set to “a set number of snapshots” or “snapshots within a given time”, then processing is necessary to remove unwanted snapshots.
On Nasuni Edge Appliance: Click Volumes, select the volume, then examine the Snapshot Retention status.
On NMC: Click Volumes, then click Snapshot Retention. For the selected volumes, click Edit Volumes.
Is snapshot directory access enabled?
Snapshot directory access can add a significant load to the Nasuni Edge Appliance.
On Nasuni Edge Appliance: Click Volumes, select the volume, then examine the Snapshot Directory Access status.
On NMC: Click Volumes, then click Snapshot Access. For the selected volumes, click Edit Volumes.
Tip: If both the SMB (CIFS) protocol and the NFS protocol are enabled on a volume, then the .snapshot directory is not available.
Is a volume deletion in progress?
When you delete a volume, the data in that volume must be processed and removed. For large volumes, this can take time.
Is Case-Sensitive Paths enabled for a CIFS share?
Using case-sensitive paths improves performance. However, some links and applications might not work if Case-Sensitive Paths is enabled for a share.
Important: Clients such as Windows can sometimes give inconsistent results when dealing with the case sensitivity of file names.
On Nasuni Edge Appliance: Click Status, then select CIFS Status. Select the share, click Show Advanced Options, then examine Case-Sensitive Paths status.
On NMC: Click Volumes, then click Shares. For the selected share, click Edit.
Examine Quality of Service rules
The larger the outbound Quality of Service, the faster incoming data is protected in cloud object storage and the more cache space is made available for data. This is especially important if snapshots are taking too long.
If the outbound Quality of Service is large or unlimited, and the inbound Quality of Service is small, the limited inbound bandwidth for return packets (such as acknowledgements) might affect the outbound bandwidth.
If the inbound Quality of Service is too low, and data must be obtained from cloud object storage, data access might be affected.
Tip: Set the outbound Quality of Service to the highest value possible. This helps snapshots complete rapidly.
On Nasuni Edge Appliance: Click Configuration, then select Quality of Service.
On NMC: Click Filers, then click Quality of Service. Select Nasuni Edge Appliances, then click Edit Filers.
Is Remote Access enabled for this volume? Enabling Remote Access adds a processing load.
Tip: Perform any necessary data ingestions to the volume before enabling Remote Access.
Otherwise, data ingestion processing can impact the synchronization of remote volumes. If Remote Access is already enabled, disable Remote Access before performing data ingestion.
On Nasuni Edge Appliance: Click Volumes, select the volume, then examine the Remote Access status.
On NMC: Click Volumes, then click Remote Access. For the selected volumes, click Edit Volumes.
Snapshots and syncs impact other processing
When snapshots and syncs run, they add a processing load.
On Nasuni Edge Appliance: Click Volumes, select the volume, then examine the Snapshot and Sync statuses.
On NMC: Click Volumes, then click Snapshot Schedule or Sync Schedule.
If you use offline files, the synchronization that is necessary when connected adds a processing load.
If you use roaming profiles, bringing the user profile files to the user machine adds a processing load.
On Macs, different file views have different performance characteristics
C. Issues sending data to Nasuni Edge Appliance
Data not arriving fast enough at Nasuni Edge Appliance
Ensure that the inbound Quality of Service (QoS) is fast enough.
D. Nasuni Edge Appliance connectivity issues
Important: Edge Appliances and the NMC must be configured with operational DNS servers and a time server (internal or external) within your environment.
If you move the Nasuni Edge Appliance, verify network and Internet connectivity
Verify that server is running properly
To reconfigure the network if the Nasuni Edge Appliance user interface is not available, you can access network settings using the service menu on the console
E. File access issues
Verify that Active Directory or LDAP Directory Services users and groups are set up correctly to permit all necessary users to access.
Verify that server is running properly
On Windows clients, files appear locked even when no one is working on them
Web Access not working
Tip: “Mobile Access” must be enabled in the customer license before Web Access can be used with a Nasuni Edge Appliance.
Web Access does not work with Internet Explorer Internet Explorer does not currently support this feature
Shares are hidden
Cannot delete directories
If "Snapshot Directory Access" is enabled on a volume and “Enable Snapshot Directories” is enabled on a CIFS share of that volume, then directories in that CIFS share on that volume cannot be deleted.
F. Multiple domain Active Directory issues
You can join multiple Active Directory domains using General Settings on the Configuration
menu. If issues arise, contact Nasuni Technical Support for assistance.
Important: To connect an Edge Appliance to a shared volume owned by another Edge Appliance, the following must be true:
The Edge Appliance must join the same domain as the owning Edge Appliance.
The domain configuration for the Edge Appliance must match the domain configuration for the owning Edge Appliance.
Important: If joining an Active Directory domain, members of the Active Directory "Protected Users" security group cannot be used to join the domain. This is due to the login restrictions for members of that security group. Nasuni recommends using a Domain Admin account that is not a part of the “Protected Users” group to join Active Directory.
G. User interface issues
If something ever goes wrong with SSL certificates and you are unable to access the Nasuni Edge Appliance user interface, use the service menu console to reset the certificate to the default self-signed certificate
H. Miscellaneous issues
Active Directory connection message in the Nasuni Edge Appliance user interface
On Nasuni Edge Appliance: Click Configuration, then select General Settings. Click Update Domains or refresh the page. This forces the test to run again, and the warning often disappears.
“Snapshot ran out of internal space” error
The snapshot (copy-on-write or COW) disk is used during the snapshot process. If any writes to the Nasuni Edge Appliance occur during a snapshot, the previous data from the cache disk is copied to the COW disk, and the new data is written to the cache disk. Hence, the term “copy- on-write”. This allows new writes to take place at any time, even during the snapshot process.
If the copy-on-write (COW) disk is too small, then a snapshot might fail. You can increase the size of the COW disk. See the Cache Configuration Guide for details.
Important: The size of the COW disk should be at least 1/4 of the size of the cache disk. The COW disk need not be larger than 250 GiB, except in special situations where ingestion might exceed the size of the cache disk, such as during initial ingestion or when ingesting extremely large files. In such cases, you can increase the size of the COW disk. For details, see Resizing the Cache and Snapshot Disks.