The following section describes the Grafana dashboards included with File IQ.
Note: File IQ dashboards use aggregated data of your event activity that the FS Event Aggregator compiles.
Edge Appliance DR and Volume Name Reuse
If an Edge Appliance (NEA) is disaster recovered (DR’ed), it is assigned a new unique identifier. As a result, it appears as a separate appliance in the Grafana dashboards. To help differentiate between the original and recovered instances, each appliance is labeled with the last few characters of its unique identifier, as illustrated in the following screenshot:
.png)
You can choose to include either one or both of the appliances in your dashboards, depending on your monitoring or analysis needs.
Similarly, if you delete a volume and later create a new one with the same name, the new volume appears as a separate instance in the Grafana dashboards. To distinguish between the original and the new volume, each is labeled with the last few characters of its unique identifier, as shown in the following screenshot:
.png)
You can also choose to include either one or both of the volumes in your dashboards, depending on your monitoring or analysis needs.
Directory Summary
Use this dashboard to identify:
The most active directories.
The volume or Edge Appliance with the most active directories.
What actions are happening in your directories.
This dashboard has a 2-hour data latency and a default “Last 3 hours” time range.
The Directory Summary dashboard includes the following filters:
Volumes - Filter results by volume.
Edge Appliances - Filter results by Edge Appliance.
Identity Providers - Filter results by identity provider.
Groups - Filter results by user group.
Note: These filters can be set to “All” or any number of volumes as required.
Panels
This dashboard contains three panels that change based on the selected filters.
The Directory Summary includes the following panels:
Total Directories – This panel displays the total number of directories in which user activity has generated events in the selected Edge Appliance.
Top 10 Directories (By Activity) – This panel displays the top 10 directories with the highest activity. The legend on the right shows the directory name and the number of associated events.
10 Most Active Directories – This panel displays the same top 10 directories as the previous pie chart, followed by a breakdown of those activities in the four CRUD columns. Clicking the hyperlink on the volume name sets the volume's Edge Appliance to the selected volume.
Group Drilldown
This dashboard is a drilled-down view of the Group Summary dashboard events. It provides a detailed view of an individual user's event activity based on the selected filters.
Use this dashboard to identify:
The most active groups and users.
A breakdown of their actions.
Which volumes, Edge Appliances, or directories certain groups or users use.
What file types are most used by groups and users.
This dashboard has a 2-hour data latency and a default “Last 3 hours” time range.
The filters are as follows:
Volumes - Filter the results by volume.
Edge Appliances - Filter the results by Edge Appliance.
Identity Providers - Filter the results by the identity provider.
Groups - Filter the results by user group.
Users - Filter the results by user
Note: Filters can be set to “All” or any number of user groups.
Panels
The Group Drilldown dashboard includes the following panels:
Active Users – This panel displays the number of active users that have generated events on any monitored Edge Appliance.
Active Groups – This panel displays the number of active groups that have generated events on any monitored Edge Appliance.
User Activity – This panel displays the activity count per user that has generated events on any monitored Edge Appliance.
The table contains the following columns:
Identity Provider – The user’s identity provider.
Group Name – The user’s group name.
User Name – The name of the user.
Activity – Total activity for the user.
Top 10 Users – This panel displays the activity of the top ten users that have generated events on any monitored Edge Appliance.
Currently Selected Filters – This panel displays the selected user’s information in the format: Identity Provider | Group Name | User Name.
Volumes – This panel displays the total number of monitored volumes on which a user has generated activity events.
Edge Appliances – This panel displays the total number of monitored Edge Appliances.
Directories – This panel displays the total number of directories on monitored volumes on which a user has generated activity events.
User Activity – This panel displays a user's activity across all monitored Edge Appliances.
Category – This panel displays the selected user’s activity events, by category, generated on any Edge Appliance.
Event Type – This panel displays the selected user’s activity events, by type, generated on any Edge Appliance.
Extension – This panel displays a user’s activity events per extension generated on any Edge Appliance.
There are 500 supported extensions in this panel; any extensions besides these are grouped as [Other]. The supported extension list can be found in the appendix.
Group Summary
This dashboard displays a high-level overview of event activity by users and groups.
Use this dashboard to identify:
The most active groups and users.
Which volumes, Edge Appliances, or directories certain groups or users use.
This dashboard has a 2-hour data latency and a default “Last 3 hours” time range.
The filters are as follows:
Volumes - Filter the results by volume.
Edge Appliances - Filter the results by Edge Appliance.
Identity Providers - Filter the results by the identity provider.
Groups - Filter the results by user group.
Panels
The Group Summary includes the following panels:
Active Users – This panel displays the number of active users that have generated events on any monitored Edge Appliance.
Active Groups – This panel displays the number of active groups that have generated events on any monitored Edge Appliance.
User Activity – This panel displays the activity count per user that has generated events on any monitored Edge Appliance.
The table contains the following columns:
Identity Provider – The user’s identity provider.
Group Name – The user’s group name.
User Name – Name of the user.
Activity – Total activity for the user.
Top 10 Users – This panel displays the activity of the top ten users that have generated events on any monitored Edge Appliance.
Summary Dashboard
The Summary dashboard displays an overview of event activity across all Edge Appliances, including volumes, active users, and active groups.
Use this dashboard to identify:
A summary of the total activity across your monitored volumes.
The top 10 active groups and users.
The top 10 most active volume or Edge Appliance.
The top 10 most active IPs.
The top 10 most active directories.
This dashboard has a 2-hour data latency and a default “Last 12 hours” time range.
The filters are as follows:
Volumes - Filter the results by volume.
Edge Appliances - Filter the results by Edge Appliance.
Note: Filters can be set to “All” or any number of volumes.
Panels
This dashboard contains eleven panels that display the following information:
Edge Appliances –This panel displays the total number of monitored Edge Appliances with event activity during the selected timeframe.
Volumes – This panel displays the total number of monitored volumes with event activity during the selected time frame.
Active Users – This panel displays the total number of users with event activity during the selected timeframe.
Active Groups – This panel displays the total number of user groups with event activity during the selected timeframe.
Activity – This panel displays a graph showing the total event activity over the selected timeframe.
Top 10 Volumes Activity – This panel displays event activity on the top 10 volumes during the selected timeframe.
Top 10 Edge Appliances Activity – This panel displays event activity on the top 10 Edge Appliances over the selected timeframe.
Top 10 Users Activity – This panel displays the event activity of the top 10 Users over the selected timeframe.
Top 10 Groups Activity – This panel displays the event activity of the top 10 Groups over the selected timeframe.
Top 10 Directories Activity – This panel displays event activity on the top 10 Directories over the selected timeframe.
Top 10 IPs Activity – This panel displays the event activity of the top 10 IPs over the selected timeframe.
User Directory Access
The User Directory Access dashboard displays the directories with the most event activity.
Use this dashboard to identify:
The directory with the most activity on a volume or Edge Appliance.
How many users are active on the most active directories.
This dashboard has a 2-hour data latency and a default “Last 7 days” time range.
The filters are as follows:
Volumes - Filter the results by volume.
Edge Appliances - Filter the results by Edge Appliance.
Results Limit - Limit the number of directories displayed within the table.
Panels
This dashboard contains one panel. It displays the following information in a table format:
Edge Appliance – Name of the Edge Appliance.
Volume – Name of the volume.
Directory – Path to the directory.
Users – Number of users that have generated events within the directory.
Access Events – Number of events that have originated from the directory.
The information displayed is sorted in descending order by access events.
Events Dashboard
The Events Dashboard displays an overview of event activity across all monitored Edge Appliances. This includes top folders, activity per Edge Appliance, volume, individual user, extension, category, and event type, all based on the selected filters.
Note: File IQ uses 7 days, 21 days, and 90 days of data retention for its Audit Events and Activity dashboards. For more information, see the Activity and Audit Event Dashboards Data Retention appendix.
Use this dashboard to identify:
The breakdown of event categories and types.
The percentage usage per volume or Edge Appliance.
The most common file types.
This dashboard has a 2-hour data latency and a default “Last 12 hours” time range.
The filters are as follows:
Volumes - Filter the results by volume.
Edge Appliances - Filter the results by Edge Appliance.
Note: These filters can be set to “All” or any number of volumes as required.
Panels
This dashboard contains eight panels that change based on the selected filters.
The Events Dashboard includes the following panels:
All Events – This panel shows the total number of events.
Top Folders – This panel displays the ten folders with the highest number of events by percentage.
Edge Appliance Activity – This panel displays the activity percentage of each Edge Appliance monitored.
Volume Activity – This panel displays the activity percentage of each volume monitored.
Users – This panel displays the activity percentage of each user on the Edge Appliance.
Extensions – This panel displays the activity percentage per extension on the Edge Appliance. There are 500 supported extensions in this panel. Any extensions besides these 500 are grouped as [Other]. The supported extensions list can be found in the appendix.
Categories – This panel displays the activity percentage per event category on the Edge Appliance.
Event Types – This panel displays the activity percentage per event type on the Edge Appliance.
Inspector Dashboard
Use the Inspector Dashboard to view the individual events being collected by FILE IQ.
Note: This dashboard can be initially slow in loading because it uses raw non-aggregated audited events. Because of this, looking over longer periods can also be slow. To reduce latency, choose shorter timeframes and more filters.
Note: File IQ uses 7 days, 21 days, and 90 days of data retention for its Audit Events and Activity dashboards. For more information, see the Activity and Audit Event Dashboards Data Retention appendix.
The data in this dashboard is near real-time and a default “Last 3 hours” timeframe.
Use this dashboard to identify:
Events over a specified time.
Locate specific events by user.
The events related to a folder or file.
The events originating from a specific IP.
The filters are as follows:
Volumes - Filter the results by volume.
Edge Appliances - Filter the results by Edge Appliance.
Event Types - Filter the results by event type.
Path From - Filter the results by path.
User Name - Filter the results by searching by user name.
Client IP - Filter the results by searching by client IP address.
Events Limit - Limit the number of events that are displayed.
Note: There is no client IP if the volume on which the events are generated is NFS.
Panel
This dashboard contains one panel that displays the following information:
Time – The UTC timestamp of the audit event.
Edge Appliances – Name of the Edge Appliance on which the event was generated.
Volume Name – Name of the volume on which the event was generated.
Volume Type – Type of volume mount on which the event was generated (NFS or SMB).
Category – Category of the generated event.
Event Type – Event type of the generated event.
Path From – Path of the file from the generated event.
New Path To – Possible new file path from the generated event. This field would be populated in cases like renaming or moving a file or directory.
User Name – Username associated with the generated event.
Group Name – Group name associated with the generated event.
Client IP – IP associated with the generated event.
File Extension – Extension type of the file from the generated event.
Share Name – Name of the share from which the generated event originated.
Note: When Administrative Users use SMB to access the NEA, their actions are associated with the Linux root account rather than the user's Active Directory account. The access shows as "root" because Samba maps the user to root, giving them full access to the file system.
Volume Usage
The Volume Usage dashboard displays detailed volume usage information and allows users to drill down to three directory levels deep in a monitored volume.
The information displayed on this page is relevant until the time of the last FS Metadata Service scan run, so the date and time pickers are absent.
Note: The Volume Usage dashboard is available in the 9.15 and later releases.
The following filters are displayed on the dashboard:
Volumes - Filter the results one volume at a time.
Number of Levels - Depth of directory level to observe. You can select up to 3 levels deep. You can only select one level at a time.
.png)
Panels
This dashboard contains seven panels that change based on the selected filters.
The panels are as follows:
Total Volume Size – This panel displays the total volume size usage on the selected volume.
Total Volume Files – This panel displays the total volume file count on the selected volume.
Total Volume Directories – This panel displays the total volume directory count on the selected volume.
Directories at Level [n] – This panel displays the following information in table format for the specified volume at the specified directory level:
Directory Name – Name of directory.
Size – Size of the directory.
% of Total Volume Size – Percentage of directory size over total volume size.
# of Files – Number of files that are children to this directory.
% of Total Volume Files – Percentage of child file count over total volume file count.
# of Directories – Number of directories that are children to this directory.
% of Total Volume Directories – Percentage of child directory count over total volume directory count.
The information displayed is sorted in descending order by directory size.
Top 20 – Directory Size – Level [n] – This panel displays the top twenty directories by size in a pie chart for the specified volume at the specified directory level.
.png)
Top 20 – File Count – Level [n] – This panel displays the top twenty directories by child file count in a pie chart for the specified volume at the specified directory level.
.png)
Top 20 – Directory Count – Level [n] – This panel displays the top twenty directories by child directory count in a pie chart for the specified volume at the specified directory level.
.png)
Global File Lock (GFL) Directories
This dashboard summarizes which directories across volumes have Global File Lock (GFL) enabled and which mode (Optimized or Advanced). The information on this page is relevant until the time of the last FS Metadata Service scan run, so the date and time pickers are absent.
Use this dashboard to identify:
The volumes that have files or folders with Global File Lock enabled.
The files that are Global File Lock enabled.
The type of Global File Lock enabled.
The filters are as follows:
Volumes - Filter the results by individual volume.
Panels
This dashboard contains four panels that change based on the selected filters.
The Global File Locks Directories includes the following panels:
Total Number of GFL Directories – This panel shows the total number of directories that have GFL enabled within the selected volume.
Total Number of GFL Files – This panel shows the total number of files that have GFL enabled within the selected volume.
Total Size of GFL Files – This panel shows the total size of all files that have GFL enabled within the selected volume.
GFL Table – This panel displays a table detailing all the directories that have GFL enabled.
The table contains the following columns:
Directory Path – Path of directory.
File Count – Child file count within the directory.
Directory Count – Child directory count within the directory.
Directory Size – Total size of the directory.
GFL Type – Mode of GFL that is enabled for the directory.
Create Time – Creation time of the directory.
Modified Time – Last modification time of the directory.
Last Access Time – Last accessed time of the directory or the subdirectories.
Volume Stats
The Volume Stats dashboard displays an overview of your volumes, showing the size, file count, and directory count of your volumes over time.
Use this dashboard to identify:
The usage summary of a volume over the last 7 days.
The largest volume.
The filters are as follows:
Volumes - Filter the results by volume.
.png)
Panels
This dashboard contains three panels. Each point on the following graphs represents an FSMS scan that has been run and completed.
The panels are as follows:
Volume Size – This panel displays the size of the volumes over a selected timeframe.
.png)
File Count – This panel displays the file count of the volumes over a selected timeframe.
.png)
Directory Count – This panel displays the directory count of the volumes over a selected timeframe.
.png)
Volume Utilization
The Volume Utilization dashboard shows a hierarchical display of your volume’s folder structure with a depth of three from the most recent volume scan. Hovering over a block provides additional information on size and creation time.
Use this dashboard to view a volume’s utilization.
The filters are as follows:
Volumes - Filter the results by individual volume.
Panel
This dashboard contains one panel.
Volume Utilization – This panel displays a colored treemap view of the selected volume. The colors correspond to the volume size:
.png)
Volumes Summary
The Volumes Summary dashboard displays an overview of the volumes at the time of the last FS Metadata Service scan.
Use this dashboard to view the current usage of a volume.
The filters are as follows:
Volumes - Filter the results by volume.
.png)
Panels
This dashboard contains the following five panels:
Total Volume Count – This panel displays the total number of volumes in the most recent FSMS scan.
.png)
Total Directories Count – This panel displays the total directories across all volumes from the most recent FSMS scan.
.png)
Total Files Count – This panel displays the total number of files across all volumes in the most recent FSMS scan.
.png)
Total Size – This panel displays the total size of the volumes in the most recent FSMS scan.
.png)
Volumes Summary – This panel displays the following information in a table format:
Volume Name – The name of the volume.
# of Dirs – Number of directories in the volume.
# of Files – Number of files in the volume.
Size – Size of the volume.
Last Scan End – The time the last scan of the volume finished.
Snapshot Time – The time the last snapshot of the volume was taken.
Database Support
The Database Support dashboard offers comprehensive, real-time insights into your File IQ Appliance’s database performance. It provides valuable visibility into key areas such as:
Database activity
Database schema sizes
Detailed statistics for all database queries
Please note that this dashboard presents live data only, and a time picker is not available.
Note: The Database Support dashboard is available in File IQ Appliance version 9.15 and later.
Panels
This File IQ Appliance Database Support dashboard is composed of seven panels. The first four panels display a set of metrics related to Database connections. These panels can be used to check if the maximum number of database connections has been reached on a File IQ Appliance.
Current Database Connections – This panel displays the sum of the current idle or active database activities.
Current Database Connections Breakdown – This panel displays a breakdown of the current database connections based on their state: idle or active connections.
Maximum Database Connections – This panel displays the maximum number of simultaneous database connections supported by the File IQ Appliance.
Note: The maximum number of database connections is currently 100.
Database Connections Usage – This panel displays a breakdown of Database connections based on their usage.
The next two panels check where the database's disk space is used. They provide a percentage of each database schema size in relation to the current database size and a detailed understanding of each table row count. This information can be used for sizing purposes.
Database Schema Size – This panel displays the estimated size of the various File IQ Appliance database schemas and provides the following information for each database schema:
The File IQ database schema name.
The size of the schema.
The size of the schema as a percentage of the current database size.
The panel can be used in conjunction with the File IQ Sizing tool to identify database storage allocation and whether the File IQ database disk needs to be resized.
Note: Figures do not take into account WAL space.
.png)
Note: The information displayed is sorted in descending order based on database percentage size.
Note: The Database Size figures are estimated figures and do not take into account the space allocated to the Write-Ahead Logging feature of the File IQ database.
Database Table Row Count – This panel displays each table's row count in the File IQ Appliance database. It is organized by database schema name and provides the following information for each table:
The File IQ database schema name.
The database table name.
The number of rows for each table.
The information displayed is sorted in descending order of # of Rows.
.png)
Note: Some of the File IQ Database internal tables are not displayed.
The following two panels are grouped in a “Database Activity & Query Performance” section. This section displays the current database activities and statistics for database queries.
Database Activity—This panel displays a detailed list of current database activities. It shows the query state and execution time on the system.
For each database activity, the following is displayed:
Username - The username associated with the activity.
Query ID: The query identifier associated with the activity. This is an internal file IQ identifier that the File IQ support team might use during an escalation process.
PID - The database process identifier associated with the activity.
State - The state of the database activity (active, idle, reserved).
Query Start Time - The start time of the associated activity.
Execution Time - The execution time in milliseconds of the activity.
Backend Type - This field indicates the type of database activity. The “client backend” type is used for File IQ, Appliance processes, and Grafana activities. Database-specific backend types display for internal database activities.
Application Name - This optional field provides additional context for the database activities. It is not used by the File IQ Appliance processes (such as volume scan, event processor, and event aggregator).
Note: The Username, Query ID, PID, and State columns are filterable.
Query Performance - This panel displays cumulative and detailed statistics of the File IQ Appliance database queries.
For each query, the following information is displayed:
Username - The username of the query.
Query ID - This File IQ internal identifier is used to identify each database query. The File IQ support team might use this internal identifier.
# of Calls - The number of times the query has been invoked.
# of Rows - The number of rows returned by the query.
Min Execution Time - The minimum execution time of the query.
Max Execution Time - The maximum execution of the query.
Mean Execution Time - The mean execution time of the query.
Total Execution Time - The total execution time of the query.
Note: The Username and Query ID columns are filterable.
Service Support
The Service Support dashboard displays key metrics for the File IQ Appliance. This dashboard can be used to gain an understanding of:
The list of NEAs that are associated with the File IQ Appliance.
The list of volumes that the File IQ Appliance scans.
The status and performance of the File IQ Appliance audit event aggregation.
The status and performance of the File IQ Appliance volume scan and volume aggregation processes.
The Service Support dashboard provides useful insights for understanding the performance and connectivity status of the File IQ Appliance. The table below outlines how this dashboard can assist with identifying and resolving potential issues.
Issue | Troubleshooting |
The Inspector dashboard is not reporting any events for a volume or NEA(s). | Use the Audit Event section of the Service Support dashboard to validate the connectivity between NEAs, volume, and File IQ Appliance. Check that the NEA is assigned to the File IQ Appliance (refer to the File IQ Install Guide). Validate that the firewall settings are not blocking the communication between your NEA(s) and Event Hub (see Firewall and Port Requirements). |
The event-based File IQ dashboards do not update or report stale data. | The File IQ appliance uses summary database tables (initialized via hourly-based aggregations). Use the Audit Event Aggregation section of the Service Support dashboard to check the aggregation execution times and status. |
The file system metadata-based dashboards do not display any data | Based on the volume size, the initial scan might take several hours, days or weeks (depending on the volume size). Use the Volume Aggregation History section of the Service Support dashboard to check the progress of the volume scan. For long-running volume scans, the dashboard provides incremental scan completion updates. |
The file system metadata-based dashboards do not update or report stale data or display empty data. | The File IQ appliance performs a daily delta scan for the volumes, with additional volume database aggregations. Check the Volume Aggregation History section to ensure the total scan process is executed in less than 24 hours. Total FSMS scan execution time(s) greater than 24 hours display stale data in the File IQ dashboards. You might need to resize the File IQ Appliance. |
Note: The Service Support dashboard is available in the 9.15 and later releases.
Tip: File IQ also provides a Database Support Dashboard that can be used with the Service Support dashboard to view File IQ system activities.
Understanding Volume Scan Progress
FSMS Volume scan progress often appears to jump in the Service Support Dashboard. For instance, in the following example, the volume scan for nfs-events-1-4 jumps from 0% to 78% very quickly.
.png)
The Volume Scan State Logs panel displays a completed percentage of the size of the scanned volume. Completion percentages typically increase smoothly on day zero of a new volume scan, whereas delta scans can surge. These jumps in scan completion occur when the scan encounters and bypasses an unchanged directory and all its children files. Occasionally, jumps occur during an initial scan if a volume has many small files accompanying a large file.
When a Volume scan is processing a large directory, the Scanned Files and Scanned Dirs figures might appear static in the Volume Scan State Log component while the Percent value of the scan increases. Both the Scanned Files and Scanned Dirs values update after File IQ fully processes the directory.
Dashboard Sections
This Service Support dashboard is organized into four different sections:
Audit Events - This section displays the list of NEAs and volumes linked to the File IQ Appliance. It provides information to help diagnose any connectivity issues with NEA(s) or volume(s). The Audit Events section has the following Panels:
NEAs That Report to FILE IQ - Date and time of the first and last events received by an NEA.
Volumes Accessed by NEAs that report to FILE IQ - Date and time of the first and last received events for a volume.
Audit Event Aggregation History - The File IQ Appliance periodically performs several database aggregations on the main File IQ audit event table. This section provides a detailed history of these aggregations including execution times and status for each aggregation via a single panel: Audit Events Aggregation State.
Volumes - This section displays the list of active volumes, the execution time of the volume scans, and a set of metrics on a volume basis. The Volumes section has the following Panels:
Current State of Volume Scan – This panel displays the state of the most recently completed or ongoing volume scan.
Volume Scan Summary – This panel displays a summary of the latest volume scans.
Volume Scan Execution Time – This panel displays the history of the volume scan execution time.
Volume Aggregation History - This section provides a detailed historical representation of the volume aggregations performed by the File IQ appliance. The Volume Aggregation History has the following panels:
Volume Scan State Logs – This panel displays a detailed history of the volume scan states.
Total FSMS Aggregation Execution Time – This panel displays the history of the FSMS Aggregation Scan Execution Time. This detects any total aggregation time taking more than 24 hours.
Volume Cleanup Execution Time – This panel displays the history of the Volume Cleanup Execution Time. Use this to assess the performance of the aggregation.
Directory Aggregation Scan Execution Time – This panel displays the history of the Directory Aggregation Scan Execution Time. This is used to assess the performance of the aggregation.
Chargeback Aggregation Scan Execution Time – This panel displays the history of the Chargeback Aggregation Scan Execution Time, and assesses the performance of the aggregation.
GFL Aggregation Scan Execution Time – This panel displays the history of the GFL Aggregation Scan Execution Time to assess the performance of the aggregation.
Volume Analytics Execution Time - This panel displays the history of the Volume Analytics Scan Execution Time to assess the performance of the aggregation
Panels
NEAs That Report to File IQ: This panel lists the Nasuni Edge Appliances that have reported activity to the File IQ Appliance.
Use this panel to troubleshoot connectivity issues between NEA(s) and a File IQ Appliance. For each Nasuni Edge Appliance, the table displays:
NEA Name - The name of the Nasuni Edge Appliance.
First Received At - The time of the first received event for the NEA.
Last Received At - The time of the last received event for the NEA.
Time Since Last Received - The time since the NEA received the last event.
.png)
Note: The NEA Name column is filterable.
Note: The dashboard time picker does not affect this panel.
Volumes Accessed by NEAs that report to FILE IQ: This panel lists the volumes that have reported the activity to the File IQ Appliance.
Use this panel to troubleshoot the lack of activity on a particular volume or a connectivity issue between a Nasuni Edge Appliance and volume. For each volume, the table displays the following:
Volume Name - The name of the volume.
First Received At - The time of the first received event for that volume.
Last Received At - The time an event was last received for that volume.
Time Since Last Received: The amount of time since the last event was received for that volume.
.png)
Note: The Volume Name column is filterable.
Note: The dashboard time picker does not affect this panel.
Audit Events Aggregation State - Audit event aggregation tasks are run periodically on the File IQ Appliance to initialize a set of summary tables in the File IQ database. The File IQ dashboards use these summary tables to speed up the loading times of the dashboard panels. This panel lists the execution times and states of the File IQ Appliance aggregation tasks based on the audit events.
The table panel displays the following for each audit event aggregation task:
Time - The time the aggregation task was started.
Category - The category of the aggregation task. The following categories include:
overview: This category groups the aggregations used for high-level summary panels in the File IQ dashboards. The aggregations in this category provide an event count summary based on the directory, event category, event types, and file extension dimensions.
group: This category groups all the aggregations related to user group identifiers – The File IQ Appliance performs different user-group-based aggregations, for example:
Aggregation to summarize the number of events per user group and event categories.
Aggregation to summarize the number of events per user group and file extension types.
total_event_count: This category is used for the total event count aggregation, which indicates the number of events on a volume and NEA basis. This aggregation is primarily used in the Summary and Event Dashboards.
Aggregation Name - The name of the aggregation task.
Aggregated Data Start Time - The start date and time for the aggregated data.
Execution Interval - This value represents the time interval for the aggregated data. The end date for the aggregated data is defined by the sum of Aggregated Data Start Date and Execution Interval. Note that a filter is available in this column to easily identify a set of aggregation tasks based on their execution interval value.
Execution Time - This column tracks the execution time of each aggregation task.
State -This column tracks the state of each aggregation task. A filter is available on this table column to help identify any failed aggregation.
Use this filter to validate a File IQ dashboard issue. For instance, a failed aggregation might cause the data to become stale in the File IQ dashboards.
Note: The table panel only displays the last 1000 aggregation tasks.
Note: The dashboard time picker can filter the table entries in this panel.
If stale data is observed in one of the File IQ dashboards, use this panel to check the performance and status of the aggregation(s) associated with the problematic File IQ dashboards or panels. The following table maps the various File IQ event-based aggregations and the combination of dashboard(s) and panel(s) that use each aggregation.
Aggregation | Dashboard/Panel |
aggregate_groups | Internal File IQ Appliance usage |
events_per_category | Event Dashboard/Categories |
events_per_daily_directory_user_access | User Directory Access/User Directory Access |
events_per_directory | Summary/top 10 directories activity |
events_per_directory_user_access | Internal File IQ Appliance usage |
events_per_event_type | Event Dashboard/Event types |
events_per_extension | Event Dashboard/Extensions |
events_per_group | Summary/Active Groups |
events_per_group_user | Group Drilldown/Active users |
events_per_group_user_category | Group Drilldown/Event Categories |
events_per_group_user_directory | Group Drilldown/Directories |
events_per_group_user_event_type | Group Drilldown/Event Types |
events_per_group_user_extension | Group Drilldown/Top 10 Users Activity |
events_per_ip | Summary/top 10 IPs Activity |
events_per_user | Summary/top 10 Users Activity |
total_event_count | Summary/Edge Appliances |
Current State of Volume Scan - This panel indicates the current or most recently executed volume scans. The table component displays the following for each scan State:
State - An indication of the volume scan state. The following table displays the supported values for the State column:
State | Description | Volume Name Displayed |
Job Executor Running | Starting the combined scan and aggregation task for all the volumes. | N/A |
Start Pulling volumes | Start pulling all the volumes. | N/A |
Completed Pulling all volumes | Finished pulling all the volumes. | N/A |
Caching IDP Information | Internal caching process. | Yes |
Start executing job: XX | Start an aggregation task for a volume. (XX is a task identifier). | Yes |
Starting aggregations | Start the aggregations for a volume. | Yes |
Starting volume cleanup | Start the volume cleanup for a volume | Yes |
Completed volume cleanup | Completed the volume cleanup for a volume | Yes |
Starting directory scan aggregator | Start the directory scan aggregation for a volume. | Yes |
Completed directory scan aggregator | Completed the directory scan aggregation for a volume. | Yes |
Starting chargeback scan aggregator | Start the chargeback scan aggregation for a volume. | Yes |
Completed chargeback aggregator | Completed the chargeback scan aggregation for a volume. | Yes |
Starting GFL scan aggregator | Start the GFL scan aggregation for a volume. | Yes |
Completed GFL aggregator | Completed the chargeback scan aggregation for a volume. | Yes |
Aggregations completed | Completed all the aggregation for a volume. | Yes |
Finished executing job: XX | Completed the aggregation task for a volume. | Yes |
Job Executor Completed | Finished the combined scan & aggregation task for all volumes. | N/A |
Volume Name - This information is only provided when the scan's state is specific to a volume. For example, the value is empty when the state of the volume scan is across all volumes.
Note: The volume name is only displayed for volume scan states associated with a single volume.Scanned Files - The number of files that have been scanned
Scanned Dirs - The number of directories that have been scanned
Scanned Size - The total scanned size so far
Percent - The percentage of the size of the volume that has been scanned
.png)
Some of the columns may not show all the time. For example, Scanned Files, Dirs and Size will only be available during the scanning phase, and Percent will only be available during the scanning phase and some of the aggregations.
Volume Scan Summary - This panel summarizes the last successful FILE IQ Appliance scan on a volume basis. The table component displays the following information for each volume:
Volume Name - The name of the volume.
Execution Time - The execution time of the last successful scan for the volume.
Compressed Size - The compressed size of the volume.
Uncompressed Size - The uncompressed size of the volume.
# of Files - The number of files in the volume.
# of Directories - The number of directories in the volume.
.png)
Volume Scan Execution Time - This panel displays a graphical history of the execution time for each volume that reports to the FILE IQ Appliance. A time series is used on a volume basis. The historical data this panel provides can be used to identify performance degradation(s) in terms of execution time for a volume or a set of volumes. The graphical representation also provides a set of statistics on a volume basis:
Total: Total execution time.
Min: Minimum execution time.
Max: Maximum execution time.
Mean: Average execution time.
.png)
Volume Scan State Logs - This panel displays a detailed FILE IQ Appliance scan state history. The supported values in the State column are the exact same as those detailed in the Current State of Volume Scan panel documentation.
Use the detailed history in this panel to troubleshoot any aspect of the File IQ Appliance volume scan process. For instance, a volume scan made by the File IQ Appliance involves a volume pull operation, a file system scan, or the execution of a set of database aggregations. The table component displays the following for each state entry:
Timestamp - The time the state entry was logged.
State - The state name of the state entry log.
Volume Name - The volume name associated with the state entry. This column is empty for all the states that apply to multiple volumes.
Scanned Files - This information is only available for the “Scanning volume for job: XX” State. It provides an incremental progress indication of the volume scan in terms of scanned files. This measures the progress of the ongoing scans and is particularly useful when adding a volume to the File IQ Appliance.
Scanned Dirs -This information is only available for the “Scanning volume for job: XX” State. It provides incremental progress of the volume scan in terms of scanned directories. This measures the progress of the ongoing scans and is particularly useful when adding a volume to the File IQ Appliance.
Scanned Size - This information is only available for the “Scanning volume for job: XX” State. It provides incremental progress of the volume scan in terms of scanned volume size. This measures the progress of the ongoing scans and is particularly useful when adding a volume to the File IQ Appliance.
Percent - This information is only available for the “Scanning volume for job: XX” State. It provides a visual indication of the scan progress as an estimated percentage of completion. This information is particularly useful when adding a volume to the File IQ Appliance.
.png)
Note: When upgrading File IQ from 9.15.X to version 10.x, several columns in this panel might not be visible initially: Scanned Files, Scanned Dirs, Scanned Size, and Percent columns. A successful File IQ volume scan has to be performed after the File IQ Appliance upgrade to version 10.x is completed, with at least one file change in the scanned volume(s) for the new columns to appear in the table panel. Similarly, if no change is performed on the volumes linked to the File IQ Appliance over one full day, only the Timestamp, State, and Volume Name columns are displayed in this table.
Total FSMS Aggregation Execution Time - This panel tracks the history of the total FSMS aggregation times for the selected time interval in the time picker. A time series is used on a volume basis.
Use this panel to identify degradation in the performance of the aggregations performed by the FILE IQ Appliance for a particular volume or set of volumes.
Note: It is normal for the aggregation process's performance to degrade slightly over time as volumes increase in size.
Volume Cleanup Execution Time - This panel tracks the history of the Volume Cleanup Execution Time for the selected time interval in the time picker. A time series is used on a volume basis.
Use this panel to identify significant degradation in the performance of this aggregation.
.png)
Directory Aggregation Scan Execution Time - This panel tracks the history of the Directory Aggregation Scan Execution Time for the selected time interval in the time picker. A time series is used on a volume basis.
Use this panel to identify significant degradation in the performance of this aggregation.
Note: It is normal for the aggregation process's performance to degrade slightly over time as volumes increase in size.
Chargeback Aggregation Scan Execution Time - This panel tracks the history of the Chargeback Aggregation Scan Execution Time for the selected time interval in the time picker. A time series is used on a volume basis.
Use this panel to identify significant degradation in the performance of this aggregation.
Note: It is normal for the aggregation process's performance to degrade slightly over time as volumes increase in size.
GFL Aggregation Scan Execution Time - This panel tracks the history of the GFL Aggregation Scan Execution Time for the selected time interval in the time picker. A time series is used on a volume basis.
Use this panel to identify significant degradation in the performance of this aggregation.
Note: It is normal for the aggregation process's performance to degrade slightly over time as volumes increase in size.
Volume Analytics Execution Time - This panel tracks the history of the Volume Analytics Execution Time for the selected time interval in the time picker. A time series is used on a volume basis.
Use this panel to identify significant degradation in the performance of this aggregation.
.png)
System Status Dashboard
The System Status dashboard provides information on system status and high-level performance metrics for the File IQ Appliance. This dashboard give a view on the operational metrics of the system : for example, the number of files and directories on all monitored volumes, and the rate of events processed per minute.
.png)
Navigation
To access the System Status dashboard, click System Status. Most of the panels on the System Status dashboard are navigational panels. Clicking one of the following panels directs you to a dashboard where you can find more detailed information.
The table below outlines where each panel on the System Status dashboard directs you when clicked.
Panel | Links to | Notes |
Number of Files | Volume Summary | Clicking this panel starts the Volume Summary dashboard. |
Number of Directories | Volume Summary | Clicking this panel starts the Volume Summary dashboard. |
Number of Edge Appliances | Events Dashboard | Clicking this panel starts the Events dashboard |
Number of Volumes | Volume Summary | Clicking this panel starts the Volume Summary dashboard. |
Total Size | Volume Summary | Clicking this panel starts the Volume Summary dashboard. |
Events per Minute | Events Dashboard | Clicking this panel starts the Events dashboard. |
Total Number of Events | Events Dashboard | Clicking this panel starts the Events dashboard. |
Files per Minute | Volume Stats | Clicking this panel starts the Volume Stats dashboard. |
Directories per Minute | Volume Stats | Clicking this panel starts the Volume Stats dashboard. |
Volume Scan Status | Service Support Dashboard | Clicking this panel starts the Service Support dashboard. |
DB Size | Database Support Dashboard | Clicking this panel starts the Database Support dashboard. |
DB Usage % | Database Support Dashboard | Clicking this panel starts the Database Support dashboard. |
Dashboard Panels
The System Status dashboard provides the following metrics and information:
Panel Name | Description | Example |
|---|---|---|
Number of Files | Number of files across all Volumes connected to Nasuni File IQ. The value turns orange if the number exceeds 300,000,000. |
|
Number of Directories | Number of Directories across all Edge Appliances and volumes. The value turns orange if the number exceeds 60,000,000. |
|
Number of Edge Appliances | Number of Edge Appliances connected to the File IQ Appliance |
|
Number of Volumes | Number of Volumes shared with the Nasuni File IQ Appliance. The value turns red when the number of connected volumes exceeds 47. |
|
Total Volume Size | Total volume size for all volumes. |
|
Events per Minute | Average number of events received by the File IQ Appliance across all of the connected Edge Appliances in previous hour. The value turns red when the value exceeds 180,000. |
|
Total Number of Events | The total number of raw events currently stored in the File IQ database. |
|
Files per Minute | Average number of discovered files per minute for the past 24 hours. |
|
Directories per Minute | Average number of discovered directories per minute for the past 24 hours. |
|
Volume Scan Status | The number of volumes managed by the appliance that have scanning enabled. It provides a breakdown of the volume count based on the last known scan status:
|
|
DB Size | The total size of the File IQ database. |
|
DB Usage % | The percentage of disk space used in the File IQ database compared to the total disk size.
|
|
Cache Size | Total disc size of the File IQ cache. |
|
Cache Usage % | The percentage of cache space used in the File IQ compared to the total cache size.
|
|
.png)
.png)
.png)
.png)
.png)
.png)
.png)
.png)
.png)
.png)
.png)
.png)
.png)
.png)
Volume Scan Detailed Status
The Volume Scan Detailed Status panel displays in real-time and provides details of each activity on a volume basis.
.png)
The Volume Scan Detailed Status panel provides the following metrics and information:
Column | Description |
|---|---|
Volume | The volume name with the last four characters of the volume identifier. |
Activity | The activity stage for the volume : PULL_PENDING, PULLING, LONG_PULL, PULL_MONITOR, LONG_PULL_MONITOR, CRAWL_PENDING, CRAWLING, AGGREGATING, and IDLE. |
Last Update | The last time the status of this volume was updated. |
Last Pull | The last pull completion time for the volume. |
Pull Duration | The duration of the last pull for the volume |
Last Scan | The last scan completion time for the volume. |
Scan Duration | The last scan duration for the volume. |
Last Aggregation | The last aggregation completion time for the volume. |
Aggregation Duration | The last aggregation duration. |
Note | Additional context information for the Activity. |
The Activity column displays the activity stage for the volume. These stages include the following:
Activity | Description |
|---|---|
PULL_PENDING | The volume is waiting in the queue to be pulled. |
PULLING | The volume is currently being pulled. The system is syncing the metadata for this volume. |
LONG_PULL | The volume is pulling; however, it is taking more than an hour. Therefore, the system will allow another pull to start if necessary. |
PULL_MONITOR | An existing pull was discovered for this volume, and it is being monitored. |
LONG_PULL_MONITOR | An existing pull was discovered for this volume; however, it is taking more than an hour. Therefore, the system will allow another pull to start if necessary. |
CRAWL_PENDING | The volume is in the queue to be scanned. |
CRAWLING | The volume is currently being scanned. The system is exploring its directory tree metadata to find changes. |
AGGREGATING | The metadata for this volume is currently being aggregated. |
IDLE | The system is not currently performing any pull, scan or aggregation actions on this volume. |
The Volume Scan Detailed Status panel has a notes column that offers users additional information and suggested actions. Potential notes include the following:
Note | Description | Course of Action |
|---|---|---|
Could not lock volume {unifs_volume_id}, skipping scan | This message appears if the scanning operation does not receive exclusive (within the Nasuni File IQ Appliance) access to the volume. When this occurs, the scan skips and tries again every 10 minutes. This might occur when another internal Nasuni File IQ Appliance process performs a task against the volume - for instance, reporting. Note: User access to the volume is unaffected. This is only internal to the Nasuni File IQ Appliance. | This issue typically corrects itself automatically, and user access to the volume is unaffected. |
Database seems to be full or down | This message indicates that the Nasuni File IQ Appliance database is either unavailable or has reached its capacity. | If you have a Premium license, check the Configuration/Retention Configuration in the File IQ Edge UI. Otherwise, start by adding 512GB or 1TB size to the Nasuni File IQ database disk. |
Caught unknown exception trying to process filesystem Metadata! | There has been an unexpected error when scanning the volume. | Under normal operations you should only see this when the system or processes are restarted. |
Volume is not Read-Only | The volume is not shared read-only with the appliance, and therefore is not scanned. | In the NMC, update the remote access to the volume to custom/read-only, disconnect, and reconnect the volume. |
Nasuni File IQ Reporting Volume
The list does not display the Nasuni File IQ reporting volume. If a reporting volume is shared and connected to the Nasuni File IQ appliance but has not been configured as the reporting volume, it appears here as failed, with a message stating “Volume is not Read-Only.” To resolve this issue, disconnect the volume or configure it as the Nasuni File IQ reporting volume using the Nasuni File IQ user interface. Once a volume becomes the reporting volume, a scan must occur for the scanning process to recognise it as the reporting volume. The scanning process typically occurs within 24 hours, during which time the volume remains on the list.
Appendix
For a full list of supported file extensions, see the File IQ Dashboards – Supported File Extensions.
Event Categories and Types
Event Category | Event Type |
Create | Create File Create Directory Create Symbolic Link Create Hard Link |
Delete | Delete File Delete Directory |
Rename | Rename |
Security | Change Owner Change Permissions Set ACL Set DOS Attribute |
Security/Metadata | Set Extended Attributes |
Metadata | Change Access/Modification Times |
Write | Write to File Truncate File Fast Push Data Push |
Read | Open File Open Directory Read File Read Directory List Extended Attributes Read Extended Attributes |
Shared Link | Access Shared Link Create Shared Link Modify Shared Link Delete Shared Link Regenerate Shared Link |
Activity and Audit Event Dashboards Data Retention
The File IQ Activity and Audit Event Dashboards use different data retention policies in place. The following table indicates the default data retention policy used by each Activity and Audit Event dashboard:
Category | Dashboards | DATA RETENTION | Notes |
Activity | Directory Summary | 90 days | The data displayed in this dashboard is limited to 90 days. |
Activity | Group Drilldown | 90 days | The data displayed in this dashboard is limited to 90 days. |
Activity | Group Summary | 90 days | The data displayed in this dashboard is limited to 90 days. |
Activity | Summary | 90 Days | The data displayed in this dashboard is limited to 90 days. |
Activity | User directory Access | 21 days | The data displayed in this dashboard is limited to 21 days. |
Audit Events | Events | 90 Days | The data displayed in this dashboard is limited to 90 days. |
Audit Events | Inspector | 7 days | The data displayed in this dashboard is limited to 7 days. |
Reporting Volume Availability in Grafana
The reporting volume is managed independently from other volumes connected to the File IQ appliance. As a result, the availability of associated audit events and metadata differs from that of other volumes.
Activity And Audit Event Dashboards
Activity on the reporting volume is displayed in the Grafana Audit Event and Activity Dashboards for all Nasuni Edge Appliances that either own or are connected to the reporting volume, have File IQ enabled, and are assigned to a File IQ appliance.
Metadata Dashboards
Metadata from the reporting volume is not displayed in Grafana metadata dashboards, including Volume Usage, Volume Summary, Volume Explorer, and related dashboards. Additionally, the reporting volume does not appear in the volume selection drop-down menus.