Configure and Use File IQ Advanced Filtering for File Metadata

Advanced Filtering for File Metadata is a Premium License feature, currently in early access, introduced in File IQ version 10.2. This feature allows users, including storage and IT administrators, to filter the file metadata that File IQ discovers.

With Advanced Filtering for File Metadata, users can easily identify files that meet specific criteria.

This feature is beneficial for tasks such as:

• Finding the 10 largest files.

• Locating files owned by a specific user.

• Identifying files that contain "projectA" in the file path.

• Identifying access control (ACL) permissions for files and directories (AD-backed UniFS volumes in NTFS exclusive mode only).

Important: Due to early access, certain functionality might change. You might encounter limitations or unexpected behavior.

Enable Advanced Filtering in File IQ

Advanced Filtering is configured through a two-step process. First, enable the feature. Second, enable volume scanning and indexing. After indexing completes, Advanced Filtering can be enabled or turned off on a per-volume basis.

To configure the File IQ Advanced Filtering, follow these steps:

1. Navigate to the Nasuni File IQ Edge User Interface using https://FILE_IQ_FQDN:8443, and log in as an Administrator.

2. Click Configuration > Advanced Filtering.

   

3. Check the Enable File IQ Metadata Filtering box.

4. Click Save.

Enable Advanced Filtering for a Volume

The second component of the configuration is to enable Advanced Filtering on your volumes. This section outlines that process.

Note: If a volume you recently added does not appear, wait until its scan is complete. Use the Service Support dashboard to monitor ongoing volume scans and ensure the volume is ready for filtering.

To configure a volume for Advanced Filtering, follow these steps:

1. Navigate to the Nasuni File IQ Edge User Interface using https://FILE_IQ_FQDN:8443, and log in as an Administrator.

2. Click Configuration > Advanced Filtering.

3. Under Volume Settings, check the box for the volume or volumes you want to apply Advanced Filtering to.

4. Click Edit (X) Volumes.

5. Click the Enabled toggle to On.
   

6. Click Save.

How Volume Indexing Works

Advanced filtering is available only for indexed volumes. Before you can use advanced filtering, a volume must be indexed first.

Volumes are indexed after scanning is complete, typically once a day. Volume indexes are updated daily.

A maximum of two volumes can be indexed simultaneously. If additional indexing tasks are pending, they are queued and processed after the previous index tasks are complete.

Note: The current volume indexing process does not utilize File IQ’s incremental scan capabilities. As a result, each indexing run takes approximately the same amount of time as the initial indexing, even if there have been little changes to the underlying volume.

Monitor Volume Indexing Status

To monitor volume indexing, open the Service Support dashboard in Grafana and scroll down to the Volume Indexing panel. This panel lists all volumes, whether they are enabled for indexing, and their current indexing state.

The Volume Indexing panel provides the following volume information:

The indexing status can be one of the following:

Volume Indexing History

Below the Volume Indexing panel is the Volume Indexing History panel. This panel includes the same information as the Volume Indexing panel, along with a timestamp column and an additional status option. By default, this panel is collapsed.

Volume indexing history is retained for one year and automatically deleted after this period.

Use the Advanced Filtering Dashboard

Volume indexing must occur before the volume appears in the Volume Name drop-down menu and is available for use in Advanced filtering. Volumes are scheduled for indexing after scanning is complete, typically once a day. Therefore, volume indexes are updated daily.

Dashboard Fields and Controls

The following numbers correspond to the dashboard components:

ACL-Based Access Filtering in the File Metadata Advanced Filtering Dashboard

Starting with File IQ version 10.3, Advanced Filtering supports access control list (ACL)–based filtering for Active Directory (AD)–backed UniFS volumes configured in NTFS exclusive mode. ACL-based filtering includes identity and permission filters, which can be combined with other filters to refine results.

Note: ACLs are stored as they are defined. Nested group memberships are not resolved.

Identity Filters (User/Group)

For supported volumes, you can specify the following:

• A user (for example, CORP\alice).

• A group (for example, CORP\Finance, BUILTIN\Administrators).

Note: Email formats (for example, alice@corp.com) are not supported.

SIDs are displayed in the Access Control pop-up for users and groups that cannot be resolved. It is not currently possible to filter files and directories directly by SID.

Permission Filters

Permission filters are based on the user-friendly allow permission types exposed by File IQ’s ACL decoder and Access Control pop-up, such as:

• Full control (FULL_CONTROL)

• Modify (MODIFY)

• Read & execute (logical combination of READ and EXECUTE)

• Read (READ)

• Write (WRITE)

• Special permissions (includes anything not cleanly fitting into the previous types)

How ACL Filters Combine with Other Filters

ACL filters are ANDed with other dashboard filters to produce results. These filters can include path, filename, size, owner, and date filters that further restrict the result set. For example: “Files under /finance where CORP\Finance has Read access and size > 1 GB.”

Internally, a file or directory is returned only if it satisfies all selected criteria. If several access types are specified in the Advanced Filtering dashboard, only the files and directories for which the specified user or group meets all filtering criteria are returned.

ACL Example Workflows

Use the following workflows to answer common ACL-based questions.

What can user X read under a given folder?

To see what user X can read within a given folder, follow these steps:

1. Open the Advanced Filtering - File Metadata dashboard.

2. Select an AD-backed UniFS volume configured in NTFS exclusive mode.

3. Set the Path filter to the folder of interest (for example, /finance/reports).

4. Set the User or Group Name ACL filter to CORP\alice.

5. Set the permission filter to Read.

6. Click Apply.

The table shows only objects that CORP\alice can read in that subtree.

Where does a sensitive group have full control?

To see where a sensitive group has full control, follow these steps:

1. Open the Advanced Filtering - File Metadata dashboard.

2. Select an AD-backed UniFS volume configured in NTFS exclusive mode.

3. In the ACL filters:

   • Set User or Group Name to CORP\Domain Admins (or another sensitive user or group).

   • Set permission to Full control.

4. (Optional) Narrow the Path filter to a high-value area (for example, /prod).

5. Review the results to identify directories and files with broad privileges.

Verifying remediation

After tightening ACLs on the Edge Appliance, follow these steps:

1. Run a new scan and allow indexing to complete.

2. Open the Advanced Filtering - File Metadata dashboard.

3. Filter for the previously overexposed group (for example, CORP\Everyone) and the relevant permission (for example, Read).

4. Click Apply.

5. Confirm that the group no longer has inappropriate permissions on sensitive paths.

Understand the Results Table

After running a query, the results display in the table below Messages. Clicking a file or directory opens the Volume Explorer dashboard for the selected volume and the selected file or directory.

The Results table provides the following information:

*Important: Access times for files and directories are updated when audit events are received from a Network Edge Appliance (NEA) running version 10.2 or later. To ensure accurate updates, configure all NEAs connected to the filtered volume to send audit events to this File IQ appliance. Note that access times for files and directories accessed before upgrading the NEAs to version 10.2 are unavailable. In such cases where access time is not available, the modified time is used for filtering and display purposes.

Run an Advanced Metadata Filter

To use the Advanced Filtering dashboard, follow these steps:

1. From the Dashboards Homepage, select the Advanced Filtering - File Metadata dashboard.

2. Navigate to the Volume Name field, and choose a volume from the drop-down.

   

3. From the File/Directory Limit drop-down, select the number of items to display in the result table.

   Note: You can display up to 10,000 items.

   Note: The Total File Count, Total File Size, and Total Directory Count display statistics for all files and directories that match your search criteria, irrespective of the File/Directory Limit.

4. Select your filter criteria from the filters located on the left-hand side. Expand and collapse sections as required.

   Use the following tips to get the most out of your filtering:

   • Any field included in the filter has a blue tick next to it, as does its section header.

   • Basic wildcards (* and ?) are accepted for all text strings.

   • Empty or zero-number fields are not included in the filter.

   • The sort order you select determines which top items appear in the filter results, making it helpful in retrieving specific "Top N" files and directories, such as the top 10 largest files. The items displayed in the results table are ranked based on the chosen sort order. After the results are displayed, they can be sorted independently according to the table's settings, which can be adjusted by clicking a column header. The default sort order is based on the Rank column. If you click a different column to sort the table, the Rank column is no longer used for sorting, and the list might not appear in the original ranked order defined by your selected sort option.

   • Click Reset All at any time to reset all filters and sort order to their default values.

5. Click Apply.

6. (Optional) If you are using File IQ version 10.3 or later and want to export the results, proceed to step 7.

   If you are using File IQ version 10.2 or earlier and want to download the results, skip to step 8.

7. Click CSV Report.

8. To download your results, follow these steps:

   1. Click the three-dot button on the top right of the table.

      

   2. Select Inspect > Data.

      

   3. Select whether you want formatted or raw data, and if you want to download an Excel-compatible CSV file.

      

   4. Click Download CSV.

Example Filter Scenarios

The table below shows example filter configurations for common search scenarios. Use these examples as a starting point to customize your search.

*Important: Access times for files and directories are updated when audit events are received from a Network Edge Appliance (NEA) running version 10.2 or later. To ensure accurate updates, configure all NEAs connected to the filtered volume to send audit events to this File IQ appliance. Note that access times for files and directories accessed before upgrading the NEAs to version 10.2 are unavailable. In such cases where access time is not available, the modified time is used for filtering and display purposes.

Unsupported Filtering Scenarios

This dashboard filters only file and directory metadata, not audit events. For example, you cannot filter for all files accessed or modified by a specific user, as this would require filtering all past audit events.

Note: File IQ 10.2 and previous versions do not retrieve complete access control lists (ACLs) for individual files and folders. As a result, you must be on version 10.3+ to filter based on user or group access permissions.

Advanced Metadata Filtering – CSV Report

The Advanced Metadata Filter - CSV Report is a Premium License feature available in File IQ version 10.3 and onwards that allows users to export the complete result set from the Advanced Metadata Filtering dashboard into a CSV file.

While the Advanced Metadata Filtering table is limited to 10,000 results for performance reasons, the CSV export is designed to handle large result sets efficiently by running as a background report. Each export supports up to 1 million rows per CSV file.

The export process is integrated with the File IQ reporting module, allowing you to monitor progress, cancel running exports, and download completed files from the Report Management page.

The CSV file contains one row per file or directory and includes the following columns:

• File name: The name of the file.

• File path: The file path.

• Owner: The entity that owns the file or directory.

• Group Owner: The group the owner entity belongs to.

• Created: The date that the file or directory was created. Dates are formatted as: YYYY-MM-DD HH:MM:SS(UTC).

• Modified: The date that the file or directory was last modified. Dates are formatted as: YYYY-MM-DD HH:MM:SS(UTC).

• Accessed: The date that the file or directory was last accessed. Dates are formatted as: YYYY-MM-DD HH:MM:SS(UTC).

• File size: The size of the file. File and directory sizes are displayed in human-readable units (for example, B, kB, MB, GB).

• Directory size: The size of the directory. File and directory sizes are displayed in human-readable units (for example, B, kB, MB, GB).

• Is File: A boolean value (either TRUE or FALSE) to differentiate results that are files from those that are folders.

Exporting the Advanced Metadata Filtering Results

After running a search in the File IQ Advanced Metadata Filtering dashboard, you can export your results to a CSV file.

Note: This feature requires Reporting to be properly configured and operational. For more information on configuring Reporting, see File IQ Reporting.

To export the Advanced Metadata Filtering dashboard to CSV, follow these steps:

1. Navigate to the Advanced Metadata Filtering dashboard.

2. Use the fields and search criteria to execute a search. For more information, see Using the Advanced Filtering Dashboard.

3. Click Apply to run the search.

4. Click CSV Report to start the CSV export.

   Note: If the button is disabled, there are no results to export.

5. The Confirm CSV Export box appears. Click Ok to continue.

   Note: If the number of matching records exceeds the configured export limit (default: 1,000,000 rows), the dialog displays a warning indicating that the export is truncated and that only the first 1,000,000 rows are included in the CSV file.

Monitoring and Downloading Exports

After executing the Advanced Metadata Filter CSV Report, the status displays on the Nasuni File IQ Report Management page.

To check the status of the export report, navigate to the File IQ UI and click Reporting → Report Management. The status appears under the Status column in the Advanced Metadata Filter Export to CSV row.

• If Web Access is configured, the Status Details column displays a downloadable link to the generated ZIP file.

• If Web Access is not configured, the Status Details column displays the file path within the reports volume. The file must then be retrieved directly from the reports volume.

Cancelling the Advanced Metadata Filter Export to CSV Report

To cancel the Advanced Metadata Filter Export to CSV report after it has been executed, follow these steps:

1. Navigate to the Nasuni File IQ Edge User Interface using https://FILE_IQ_FQDN:8443, and log in as an Administrator.

2. Click the Reporting tab → Report Management.

3. On the Nasuni File IQ Report Management page, check the box next to the report, and click Cancel Now.

   

Running Multiple CSV Exports

When multiple exports are triggered, report executions are queued automatically. The Report Management page in the File IQ UI displays only the most recent execution, while all queued exports continue to run in the background.

The Report History in Grafana provides the full execution history. To access the Report Management view from the Grafana Homepage, scroll down to the bottom of the page and click Report History.

Troubleshooting

ACL Data Does not Appear in Dashboards

ACL filters are missing in Volume Explorer, Volume Version Explorer, or Advanced Filtering.

Check the following:

• Volume type: Confirm the volume is AD-backed and configured in NTFS exclusive mode.

• Feature flags and license: Verify that ACL scanning is enabled for this volume.

• Scans: Ensure at a full volume rescan completed after upgrading to File IQ 10.3.

• Indexing: Check that volume indexing tasks for the relevant scan completed successfully.

Advanced Filtering is not Available in the File IQ UI

If you do not see the Advanced Filtering menu option in the File IQ user interface and are using version 10.2, it indicates that this feature has not been enabled for your account. If you are a File IQ Premium user, reach out to your Nasuni account representative to discuss the availability of this feature.

Advanced Filtering Configuration Page Shows “Nasuni File IQ Premium not Licensed”

Advanced Filtering is a premium feature and is not included in the File IQ Basic plan.

To begin using this feature, contact your Nasuni account representative. They can guide you through the licensing process, answer any questions you might have, and ensure a smooth transition to accessing this premium feature.

CSV Export is Truncated

Issue: The exported CSV file contains fewer rows than expected and is marked as (truncated).

Cause: The number of matching records exceeds the maximum export limit (default: 1,000,000 rows).

CSV Export Stuck in Queued State

Issue: In File IQ Report Management, the Advanced Metadata Filtering – Export to CSV job remains in Queued status and does not start executing.

Cause: Reporting is not configured. The CSV export runs as a background report and depends on the Reporting framework being properly configured.

Filters Return no Results

Applying ACL filters for a user or group yields an empty result set, even when you expect access.

The following are possible causes and remedies:

• Name mismatch: Ensure you are using the same form of the name that the Edge Applience (filer)/AD uses (domain prefix, case, and so on). Make sure you use the SAM account name (DOMAIN\user).

• Wrong volume or path: Confirm you are searching on the correct volume and under the correct path prefix.

• Scans: Ensure that a full volume rescan completed after installing or upgrading to File IQ 10.3.

• Indexing: Check that volume indexing tasks for the relevant scan completed successfully.

• Scan or indexing lag: ACLs might have changed on the Edge Appliance since the last scan; run a new scan and wait for indexing.

• Complex Filters: When several filters or several access types are selected, only files and directories that match all selected filters will be returned. Try to simplify your query first.

• Overly strict filters: Temporarily remove non-essential filters (for example, size or date filters) to verify that ACL filtering works in isolation.

Link to the Report not Displayed

Issue: The Status Details (in FIQ UI) or Download column (in Dashboard) does not show a path or link to the CSV file after the export completes.

Cause: A reporting share has not been configured, or Web Access is not enabled (necessary to see a direct download link to the CSV file).

Lengthy Indexing Times

The following factors usually drive indexing time:

• Volume size: Large volumes (over 100 million files) might take several hours to index.

• VM memory size: Ensure it is suitably sized by navigating to Status > Nasuni File IQ Sizing in the File IQ UI.

• File IQ DB disk specifications: Since both the File IQ database and index are on the same File IQ DB disk, you might need to increase IOPS or throughput to ensure adequate performance. You can use your cloud provider’s virtual machine and disk monitoring performance charts to monitor disk utilization. Also, ensure that the File IQ DB disk meets the performance requirements specified in the sizing spreadsheet.

Memory Usage

File IQ might require additional memory for Advanced Filtering.

We recommend checking the memory requirements in the File IQ UI under Status > Nasuni File IQ Sizing to confirm them.

Results Differ from Current Edge Appliance Permissions

On the Edge Appliance, user CORP\alice appears to have access, but File IQ shows no results, or vice versa.

The following are possible explanations:

• Time lag: File IQ reflects ACLs from the last completed scan; recent permission changes are not visible until the next scan and index run.

• Inheritance nuances: Complex inheritance chains can be interpreted differently. File IQ computes effective permissions based on stored ACLs at scan time.

Recommended actions:

1. Wait for a fresh scan for the volume.

2. Wait for indexing to complete.

3. Re-test with the same filters.

4. Simplify your filters.

Unable to Execute Export from Report Management

Issue: The Execute Now option is disabled for the Advanced Metadata Filter Export to CSV report in Report Management.

Cause: This export can only be triggered from the Advanced Metadata Filtering dashboard, where filter criteria and volume selection are defined.

Common Questions

Q: Do ACL changes take effect immediately in dashboards?

A: No. Changes are reflected only after the next volume scan and subsequent indexing run.

Q: Does File IQ modify ACLs on the Edge Appliance?

A: No. File IQ is read-only with respect to ACLs; it neither changes nor enforces permissions.

Q: Is ACL data itself sensitive?

A: Yes. ACLs reveal who can access which data. Restrict Grafana access to trusted administrators and security staff.