The Windows file shares import agent for Elimity Insights imports principals, permissions and resources from Windows files shares and uploads the results to your Elimity Insights server. You only have to configure the SMB credentials and the connection with your Elimity Insights server. The import agent will then connect to Windows file shares, perform the necessary requests, transform the results to a format compatible with Elimity Insights and send the transformed data to your Elimity Insights server.

Installation

The import agent is available as a Docker image for Linux and Windows. Refer to the dedicated knowledge base article for all details about working with agents and gateways. Visit https://console.cloud.google.com/artifacts/docker/elimity-general/europe-west1/docker/smb-import-agent to get a list of available image tags for this specific agent.

Step-by-step deployment guide

The following sections will explain the different steps you'll need to take to deploy the Windows file shares agent for Elimity Insights.

1. Creating a Windows file shares source in Elimity Insights

Firstly create a new Windows file shares source in Elimity Insights, but do not enable automatic imports. Instead, generate API credentials for this source and note down the resulting identifier and token.

2. Configuring the agent

To configure your import agent, mount a JSON configuration file at `/app/config/config.json` with the properties listed below. You can find an example in the attachments at the bottom of this page.

• `insights_certificate.type`: one of `absent` or `present`
  • if the certificate type is `present`:
    • `insights_certificate.certificate_file`: path of the mTLS client certificate file
    • `insights_certificate.key_file`: path of the mTLS client key file
• `insights_source_id`: source identifier you noted down in step 1
• `insights_source_token`: source token you noted down in step 1
• `insights_url`: URL of your Elimity Insights server
• `insights_verify_ssl`: verify SSL certificate of the Elimity Insights server, default `true`
• `log_sink.type`: one of `file` or `std_err`
  • if the sink type is `file`:
    • `log_sink.file`: path of the log file
• `max_depth`: global upper limit on the scanning depth, 0 to indicate no upper limit
• `max_depth_overrides`: object mapping file identifiers to max depth overrides
  • if the import agent encounters a file for which the identifier occurs in this object, then it resets the upper limit for scanning depth to the matched value
  • this is especially useful if you want to scan specific folders more deeply, or if you want to entirely skip stepping into some folders
• `only_dirs`: only scan folders, not files
• `schedule`: optional object describing when the import agent should run; omit if you just want to run the agent once
  
  • `day`: optional weekday when the import agent should run (e.g. `monday`); omit if you want to run the agent every day of the week
  • `time`: time-of-day in `HH:MM` format
• `smb_password`: password for authenticating to the SMB server
• `smb_username`: username for authenticating to the SMB server
  • Note: Depending on your configuration, it might be that you have to provide a fully qualified username including your domain, e.g., "username@yourdomain.com" instead of just "username"
• `targets`: a list of configuration objects describing which SMB servers to import from
  • `connection_pool_size`: maximum number of parallel SMB connections to use for importing from this target, default `1`
  • `filter.type`: one of `all` or `allow_list`
    • if the filter type is `allow_list`:
      • `filter.selections`: a list of configuration objects describing which shares to import from
        • `max_depth`: optional override for the global upper limit on scanning depth
        • `path`: file path within the share to start scanning from
        • `path_prefix`: prefix to apply for file paths when importing into Elimity Insights
        • `share`: Windows file share to import from
  • `server`: hostname of the SMB server to connect with
• `timeout`: timeout in seconds for waiting on SMB server responses

3. Deploying the agent

Having configured the agent and having created a source in Elimity Insights, you can now deploy the agent to regularly import data from your Windows file shares and upload it to Elimity Insights. Since we distribute the agent as a Docker image, our recommendation for deployment is to use your cloud provider’s dedicated job execution platform (e.g. Google Cloud Run, Azure Container Apps, …). If that's not an option, you can also manually deploy the image on e.g. Windows Server. Refer to the dedicated knowledge base article about installing import agents for additional details.

4. Following up on the import

The import agent outputs logs to indicate its progress, for a manual Windows Server deployment you can check these with `docker-compose logs smb-import-agent`.

Guidelines for Importing Large File Share Repositories

Importing large file share repositories can be challenging due to several factors. Scanning millions of files can be time-consuming and may put a significant load on your file server. Additionally, analyzing the resulting data can be complex. To ensure an efficient process, it’s important to define your goals and start small with the initial import, expanding gradually.

To help you along, you can follow the following steps for efficient file share imports:

1. Define Your Goals

• Identify the purpose of the import. For example:
  • Cleaning up access across your entire repository.
  • Monitoring access to critical files or folders.
• Evaluate the structure of your current File Share permissions. If explicit access is granted only to top-level folders, start by importing the permissions for those folders.

2. Plan the Import Scope

• List the File Share servers to scan.
• Specify folders to include:
  • Decide whether to scan all folders or only specific ones.
  • Choose whether to scan sub-folders only or include individual files, and how many levels deep.

3. Assess Folder Sizes

• Analyze the contents of the selected folders, estimating the number of files and sub-folders.

4. Model the Data in Elimity Insights

• Choose between one large source or several smaller sources:
  • One large source provides a unified view but may cause heavy imports.
  • Smaller sources allow separate scheduling and potentially better performance.
• Consider the following:
  • Folder size: Limit each source to 200,000 files or folders to keep import times manageable.
  • Data freshness: Is it okay to import the data for example once a week, or do you need more frequent insights in the access to critical folders?
  • Server load: If the scan takes a while, is the server still performing well enough for other users? If not, should you limit the import to night times or even weekends?
  • Access control: Is every user of Elimity Insights allowed to view all data, or do you want to limit certain users to specific folders? If so, you can split up the data in different sources and then use the access profiles of Elimity Insights to grant users access to the sources they need.

5. Give the agent permission to scan the chosen folders

• Create a user account that has read-only permissions on all the needed folders and their contents.

6. Configure the Data Import Agent

Use Elimity's agent to manage imports based on your defined schedule. Refer to the detailed setup instructions above.