On This Page

Home / Stream/ Integrations/ Sources/ Collector Sources/Script Collector

Script Collector ​

Cribl Stream supports flexible data collection configured by your custom scripts.

In Cribl.Cloud, the Script Collector is only available on customer-managed hybrid Worker Nodes.

How the Collector Pulls Data ​

When you run a Script Collector in Discovery mode, the first available Worker returns one line of data per discovered item. Each item (line) turns into a Collection task on the Leader Node.

In Full Run mode, the Leader passes the item to collect into the script in the $CRIBL_COLLECT_ARG variable, and spreads collection across all available Workers.

If you run the Script Collector with a given Time Range, the earliest and latest values will be available to the script via the environment variables EARLIEST and LATEST, respectively.

Important: Secure and Use the Script Collector Properly ​

Be aware that scripts will allow you to execute almost anything on the system where Cribl Stream is running. Make sure you understand the impact of what you’re executing before you do so. These scripts run with the same level of permissions as the user running Cribl Stream. So, if you are running it as root, these commands will run with root user permissions. If your system requires greater security, consider disabling the Script Collector. See How to Disable the Script Collector for more information.

Also, always validate and test your scripts before running them on your system. Scripts must finish on their own. The Collector will not stop a script once its task has started. You must ensure scripts do not use infinite loops.

Configure a Script Collector ​

  1. Navigate to Products > Stream > Worker Groups. Select a Worker Group, then go to Data > Sources. Choose the Source and select Add Collector.
  2. In the New Collector modal, configure the following under Collector Settings:
    • Collector ID: Unique ID for this Collector. E.g., sh2GetStuff. If you clone this Collector, Cribl Stream will add -CLONE to the original Collector ID.
    • Discover script: Script to