These docs are for Cribl Stream 4.19 and are no longer actively maintained.
See the latest version (4.20).
Internal Logs
Cribl Stream generates internal application logs that monitor its own operations and health. They provide valuable insights into the system’s behavior, performance, and potential issues.
Distributed deployments emit a larger set of logs than single-instance deployments. We’ll describe the distributed set first.
You can display and export all internal logs by selecting Monitoring in the sidebar, then selecting Logs (Stream) or selecting Logs in the sidebar (Edge). Logs’ persistence depends on event volume, not time - for details, see Log Rotation and Retention.
Several logs listed on this page are exposed only in customer-managed (on-prem) deployments. In Cribl.Cloud, Leaders support Cribl Stream Worker Node logs on hybrid Workers.
However, Organization Members who have the Admin Permission on Cribl Search can use that product to search the
cribl_internal_logsDataset for additional details about the Leader and its Cribl-managed Workers.
Leader Node Logs (Distributed)
The API/main process emits the following logs into the Leader Node’s $CRIBL_HOME/log/ directory.
| Logfile Name | Description | Equivalent on Logs page |
|---|---|---|
cribl.log | Principal log in Cribl Stream. Includes telemetry/license-validation logs. Corresponds to top-level cribl.log on Diag page. | Leader > API Process |
access.log | API calls, for example, GET /api/v1/version/info. | Leader > Access |
audit.log | Actions pertaining to files, for example, create, update, commit, deploy, delete. | Leader > Audit |
notifications.log | Messages that appear in the Notification list in the UI. For each event, the time and _time fields record the start of the monitored interval (matching starttime), not when the Notification was written. | Leader > Notifications |
ui-access.log | Interactions with different UI components described as URLs, for example, /settings/apidocs, /dashboard/logs. | Leader > UI Access |
The API/main process emits the following service logs into the Leader Node’s $CRIBL_HOME/log/service/ directory. Each service includes a cribl.log file that logs the service’s internal telemetry and an access.log file that logs which API calls the service has handled.
| Service Name | Description | Equivalent on Logs page |
|---|---|---|
| Connections Service | Handles all Worker connections and communication, including heartbeats, bundle deploys, teleporting, restarting, and so on. Workers are assigned to connection processes using a round-robin algorithm. | Leader > Connections Service |
| Lease Renewal Service | Handles lease renewal for the primary Leader Node. | Leader > Lease Renewal Service |
| Metrics Service | Handles in-memory metrics, merging of incoming packets, metrics persistence and rehydration, and UI queries for metrics. | Leader > Metrics Service |
| Notifications Service | Triggers Notifications based on its configuration. | Leader > Notifications Service |
The Config Helper process for each Worker Group/Fleet emits the following log in $CRIBL_HOME/log/group/GROUPNAME.
| Logfile Name | Description | Equivalent on Logs page |
|---|---|---|
cribl.log | Messages about config maintenance, previews, and so on. | GROUPNAME > Config helper |
Outpost Logs
The Outpost service emits the following logs into the $CRIBL_HOME/log/service/outpost/N/ directory,
where N is the Outpost Node Process ID:
| Logfile Name | Description | Equivalent on the Logs tab in Outpost info drawer |
|---|---|---|
cribl.log | Internal telemetry logs. | Outpost Service N |
metrics.log | Outpost Node metrics. | Outpost Service N - Metrics |
Worker Node Logs (Distributed)
The API Process emits the following log in $CRIBL_HOME/log/.
| Logfile Name | Description | Equivalent on Logs page |
|---|---|---|
cribl.log | Messages about the Worker/Edge Node communicating with the Leader Node (that is, with its API Process) and other API requests, for example, sending metrics, reaping job artifacts. | GROUPNAME > Worker:HOSTNAME > API Process |
Each Worker Process emits the following logs in $CRIBL_HOME/log/worker/N/, where N is the Worker/Edge Node Process ID. (The metrics.log file is written only when HTTP-based Destinations exist.)
| Logfile Name | Description | Equivalent on Logs page |
|---|---|---|
cribl.log | Messages about the Worker/Edge Node processing data. | GROUPNAME > Worker:HOSTNAME > Worker Process N |
metrics.log | Messages about the Worker/Edge Node’s outbound HTTP request statistics. | GROUPNAME > Worker:HOSTNAME > Worker Process N |
If TCP load balancing is enabled in at least one supported Source (such as Syslog, TCP JSON, or Cribl TCP),
the Worker Process also emits the following log to $CRIBL_HOME/log/worker/LB:
| Logfile Name | Description | Equivalent on Logs page |
|---|---|---|
cribl.log | Internal telemetry of the load balancing process. | GROUPNAME > Worker:HOSTNAME > Load Balancer |
Load balancing logs use different channel names to indicate the Source they come from than Worker Process logs. To search by channel name, use
channel == "<source-id>_lb"instead ofchannel == "input:<source-id>". For example, for the built-in TCP JSON Source, enterchannel == "in_tcp_json_lb"in the search box instead ofchannel == "input:in_tcp_json".
For convenience, the UI aggregates the Worker/Edge Node Process logs as follows.
| Logfile Name | Description | Equivalent on Logs page |
|---|---|---|
| N/A | Aggregation of all the Worker Process N logs and the API Process log. | GROUPNAME > WORKER_NAME |
In Cribl Stream, the logs listed above are currently available only on customer-managed hybrid Workers. The single-instance logs listed below are not relevant to Cribl.Cloud.
Single-Instance Logs
The API/main process emits the same logs as it does for a Distributed deployment, in$CRIBL_HOME/log/:
cribl.logaccess.logaudit.lognotifications.logui-access.log
Each Worker/Edge Node Process emits the following logs in $CRIBL_HOME/log/worker/N/, where N is the Worker/Edge Node Process ID. (The metrics.log file is written only when HTTP-based Destinations exist.)
| Logfile Name | Description | Equivalent on Logs page |
|---|---|---|
cribl.log | Messages about the Worker/Edge Node processing data. | GROUPNAME > Worker:HOSTNAME > Worker Process N |
metrics.log | Messages about the Worker/Edge Node’s outbound HTTP request statistics. | GROUPNAME > Worker:HOSTNAME > Worker Process N |
_raw stats Event Fields
Each Worker/Edge Node Process logs this information at a 1-minute frequency. So each event’s scope covers only that Worker/Edge Node Process, over a 1-minute time span defined by the startTime and