These docs are for Cribl Stream 4.17 and are no longer actively maintained.

See the latest version (4.20).

Cribl Stream can process a syslog stream directly. Moving to Cribl Stream from existing syslog-ng or rsyslog servers fully replaces those solutions with one that is fully supported and easily managed.

Processing syslog in Cribl Stream allows you to readily address these common challenges of ingesting data from syslog senders:

• Architecture: Cribl Stream routes the syslog stream directly, immediately, and securely to the destinations of your choice, reducing latency and management overhead.
• Volume: Removing redundant data and unnecessary fields in Cribl Stream typically reduces volume 20-30% overall. It also optimizes the data for downstream services like Splunk or Elasticsearch.
• Timestamp handling: Cribl Stream intelligently processes events sent from different time zones. It can embed a new, consistent timestamp, and can auto-correct timestamps that are off by an exact number of hours.
• Severity/Facility accuracy: Each syslog event begins with a bracketed integer that represents Facility and Severity, as defined in the Syslog Protocol. Cribl Stream translates this code (e.g., <165>, <0>) into the correct Facility and Severity values.
• Metadata: Cribl Stream can automatically set metadata fields, including sourcetype and index.

This tutorial outlines best practices for replacing your syslog server with Cribl Stream. To go even a bit deeper, check out this Cribl Office Hours video.

The Goal: Optimizing Syslog Events ​

By default, a Cribl Stream Syslog Source produces eight fields: _time, appname, facility (numeric), facilityName (text), host, message, severity (numeric), and severityName (text).

While this default parsing makes the output much more readable, we haven’t saved any volume - and we now have redundant pairs of fields (numeric versus text) that represent facility and severity.

Our next logical step is to streamline syslog events to something more like this:

This accomplishes all of the following:

• Extracts the essentials.
• Removes the redundancies.
• Adds a new field to identify the Cribl Stream Pipeline (which we’re about to build).
• Adds metadata that the Destination needs.
• Shrinks the outbound _raw payload to just its message component.

Once we optimize syslog in this way, we can achieve still further efficiencies by dropping or downsampling frequent events, and by balancing high-volume syslog inputs across Cribl Stream worker processes.

Architecture: On-Prem Cribl Stream Deployments​ ​

Syslog data, especially when sent via UDP, is best collected as close to the source as possible. Ideally, you should capture syslog data at its origin. (This is true of syslog in general, not just syslog processed in Cribl Stream.)

Also, because syslog senders have no built-in load balancing, Cribl strongly recommends using a load balancer to distribute the load across multiple Worker Nodes.

The load balancer shown above is dedicated only to communication between the syslog senders and the Stream Worker Nodes. Any load balancing between the Worker Group and its Stream Leader would be handled by a separate load balancer.

Load Balancers ​

The load balancer in front of the Leader UI can be an Application Load Balancer or a Network Load Balancer.

The load balancer between the Cribl Workers and the Cribl Leader listening on port 4200 must be a Network Load Balancer or other load balancer that supports health checks.

For load balancer configuration examples showing how to distribute syslog traffic across multiple Cribl Stream Worker Nodes, see Third-Party Load Balancer Example Configurations for Cribl Stream Syslog.

High-Volume Syslog Senders ​

For general guidance on handling high-volume senders in Cribl Stream, see Scaling for High-Volume Syslog Senders for details on capacity planning, TCP load balancing, and other performance considerations.

For individual high-volume syslog senders sending via TCP above 300 GB/day, it is recommended to enable the TCP Load Balancing function (available in Cribl Stream single-instance and Distributed deployments). This feature prevents “TCP pinning” by distributing the syslog traffic across all Worker Processes, improving overall performance. When enabled, a dedicated load balancer process distributes incoming data via IPC sockets. For details, see TCP Load Balancing.

UDP Versus TCP ​

For any given syslog device, you might need to choose between using UDP or TCP. Which is best depends on the characteristics of the sender. Here are some basic guidelines:

• For both the sender and Cribl Stream, UDP imposes lower overhead than TCP. The stateless nature of UDP allows each log event to be directed to a different worker thread than the last. This ensures maximum utilization of the Worker Nodes. See Sizing and Scaling for more details.
• For lower-volume senders, use TCP if the sender supports it.
• For all other use cases, use UDP.

Optimize Syslog with Cribl.Cloud ​

When deploying syslog solutions with Cribl.Cloud, consider these recommendations for optimal performance and security.

Prioritize Hybrid Stream Workers for On-Premise Syslog ​

For syslog senders located within your on-premise environment, deploying Cribl Hybrid Stream Workers is strongly recommended. Place two or more Workers in the same network proximity as your syslog devices, managed by your Cribl.Cloud Leader. This strategy offers significant advantages:

• Enhanced UDP delivery: We recommend local processing due to UDP’s “best-effort” nature and lack of encryption. Hybrid Workers improve delivery reliability and prevent sensitive data from traversing the internet unencrypted.
• Robust Security: By keeping unencrypted TCP traffic within your local network (especially from senders lacking TCP/TLS support), you maintain data security. Cribl Stream can then encrypt data in transit between hybrid Workers and Cribl.Cloud.
• Simpler Port Management: Hybrid Workers provide flexible port configuration, enabling dedicated ports per device type for streamlined management and monitoring.
• Improved Monitoring: Dedicated ports enable the Cribl Monitoring dashboards to report volumes for each major type of syslog sender. While this benefit technically applies to Cribl.Cloud deployments as well, the number of available ports is limited. See the next section for additional details on Cribl Cloud port limitations.

If your environment does not allow for the deployment of Hybrid Cribl Stream Workers, the details below will help you optimize delivery to Stream Workers hosted in Cribl Cloud.

Port Considerations in Cribl.Cloud ​

Cribl.Cloud provides a limited selection of pre-configured ports for common use cases, primarily for receiving Syslog data.

In addition to the pre-configured Syslog ports, Cribl.Cloud offers a range of configurable TCP ports from 20000 to 20010. These ports can be assigned to any push-based Source (including but not limited to Syslog) to receive any supported data type.

Pre-configured Syslog Ports ​

Cribl.Cloud provides the following fixed ports specifically for Syslog traffic:

For unencrypted UDP delivery of Syslog traffic, only ports 514 and 9514 are available.

Port Flexibility with Hybrid Stream Workers ​

For maximum port flexibility, Hybrid Stream Workers support any port, offering a significant advantage over the standard, more restricted port options available directly in Cribl-managed Workers.

Important Security Considerations ​

Avoid sending unencrypted traffic over the Internet. When using Cribl.Cloud, using syslog with TLS is strongly recommended. Unencrypted options accommodate senders lacking TLS support. For such senders, consider deploying Hybrid Stream Workers on the same LAN to keep unencrypted traffic on private networks.

Pipeline Planning ​

In Cribl Stream, we speak of pre-processing Pipelines, processing Pipelines (or just plain Pipelines), and post-processing Pipelines. Cribl recommends combining the first two in an arrangement that we’ve found to be optimal for syslog.

Pre-Processing Syslog Sources ​

Pre-processing Pipelines are useful for standardizing or normalizing incoming data from multiple Syslog Sources. Pre-processing can ensure consistency, streamline ingest-time processing for syslog events, and reduce the need to replicate the functionality and logic in each processing Pipeline that is associated with a Syslog Source.

However, pre-processing may not be necessary or desirable in every case. For example, if you have unique processing requirements for different subsets of data, it may be more effective to configure separate processing Pipelines according to your needs.

The Cribl Pack for Syslog Input provides a pre-processing Pipeline that modifies the data that Cribl Stream delivers. Before you deploy the pre-processing Pipeline, review the Pipeline’s comments as described in Importing the Pre-Processing Pipeline to understand the available functions and options and enable or disable them as desired.

If your syslog senders produce events with embedded newline or carriage-return characters (multi-line or otherwise malformed syslog), treat them as malformed syslog rather than standard single-line events. In those cases, consider ingesting the data via a Raw TCP or Raw UDP Source with a custom Event Breaker, as described in Alternative Sources for Malformed Syslog .

Pipeline for Selected Syslog Data Subsets ​

Configure dedicated Pipelines (and Routes!) for each distinct subset of data that arrives via syslog senders. These subsets might include DHCP logs from the router, traffic logs from the firewall, operational logs from load balancers, and virtualization logs from on-prem virtualization hypervisors.

Certain kinds of subset-specific processing have become Cribl Stream best practices. These include:

• For ESXi or other hypervisor logs, use the Drop Function to drop debug-level logs.
• For firewall logs, enrich with DNS names, GeoIP fields, and lookups from a threat list.
• For load balancer logs, use the Suppress Function to reduce volume.

Unless you’re new to Cribl Stream, you’ve already created your own Pipelines, so we’re not going to review that here. (If you are new to Cribl Stream, consider running through the free Cribl Stream Overview sandbox course ASAP.)

Importing the Pre-Processing Pipeline (Optional) ​

Even before setting up a Cribl Stream Syslog Source, you have the option to install the Cribl Pack for Syslog Input (a.k.a. cribl-syslog-input Pack), which provides the pre-processing Pipeline for mixed-use Syslog Cribl Sources. This is particularly helpful when you have various types of syslog devices sending data to a single port, such as port 514. When you configure dedicated ports for each type of device, it’s generally best to use a Pipeline or Pack that is specifically tailored to that type of data.

If this is your first time installing from the Cribl Dispensary, see the full directions with screenshots. Otherwise:

1. Open the Cribl Pack Dispenary’s Cribl Pack for Syslog Input page.
2. Under Releases at right, click the link for the latest release.
3. In the resulting Assets section, right-click the .crbl file to download it locally.
4. In Cribl Stream, select a Worker Group, then select Processing > Packs.
5. Click the Add New button, and select Import from file.
6. Select the Pack you downloaded, and follow the prompts from there. In most cases, when installing a Dispensary Pack, you should not enter an override value for New Pack ID.
7. Review the Pack’s README file, available in the Pack’s Settings link and also online.

Let’s examine what this Syslog Input Pack provides, starting with the Routes page.