OpenCensus

OpenCensus is being archived! Read the blog post to learn more

Navigation :

Requirements
Installation
Brief Overview
Getting started
Create and Record Metrics
Enable Views
Exporting to Prometheus

In this quickstart, we’ll glean insights from code segments and learn how to:

Collect metrics using OpenCensus Metrics and Tags
Register and enable an exporter for a backend of our choice
View the metrics on the backend of our choice

Requirements

Python2 and above
Prometheus as our choice of metrics backend: we are picking it because it is free, open source and easy to setup

For assistance setting up Prometheus, Click here for a guided codelab.

You can swap out any other exporter from the list of Python exporters

Installation

pip install --upgrade opencensus opencensus-ext-prometheus prometheus-client

Brief Overview

By the end of this tutorial, we will do these four things to obtain metrics using OpenCensus:

Create quantifiable metrics (numerical) that we will record
Create tags that we will associate with our metrics
Organize our metrics, similar to writing a report, in to a View
Export our views to a backend (Prometheus in this case)

Getting Started

Unsure how to write and execute Python code? Click here.

We will be a simple “read-evaluate-print” (REPL) app. In there we’ll collect some metrics to observe the work that is going on within this code, such as:

Latency per processing loop
Number of lines read
Number of errors
Line lengths

First, create a file called repl.py.

touch repl.py

Next, put the following code inside of repl.py:

#!/usr/bin/env python

import sys

def main():
    # In a REPL:
    #1. Read input
    #2. process input
    while True:
        line = sys.stdin.readline()
        print(line.upper())

if __name__ == '__main__':
      main()

You can run the code via python repl.py.

Create and record Metrics

Now we’ll import the required packages and instrument our code.

Snippet
All

from opencensus.stats import aggregation as aggregation_module
from opencensus.stats import measure as measure_module
from opencensus.stats import stats as stats_module
from opencensus.stats import view as view_module
from opencensus.tags import tag_key as tag_key_module
from opencensus.tags import tag_map as tag_map_module
from opencensus.tags import tag_value as tag_value_module

# Create the measures
# The latency in milliseconds
m_latency_ms = measure_module.MeasureFloat("repl_latency", "The latency in milliseconds per REPL loop", "ms")

# Counts/groups the lengths of lines read in.
m_line_lengths = measure_module.MeasureInt("repl_line_lengths", "The distribution of line lengths", "By")

#!/usr/bin/env python

import sys
import time

from opencensus.stats import aggregation as aggregation_module
from opencensus.stats import measure as measure_module
from opencensus.stats import stats as stats_module
from opencensus.tags import tag_key as tag_key_module
from opencensus.tags import tag_map as tag_map_module
from opencensus.tags import tag_value as tag_value_module

# Create the measures
# The latency in milliseconds
m_latency_ms = measure_module.MeasureFloat("repl/latency", "The latency in milliseconds per REPL loop", "ms")

# Counts/groups the lengths of lines read in.
m_line_lengths = measure_module.MeasureInt("repl_line_lengths", "The distribution of line lengths", "By")

# The stats recorder
stats_recorder = stats_module.stats.stats_recorder

# Create the tag key
key_method = tag_key_module.TagKey("method")
# Create the status key
key_status = tag_key_module.TagKey("status")
# Create the error key
key_error = tag_key_module.TagKey("error")

def main():
    # In a REPL:
    # 1. Read input
    # 2. process input
    while True:
        readEvaluateProcessLine()

def readEvaluateProcessLine():
    line = sys.stdin.readline()
    start = time.time()
    print(line.upper())

    # Now record the stats
    # Create the measure_map into which we'll insert the measurements
    mmap = stats_recorder.new_measurement_map()
    end_ms = (time.time() - start) * 1000.0 # Seconds to milliseconds

    # Record the latency
    mmap.measure_float_put(m_latency_ms, end_ms)

    # Record the line length
    mmap.measure_int_put(m_line_lengths, len(line))

    tmap = tag_map_module.TagMap()
    tmap.insert(key_method, tag_value_module.TagValue("repl"))
    tmap.insert(key_status, tag_value_module.TagValue("OK"))

    # Insert the tag map finally
    mmap.record(tmap)

if __name__ == "__main__":
    main()

With views and all enabled

In order to analyze these stats, we’ll need to aggregate our data with Views.

#!/usr/bin/env python

import sys
import time

from opencensus.stats import aggregation as aggregation_module
from opencensus.stats import measure as measure_module
from opencensus.stats import stats as stats_module
from opencensus.stats import view as view_module
from opencensus.tags import tag_key as tag_key_module
from opencensus.tags import tag_map as tag_map_module
from opencensus.tags import tag_value as tag_value_module

# Create the measures
# The latency in milliseconds
m_latency_ms = measure_module.MeasureFloat("repl_latency", "The latency in milliseconds per REPL loop", "ms")

# Counts/groups the lengths of lines read in.
m_line_lengths = measure_module.MeasureInt("repl_line_lengths", "The distribution of line lengths", "By")

# The stats recorder
stats_recorder = stats_module.stats.stats_recorder

# Create the tag key
key_method = tag_key_module.TagKey("method")
# Create the status key
key_status = tag_key_module.TagKey("status")
# Create the error key
key_error = tag_key_module.TagKey("error")

latency_view = view_module.View("demo_latency", "The distribution of the latencies",
    [key_method, key_status, key_error],
    m_latency_ms,
    # Latency in buckets:
    # [>=0ms, >=25ms, >=50ms, >=75ms, >=100ms, >=200ms, >=400ms, >=600ms, >=800ms, >=1s, >=2s, >=4s, >=6s]
    aggregation_module.DistributionAggregation([0, 25, 50, 75, 100, 200, 400, 600, 800, 1000, 2000, 4000, 6000]))