Machine Learning Glossary: TensorFlow

Page Summary

outlined_flag

• This glossary page provides definitions for TensorFlow-related terms.

• Many terms link to the broader machine learning glossary for further information.

• Definitions cover fundamental concepts, TensorFlow APIs, and Google Cloud TPU details.

• Users can understand key aspects of TensorFlow, including graphs, tensors, and execution environments.

• The glossary helps navigate terminology for model training, deployment, and hardware acceleration using TPUs.

This page contains TensorFlow glossary terms. For all glossary terms, click here.

C

Cloud TPU

A specialized hardware accelerator designed to speed up machine learning workloads on Google Cloud.

D

Dataset API (tf.data)

A high-level TensorFlow API for reading data and transforming it into a form that a machine learning algorithm requires. A tf.data.Dataset object represents a sequence of elements, in which each element contains one or more Tensors. A tf.data.Iterator object provides access to the elements of a Dataset.

device

An overloaded term with the following two possible definitions:

1. A category of hardware that can run a TensorFlow session, including CPUs, GPUs, and TPUs.
2. When training an ML model on accelerator chips (GPUs or TPUs), the part of the system that actually manipulates tensors and embeddings. The device runs on accelerator chips. In contrast, the host typically runs on a CPU.

E

eager execution

A TensorFlow programming environment in which operations run immediately. In contrast, operations called in graph execution don't run until they are explicitly evaluated. Eager execution is an imperative interface, much like the code in most programming languages. Eager execution programs are generally far easier to debug than graph execution programs.

Estimator

A deprecated TensorFlow API. Use tf.keras instead of Estimators.

F

feature engineering

A process that involves the following steps:

1. Determining which features might be useful in training a model.
2. Converting raw data from the dataset into efficient versions of those features.

For example, you might determine that temperature might be a useful feature. Then, you might experiment with bucketing to optimize what the model can learn from different temperature ranges.

Feature engineering is sometimes called feature extraction or featurization.

Click the icon for additional notes about TensorFlow.

In TensorFlow, feature engineering often means converting raw log file entries to tf.Example protocol buffers. See also tf.Transform.

---

See Numerical data: How a model ingests data using feature vectors in Machine Learning Crash Course for more information.

feature spec

Describes the information required to extract features data from the tf.Example protocol buffer. Because the tf.Example protocol buffer is just a container for data, you must specify the following:

• The data to extract (that is, the keys for the features)
• The data type (for example, float or int)
• The length (fixed or variable)

G

graph

In TensorFlow, a computation specification. Nodes in the graph represent operations. Edges are directed and represent passing the result of an operation (a Tensor) as an operand to another operation. Use TensorBoard to visualize a graph.

graph execution

A TensorFlow programming environment in which the program first constructs a graph and then executes all or part of that graph. Graph execution is the default execution mode in TensorFlow 1.x.

Contrast with eager execution.

H

host

When training an ML model on accelerator chips (GPUs or TPUs), the part of the system that controls both of the following:

• The overall flow of the code.
• The extraction and transformation of the input pipeline.

The host typically runs on a CPU, not on an accelerator chip; the device manipulates tensors on the accelerator chips.

L

Layers API (tf.layers)

A TensorFlow API for constructing a deep neural network as a composition of layers. The Layers API lets you build different types of layers, such as:

• tf.layers.Dense for a fully-connected layer.
• tf.layers.Conv2D for a convolutional layer.

The Layers API follows the Keras layers API conventions. That is, aside from a different prefix, all functions in the Layers API have the same names and signatures as their counterparts in the Keras layers API.

M

mesh

In ML parallel programming, a term associated with assigning the data and model to TPU chips, and defining how these values will be sharded or replicated.

Mesh is an overloaded term that can mean either of the following:

• A physical layout of TPU chips.
• An abstract logical construct for mapping the data and model to the TPU chips.

In either case, a mesh is specified as a shape.

metric

A statistic that you care about.

An objective is a metric that a machine learning system tries to optimize.

N

node (TensorFlow graph)

An operation in a TensorFlow graph.

O