Structs

Structs are the preferred way of defining structured data types in msgspec. They’re written in C and are quite speedy and lightweight (measurably faster to create/compare/encode/decode than similar options like dataclasses, attrs, or pydantic). They’re great for representing structured data both for serialization and for use in an application.

Structs are defined by subclassing from msgspec.Struct and annotating the types of individual fields. Default values can also be provided for any optional arguments. Here we define a struct representing a user, with one required field and two optional fields.

>>> import msgspec

>>> class User(msgspec.Struct):
...     """A struct describing a user"""
...     name : str
...     email : str | None = None
...     groups : set[str] = set()
  • name is a required field expecting a str

  • email is an optional field expecting a str or None, defaulting to None if no value is provided.

  • groups is an optional field expecting a set of str. If no value is provided, it defaults to the empty set.

Struct types automatically generate a few methods based on the provided type annotations:

  • __init__

  • __repr__

  • __copy__

  • __replace__

  • __eq__ & __ne__

  • __match_args__ (for Python 3.10+’s pattern matching)

  • __rich_repr__ (for pretty printing support with rich)

>>> alice = User("alice", groups={"admin", "engineering"})

>>> alice
User(name='alice', email=None, groups={'admin', 'engineering'})

>>> bob = User("bob", email="[email protected]")

>>> bob
User(name='bob', email='[email protected]', groups=set())

>>> alice.name
"alice"

>>> bob.groups
set()

>>> alice == bob
False

>>> alice == User("alice", groups={"admin", "engineering"})
True

Note that it is forbidden to override __init__ / __new__ in a struct definition, but other methods can be overridden or added as needed. If you need to customize the generated __init__, see Post-Init Processing.

The struct fields are available via the __struct_fields__ attribute (a tuple of the fields in argument order ) if you need them. Here we add a method for converting a struct to a dict.

>>> class Point(msgspec.Struct):
...     """A point in 2D space"""
...     x : float
...     y : float
...
...     def to_dict(self):
...         return {f: getattr(self, f) for f in self.__struct_fields__}
...

>>> p = Point(1.0, 2.0)

>>> p.to_dict()
{"x": 1.0, "y": 2.0}

Default Values

Struct fields may be given default values, which are used if no value is provided to __init__, or when decoding a message. Default values are configured as part of a Struct definition by assigning them after a field’s type annotation.

>>> from msgspec import Struct, field

>>> import uuid

>>> class Example(Struct):
...     a: int = 1
...     b: uuid.UUID = field(default_factory=uuid.uuid4)
...     c: list[int] = []

>>> Example()
Example(a=1, b=UUID('f63219d5-e9ca-4ae8-afd0-cba30e84222d'), c=[])

>>> Example(a=2)
Example(a=2, b=UUID('319a6c0f-2841-4439-8bc8-2c1daf7d77a2'), c=[])

>>> Example().c is Example().c  # new list instance used each time
False

Default values may be one of 3 kinds:

  • A “static” default value. Here the same default value is used for all instances. These are specified by assigning the default value itself as part of the field definition (as in a above). Most default values will be of this variety.

  • A “dynamic” default value. Here a new default value is used for every instance. These are specified by passing a 0-argument callable to the default_factory argument of msgspec.field (as in b above). This function will be called as needed to create a new default value per instance. These are mainly useful for occasions where you need dynamic defaults, or when a default value is a mutable object that you don’t want to share between all instances of the struct (a common gotcha in Python). Note that since the default_factory callables take no arguments, you might need to make use of a lambda or functools.partial to forward any additional parameters needed to the default factory.

  • Builtin empty mutable collections ([], {}, set(), and bytearray()) may be used as default values (as in c above). Since defaults of these types are so common, these are “syntactic sugar” for specifying the corresponding default_factory (to avoid accidental sharing of mutable values). A default of [] is identical to a default of field(default_factory=list), with a new list instance used each time. Specifying a non-empty mutable collection (e.g. [1, 2, 3]) as a default value will cause the struct definition to error (you should manually define a default_factory in this case).

Post-Init Processing

If a struct type defines a __post_init__(self) method, this will be called at the end of the generated __init__ method. It has the same semantics as the dataclasses method of the same name. This method may be useful for adding additional logic to the init (such as custom validation).

In addition to in __init__, the __post_init__ hook is also called when:

  • Decoding into a struct type (e.g. msgspec.json.decode(..., type=MyStruct))

  • Converting into a struct type (e.g. msgspec.convert(..., type=MyStruct))

In these cases any TypeError or ValueError exceptions raised by this method will be considered “user facing” and converted into a msgspec.ValidationError with additional context. All other exceptions will be raised directly.

>>> import msgspec

>>> class Interval(msgspec.Struct):
...     low: float
...     high: float
...
...     def __post_init__(self):
...         if self.low > self.high:
...             raise ValueError("`low` may not be greater than `high`")

>>> Interval(1, 2)  # valid interval
Interval(low=1, high=2)

>>> Interval(2, 1)  # invalid interval
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 6, in __post_init__
ValueError: `low` may not be greater than `high`

>>> msgspec.json.decode(b'{"low": 2, "high": 1}', type=Interval)  # invalid interval from JSON
Traceback (most recent call last):
  File "<stdin>", line 6, in __post_init__
ValueError: `low` may not be greater than `high`

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
msgspec.ValidationError: `low` may not be greater than `high`

Field Ordering

When defining a new struct type, fields are stored in the order they’re defined (keyword-only fields excluded, more on this later). This is nice for readability since the generated __init__ matches the field order.

class Example(msgspec.Struct):
    a: str
    b: int = 0

The generated __init__() for User looks like:

def __init__(self, a: str, b: int = 0):

One consequence of this is that you can’t put fields without defaults after fields with defaults, since the Python VM doesn’t allow keyword arguments before positional arguments. The following struct definition will error:

>>> class Invalid(msgspec.Struct):
...     a: str = ""
...     b: int  # oop, no default!
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: Required field 'b' cannot follow optional fields. Either reorder
the struct fields, or set `kw_only=True` in the struct definition.

Thankfully the error message includes some solutions:

  • Reorder the struct fields, putting all required fields before all optional fields.

  • Set kw_only=True in the struct definition. This option makes all fields defined on the struct keyword-only parameters.

Keyword-only parameters have no such restriction; required and optional parameters can be mixed in any order.

>>> class Example(msgspec.Struct, kw_only=True):
...     a: str = ""
...     b: int  # this is fine with kw_only=True

>>> Example(a="example", b=123)
Example(a='example', b=123)

Note that the kw_only setting only affects fields defined on that class, not those defined on base or subclasses. This means you can define keyword-only parameters on a base class then add positional parameters in a subclass. All keyword-only parameters are reordered to go after all positional fields.

>>> class Base(msgspec.Struct, kw_only=True):
...     a: str = ""
...     b: int

>>> class Subclass(Base):
...     c: float
...     d: bytes = b""

The generated __init__() for Subclass looks like:

def __init__(self, c: float, d: bytes = b"", *, a: str, b: int = 0):

The field ordering rules for Struct types are identical to those for dataclasses, see the dataclasses docs for more information.

Class Variables

Like dataclasses, msgspec.Struct types will exclude any attribute annotations wrapped in typing.ClassVar from their fields.

>>> import msgspec

>>> from typing import ClassVar

>>> class Example(msgspec.Struct):
...     x: int
...     a_class_variable: ClassVar[int] = 2

>>> Example.a_class_variable
2

>>> Example(1)  # only `x` is counted as a field
Example(x=1)

Note that if using PEP 563 “postponed evaluation of annotations” (e.g. from __future__ import annotations) only the following spellings will work:

  • ClassVar or ClassVar[<type>]

  • typing.ClassVar or typing.ClassVar[<type>]

Importing ClassVar or typing under an aliased name (e.g. import typing as typ or from typing import ClassVar as CV) will not be properly detected.

Type Validation

Unlike some other libraries (e.g. pydantic), the type annotations on a msgspec.Struct class are not checked at runtime during normal use. Types are only checked when decoding a serialized message when using a typed decoder.

>>> import msgspec

>>> class Point(msgspec.Struct):
...     x: float
...     y: float

>>> # Improper types in *your* code aren't checked at runtime
... Point(x=1, y="oops")
Point(x=1, y='oops')

>>> # Improper types when decoding *are* checked at runtime
... msgspec.json.decode(b'{"x": 1.0, "y": "oops"}', type=Point)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
msgspec.ValidationError: Expected `float`, got `str` - at `$.y`

This is intentional. Static type checkers like mypy/pyright work well with msgspec, and can be used to catch bugs without ever running your code. When possible, static tools or unit tests should be preferred over adding expensive runtime checks which slow down every __init__ call.

The input(s) to your programs however cannot be checked statically, as they aren’t known until runtime. As such, msgspec does perform type validation when decoding messages (provided an expected decode type is provided). This validation is fast enough that it is negligible in cost - there is no added performance benefit when not using it. In fact, in most cases it’s faster to decode a message into a type validated msgspec.Struct than into an untyped dict.

Pattern Matching

If using Python 3.10+, msgspec.Struct types can be used in pattern matching blocks. Replicating an example from PEP 636:

# NOTE: this example requires Python 3.10+
>>> import msgspec

>>> class Point(msgspec.Struct):
...     x: float
...     y: float

>>> def where_is(point):
...     match point:
...         case Point(0, 0):
...             print("Origin")
...         case Point(0, y):
...             print(f"Y={y}")
...         case Point(x, 0):
...             print(f"X={x}")
...         case Point():
...             print("Somewhere else")
...         case _:
...             print("Not a point")

>>> where_is(Point(0, 6))
"Y=6"

Equality and Order

By default struct types define an __eq__ method based on the type definition. This enables support for equality comparisons. Additionally, you may configure order=True to make a struct type orderable through generation of __lt__, __le__, __gt__, and __ge__ methods. These methods compare and order instances of a struct type the same as if they were tuples of their field values (in definition order).

>>> class Point(msgspec.Struct, order=True):
...     x: float
...     y: float

>>> Point(1, 2) == Point(1, 2)
True

>>> Point(1, 2) < Point(3, 4)
True

In rare instances you may opt to disable generation of the __eq__ method by configuring eq=False. Equality checks will then fall back to identity comparisons, where the only value a struct instance of that type will compare equal to is itself.

>>> class Point(msgspec.Struct, eq=False):
...     x: float
...     y: float


>>> p = Point(1, 2)

>>> p == Point(1, 2)
False

>>> p == p  # identity comparison only
True

Frozen Instances

A struct type can optionally be marked as “frozen” by specifying frozen=True. This disables modifying attributes after initialization, and adds a __hash__ method to the class definition. Note that for the __hash__ to work, all fields on the struct must also be hashable.

>>> class Point(msgspec.Struct, frozen=True):
...     """This struct is immutable & hashable"""
...     x: float
...     y: float
...

>>> p = Point(1.0, 2.0)

>>> {p: 1}  # frozen structs are hashable, and can be keys in dicts
{Point(1.0, 2.0): 1}

>>> p.x = 2.0  # frozen structs cannot be modified after creation
Traceback (most recent call last):
    ...
AttributeError: immutable type: 'Point'

Tagged Unions

By default a serialized struct only contains information on the values present in the struct instance - no information is serialized noting which struct type corresponds to the message. Instead, the user is expected to know the type the message corresponds to, and pass that information appropriately to the decoder.

>>> import msgspec

>>> class Get