Supported Types

msgspec uses Python type annotations to describe the expected types. Most combinations of the following types are supported (with a few restrictions):

Builtin Types

Msgspec types

Standard Library Types

Typing module types

Abstract types

Third-Party Libraries

Additional types may be supported through extensions.

Note that except where explicitly stated, subclasses of these types are not supported by default (see Extending for how to add support yourself).

Here we document how msgspec maps Python objects to/from the various supported protocols.

None

None maps to the null value in all supported protocols. Note that TOML lacks a null value, attempting to encode a message containing None to TOML will result in an error.

>>> msgspec.json.encode(None)
b'null'

>>> msgspec.json.decode(b'null')
None

If strict=False is specified, a string value of "null" (case insensitive) may also be coerced to None. See “Strict” vs “Lax” Mode for more information.

>>> msgspec.json.decode(b'"null"', type=None, strict=False)
None

bool

Booleans map to their corresponding true / false values in both all supported protocols.

>>> msgspec.json.encode(True)
b'true'

>>> msgspec.json.decode(b'true')
True

If strict=False is specified, values of "true" / "1" / 1 or "false" / "0" / 0 (case insensitive for strings) may also be coerced to True / False respectively. See “Strict” vs “Lax” Mode for more information.

>>> msgspec.json.decode(b'"false"', type=bool, strict=False)
False

>>> msgspec.json.decode(b'"TRUE"', type=bool, strict=False)
True

>>> msgspec.json.decode(b'1', type=bool, strict=False)
True

int

Integers map to integers in all supported protocols.

Support for large integers varies by protocol:

  • msgpack only supports encoding/decoding integers within [-2**63, 2**64 - 1], inclusive.

  • json, yaml, and toml have no restrictions on encode or decode.

>>> msgspec.json.encode(123)
b"123"

>>> msgspec.json.decode(b"123", type=int)
123

If strict=False is specified, string values may also be coerced to integers, following the same restrictions as above. Likewise floats that have an exact integer representation (i.e. no decimal component) may also be coerced as integers. See “Strict” vs “Lax” Mode for more information.

>>> msgspec.json.decode(b'"123"', type=int, strict=False)
123

>>> msgspec.json.decode(b'123.0', type=int, strict=False)
123

float

Floats map to floats in all supported protocols. Note that per RFC8259, JSON doesn’t support nonfinite numbers (nan, infinity, -infinity); msgspec.json handles this by encoding these values as null. The msgpack, toml, and yaml protocols lack this restriction, and can accurately roundtrip any IEEE754 64 bit floating point value.

For all protocols, if a float type is specified and an int value is provided, the int will be automatically converted.

>>> msgspec.json.encode(123.0)
b"123.0"

>>> # JSON doesn't support nonfinite values, these serialize as null
... msgspec.json.encode(float("nan"))
b"null"

>>> msgspec.json.decode(b"123.0", type=float)
123.0

>>> # Ints are automatically converted to floats
... msgspec.json.decode(b"123", type=float)
123.0

If strict=False is specified, string values may also be coerced to floats. Note that in this case the strings "nan", "inf" / "infinity", "-inf" / "-infinity" (case insensitive) will coerce to nan / inf / -inf. See “Strict” vs “Lax” Mode for more information.

>>> msgspec.json.decode(b'"123.45"', type=float, strict=False)
123.45

>>> msgspec.json.decode(b'"-inf"', type=float, strict=False)
-inf

str

Strings map to strings in all supported protocols.

Note that for JSON, only the characters required by RFC8259 are escaped to ascii; unicode characters (e.g. "𝄞") are not escaped and are serialized directly as UTF-8 bytes.

>>> msgspec.json.encode("Hello, world!")
b'"Hello, world!"'

>>> msgspec.json.encode("𝄞 is not escaped")
b'"\xf0\x9d\x84\x9e is not escaped"'

>>> msgspec.json.decode(b'"Hello, world!"')
"Hello, world!"

bytes / bytearray / memoryview

Bytes-like objects map to base64-encoded strings in JSON, YAML, and TOML. The bin type is used for MessagePack.

>>> msg = msgspec.json.encode(b"\xf0\x9d\x84\x9e")

>>> msg
b'"85+Eng=="'

>>> msgspec.json.decode(msg, type=bytes)
b'\xf0\x9d\x84\x9e'

>>> msgspec.json.decode(msg, type=bytearray)
bytearray(b'\xf0\x9d\x84\x9e')

Note

For the msgpack protocol, memoryview objects will be decoded as direct views into the larger buffer containing the input message being decoded. This may be useful for implementing efficient zero-copy handling of large binary messages, but is also a potential footgun. As long as a decoded memoryview remains in memory, the input message buffer will also be persisted, potentially resulting in unnecessarily large memory usage. The usage of memoryview types in this manner is considered an advanced topic, and should only be used when you know their usage will result in a performance benefit.

For all other protocols memoryview objects will still result in a copy, and will likely be slightly slower than decoding into a bytes object

datetime

The encoding used for datetime.datetime objects depends on both the protocol and whether these objects are timezone-aware or timezone-naive:

  • JSON: Timezone-aware datetimes are encoded as RFC3339 compatible strings. Timezone-naive datetimes are encoded the same, but lack the timezone component (making them not strictly RFC3339 compatible, but still ISO8601 compatible).

  • MessagePack: Timezone-aware datetimes are encoded using the timestamp extension. Timezone-naive datetimes are encoded the same, but lack the timezone component (making them not strictly RFC3339 compatible, but still ISO8601 compatible). During decoding, both string and timestamp-extension values are supported for flexibility.

  • YAML: Datetimes are encoded using YAML’s native datetime type. Both timezone-aware and timezone-naive datetimes are supported.

  • TOML: Datetimes are encoded using TOML’s native datetime type. Both timezone-aware and timezone-naive datetimes are supported.

Note that you can require a datetime.datetime object to be timezone-aware or timezone-naive by specifying a tz constraint (see Datetime Constraints for more information).

>>> import datetime

>>> tz = datetime.timezone(datetime.timedelta(hours=6))

>>> tz_aware = datetime.datetime(2021, 4, 2, 18, 18, 10, 123, tzinfo=tz)

>>> msg = msgspec.json.encode(tz_aware)

>>> msg
b'"2021-04-02T18:18:10.000123+06:00"'

>>> msgspec.json.decode(msg, type=datetime.datetime)
datetime.datetime(2021, 4, 2, 18, 18, 10, 123, tzinfo=datetime.timezone(datetime.timedelta(seconds=21600)))

>>> tz_naive = datetime.datetime(2021, 4, 2, 18, 18, 10, 123)

>>> msg = msgspec.json.encode(tz_naive)

>>> msg
b'"2021-04-02T18:18:10.000123"'

>>> msgspec.json.decode(msg, type=datetime.datetime)
datetime.datetime(2021, 4, 2, 18, 18, 10, 123)

>>> msgspec.json.decode(b'"oops"', type=datetime.datetime)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
msgspec.ValidationError: Invalid RFC3339 encoded datetime

Additionally, if strict=False is specified, all protocols will decode ints, floats, or strings containing ints/floats as timezone-aware datetimes, interpreting the value as seconds since the epoch in UTC (a Unix Timestamp). See “Strict” vs “Lax” Mode for more information.

>>> msgspec.json.decode(b"1617405490.000123", type=datetime.datetime)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
msgspec.ValidationError: Expected `datetime`, got `float`

>>> msgspec.json.decode(b"1617405490.000123", type=datetime.datetime, strict=False)
datetime.datetime(2021, 4, 2, 18, 18, 10, 123, tzinfo=datetime.timezone.utc)

date

datetime.date values map to:

  • JSON: RFC3339 encoded strings

  • MessagePack: RFC3339 encoded strings

  • YAML: YAML’s native date type

  • TOML TOML’s native date type

>>> import datetime

>>> date = datetime.date(2021, 4, 2)

>>> msg = msgspec.json.encode(date)

>>> msg
b'"2021-04-02"'

>>> msgspec.json.decode(msg, type=datetime.date)
datetime.date(2021, 4, 2)

>>> msgspec.json.decode(b'"oops"', type=datetime.date)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
msgspec.ValidationError: Invalid RFC3339 encoded date

time

The encoding used for datetime.time objects is dependent on both the protocol and whether these objects are timezone-aware or timezone-naive:

  • JSON, MessagePack, and YAML: Timezone-aware times are encoded as RFC3339 compatible strings. Timezone-naive times are encoded the same, but lack the timezone component (making them not strictly RFC3339 compatible, but still ISO8601 compatible).

  • TOML: Timezone-naive times are encoded using TOML’s native time type. Timezone-aware times aren’t supported.

Note that you can require a datetime.time object to be timezone-aware or timezone-naive by specifying a tz constraint (see Datetime Constraints for more information).

>>> import datetime

>>> tz = datetime.timezone(datetime.timedelta(hours=6))

>>> tz_aware = datetime.time(18, 18, 10, 123, tzinfo=tz)

>>> msg = msgspec.json.encode(tz_aware)

>>> msg
b'"18:18:10.000123+06:00"'

>>> msgspec.json.decode(msg, type=datetime.time)
datetime.time(18, 18, 10, 123, tzinfo=datetime.timezone(datetime.timedelta(seconds=21600)))

>>> tz_naive = datetime.time(18, 18, 10, 123)

>>> msg = msgspec.json.encode(tz_naive)

>>> msg
b'"18:18:10.000123"'

>>> msgspec.json.decode(msg, type=datetime.time)
datetime.time(18, 18, 10, 123)

>>> msgspec.json.decode(b'"oops"', type=datetime.time)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
msgspec.ValidationError: Invalid RFC3339 encoded time

timedelta

datetime.timedelta values map to extended ISO 8601 duration strings in all protocols.

The format as described in the ISO specification is fairly lax and a bit underspecified, leading most real-world implementations to implement a stricter subset.

The duration format used here is as follows:

[+/-]P[#D][T[#H][#M][#S]]
  • The format starts with an optional sign (- or +). If negative, the whole duration is negated.

  • The letter P follows (case insensitive)

  • There are then four segments, each consisting of a number and unit. The units are D, H, M, S (case insensitive) for days, hours, minutes, and seconds respectively. These segments must occur in this order.

    • If a segment would have a 0 value it may be omitted, with the caveat that at least one segment must be present.

    • If a time (hour, minute, or second) segment is present then the letter T (case insensitive) must precede the first time segment. Likewise if a T is present, there must be at least 1 segment after the T.

    • Each segment is composed of 1 or more digits, followed by the unit. Leading 0s are accepted. The final segment may include a decimal component if needed.

A few examples:

"P0D"                # 0 days
"P1D"                # 1 Day
"PT1H30S"            # 1 Hour and 30 minutes
"PT1.5H"             # 1 Hour and 30 minutes
"-PT1M30S"           # -90 seconds
"PT1H30M25.5S"       # 1 Hour, 30 minutes, and 25.5 seconds

While msgspec will decode duration strings making use of the H (hour) or M (minute) units, durations encoded by msgspec will only consist of D (day) and S (second) segments.

The implementation in msgspec is compatible with the ones in:

  • Java’s time.Duration.parse (docs)

  • Javascript’s proposed Temporal.Duration standard API (docs)

  • Python libraries like pendulum or pydantic.

Duration strings produced by msgspec should be interchangeable with these libraries, as well as similar ones in other language ecosystems.

>>> from datetime import timedelta

>>> msgspec.json.encode(timedelta(seconds=123))
b'"PT123S"'

>>> msgspec.json.encode(timedelta(days=1, seconds=30, microseconds=123))
b'"P1DT30.000123S"'

>>> msgspec.json.decode(b'"PT123S"', type=timedelta)
datetime.timedelta(seconds=123)

>>> msgspec.json.decode(b'"PT1.5M"', type=timedelta)
datetime.timedelta(seconds=90)

>>> msgspec.json.decode(b'"oops"', type=datetime.timedelta)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
msgspec.ValidationError: Invalid ISO8601 duration

Additionally, if strict=False is specified, all protocols will decode ints, floats, or strings containing ints/floats as timedeltas, interpreting the value as total seconds. See “Strict” vs “Lax” Mode for more information.

>>> msgspec.json.decode(b"123.4", type=datetime.timedelta)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
msgspec.ValidationError: Expected `duration`, got `float`

>>> msgspec.json.decode(b"123.4", type=datetime.timedelta, strict=False)
datetime.timedelta(seconds=123, microseconds=400000)

uuid

uuid.UUID values are serialized as RFC4122 encoded canonical strings in all protocols by default. Subclasses of uuid.UUID are also supported for encoding only.

>>> import uuid

>>> u = uuid.UUID("c4524ac0-e81e-4aa8-a595-0aec605a659a")

>>> msgspec.json.encode(u)
b'"c4524ac0-e81e-4aa8-a595-0aec605a659a"'

>>> msgspec.json.decode(b'"c4524ac0-e81e-4aa8-a595-0aec605a659a"', type=uuid.UUID)
UUID('c4524ac0-e81e-4aa8-a595-0aec605a659a')

>>> msgspec.json.decode(b'"oops"', type=uuid.UUID)
Traceback (most recent call last):
    File "<stdin>", line 1, in <module>
msgspec.ValidationError: Invalid UUID

Alternative formats are also supported by the JSON and MessagePack encoders. The format may be selected by passing it to uuid_format when creating an Encoder. The following options are supported:

  • canonical: UUIDs are encoded as RFC4122 canonical strings (same as str(uuid)). This is the default.

  • hex: UUIDs are encoded as RFC4122 hex strings (same as uuid.hex).

  • bytes: UUIDs are encoded as binary values of the uuid’s big-endian 128-bit integer representation (same as uuid.bytes). This is only supported by the MessagePack encoder.

When decoding, any of the above formats are accepted.

>>> enc = msgspec.json.Encoder(uuid_format="hex")

>>> uuid_hex = enc.encode(u)

>>> uuid_hex
b'"c4524ac0e81e4aa8a5950aec605a659a"'

>>> msgspec.json.decode(uuid_hex, type=uuid.UUID)
UUID('c4524ac0-e81e-4aa8-a595-0aec605a659a')

>>> enc = msgspec.msgpack.Encoder(uuid_format="bytes")

>>> uuid_bytes = enc.encode(u)

>>> msgspec.msgpack.decode(uuid_bytes, type=uuid.UUID)
UUID('c4524ac0-e81e-4aa8-a595-0aec605a659a')

decimal

decimal.Decimal values are encoded as their string representation in all protocols by default. This ensures no precision loss during serialization, as would happen with a float representation.

>>> import decimal

>>> x = decimal.Decimal("1.2345")

>>> msg = msgspec.json.encode(x)

>>> msg
b'"1.2345"'

>>> msgspec.json.decode(msg, type=decimal.Decimal)
Decimal('1.2345')

>>> msgspec.json.decode(b'"oops"', type=decimal.Decimal)
Traceback (most recent call last):
    File "<stdin>", line 1, in <module>
msgspec.ValidationError: Invalid decimal string

For JSON and MessagePack you may instead encode decimal values the same as numbers by creating a Encoder and specifying decimal_format='number'.

>>> encoder = msgspec.json.Encoder(decimal_format="number")

>>> encoder.encode(x)
b'1.2345'

Note

It is impossible to represent NaN, sNaN, Inf, and other non-finite decimal values as numbers. ValueError will be raised in this case. Prefer string format if these values are needed.

For JSON and MessagePack you may also pass a callable to decimal_format to customize how decimals are encoded. The callable will be called with the decimal.Decimal instance as the only argument, and must return a JSON-serializable value (but not another decimal.Decimal or a nested structure containing decimal.Decimal). This is useful for custom rounding, formatting, or transforming decimal values before encoding.

>>> import decimal

>>> class Price(msgspec.Struct):