If you are a Rubyist, you’ve likely been writing # frozen_string_literal: true at the top of most of your Ruby source code files, or at the very least, that you’ve seen it in some other projects.

Based on informal discussions at conferences and online, it seems that what this magic comment really is about is not always well understood, so I figured it would be worth talking about why it’s there, what it does exactly, and what its future might look like.

Ruby Strings Are Mutable

Before we can delve into what makes frozen string literals special, we first need to talk about the Ruby String type, because it’s quite different from the equivalent type in other popular languages.

In the overwhelming majority of popular languages, strings are immutable. That’s the case in Java, JavaScript, Python, Go, etc.

There are a few exceptions, though, like Perl, PHP1, C/C++ (except for literals), and of course Ruby:

>> str = String.new
=> ""
>> str.object_id
=> 24952
>> str << "foo"
=> "foo"
>> str
=> "foo"
>> str.capitalize!
=> "Foo"
>> str.upcase!
=> "FOO"
>> str
=> "FOO"
>> str.object_id
=> 24952

Implementation-wise, they’re just an array of bytes, with an associated encoding to know how these bytes should be interpreted:

class String
  attr_reader :encoding

  def initialize
    @bytes = []
    @encoding = Encoding::UTF_8
  end
end

That too is quite unusual.

String Encoding

Most languages, especially the ones I listed above, instead have chosen a specific internal encoding, and all strings are encoded that way. For instance, in Java and JavaScript, strings are encoded in UTF-16 because they were created somewhat at the same time as the first Unicode specification, and at that time, many people thought that surely 16 bits should be enough to encode all possible characters, but that later turned out to be wrong. Most newer languages uses UTF-8, or a limited set of internal encodings.

For instance, in Python, strings can be encoded in either ISO-8859-1 (AKA Latin 1), UTF-16 or UtF-32. But from a user perspective, it’s an implementation detail, and you can’t really tell what encoding a particular string is using. Semantically, strings are Unicode sequences, how that sequence is encoded in memory is abstracted away.

In these languages, whenever you have to handle text in another encoding, you start by re-encoding it into the internal representation. In Ruby however, strings with different internal encodings can exist in the same program, and Ruby supports over a hundred different encodings:

>> Encoding.list.size
=> 103

While I’m not 100% percent certain of why Ruby went that way, I highly suspect it is in big part due to Ruby’s Japanese origin. In the early days of the Unicode specification, there was an attempt at unifying some of the “common” Chinese, Korean, and Japanese characters, as what is now called the Han unification. Because of that character unification attempt, Unicode had lots of problems for Japanese text, hence the Japanese IT industry didn’t adopt Unicode as fast as the Western IT industry did, and for a very long time, Japanese-specific encoding such as Shift JIS remained widespread.

As such, being able to work with Japanese text without going through a forced Unicode conversion was an important feature for a large part of Ruby’s core contributors.

But let’s go back to mutability.

Pros And Cons

Like most things in engineering, both immutable and mutable strings have pros and cons, so it’s not like one choice is inherently superior to the other.

One of the advantages of immutable strings is that you can more easily share them, for instance:

sliced_string = very_long_string[1..-1]

In the above case, if strings are mutable, you need to copy all but one of the bytes of very_long_string into sliced_string, which can be costly. But if strings are immutable, you can instead have sliced_string internally be pointing at the content of very_long_string with just an offset. That is what some languages call String Views, or String slices.

Another advantage of immutable strings is that they allow for interning. The idea is simple, if strings can’t be mutated, whenever you have multiple instances of strings with identical content, you can coalesce them into a single instance. This deduplication can be done more or less aggressively, as it’s always a tradeoff in how much CPU time you want to spend searching for duplicates in the hope of saving some memory.

Some other advantages include not having to worry about mutation in multi-threaded code, as well as dictionary keys. Strings are used a lot as dictionary keys. If you mutate a string, you change its hash code, and that basically breaks hash tables.

On the other hand, mutable strings are very handy in some scenarios, like to iteratively build a final string:

buffer = ""
10.times do
  buffer << "hello"
end

Whereas in a language with immutable strings like Java, concatenating strings in a loop is known as a classic performance gotcha:

String buffer = "";
for (int i = 0; i < 10; i++) {
  buffer += "hello";
}

In the above example, on every loop, the += operator causes a new string to be allocated, and the content to be copied, which gets exponentially more expensive as the string grows. Instead, you are supposed to use a different object as a buffer: StringBuilder:

StringBuilder buffer = new StringBuilder();
for (int i = 0; i < 10; i++) {
  buffer.append("hello");
}
buffer.toString();

That’s the Java equivalent of appending strings to an array and then calling array.join(""). It’s a common enough mistake that at some point the Java compiler gained the ability to detect that pattern and automatically replace it with the equivalent code using StringBuilder.

While having to use a different buffer type isn’t the end of the world, I do very much like that it’s not necessary in Ruby.

But more generally, the advantage of mutable strings is that for some algorithms, being able to modify the string in place saves a lot of memory allocations and copying.

Ruby Actually Has Both

Earlier in this post, I said Ruby had mutable strings, but it’s not quite true. Ruby actually has both mutable and immutable strings, because in Ruby, every mutable object can be frozen, hence, Ruby has both mutable and immutable strings, and it takes advantage of this.

A fun way to poke at Ruby internals is through the ObjectSpace.dump method.

require "json"
require "objspace"

def dump(obj)
  JSON.pretty_generate(JSON.parse(ObjectSpace.dump(obj)))
end

str = "Hello World" * 80
puts dump(str)

The above script will output something like:

{
  "address": "0x105068e10",
  "type": "STRING",
  "slot_size": 40,
  "bytesize": 880,
  "memsize": 921,
  ...
}

It tells us the string content is 880B (bytesize) and that Ruby allocated a 40B wide slot (slot_size), hence the string content is stored in an external buffer for a total of 921B (memsize).

Now, look what happens if we slice that string:

require "json"
require "objspace"

def dump(obj)
  JSON.pretty_generate(JSON.parse(ObjectSpace.dump(obj)))
end

str = "Hello World" * 80
puts "initial str: #{dump(str)}\n"

slice = str[40..-1]

puts "str after:\n#{dump(str)}\n"
puts "slice:\n#{dump(slice)}\n"

str after:
{
  "address": "0x105178e18",
  "type": "STRING",
  "slot_size": 40,
  "shared": true,
  "references": [ "0x1051786c0" ],
  "memsize": 40,
  ...
}

slice:
{
  "address": "0x1051786e8",
  "type": "STRING",
  "slot_size": 40,
  "shared": true,
  "references": [ "0x1051786c0" ],
  "memsize"