Feature #20205
open
Enable `frozen_string_literal` by default
Description
Context¶
The frozen_string_literal: true pragma was introduced in Ruby 2.3, and as far as I'm aware the plan was initially to make it the default for Ruby 3.0, but this plan was abandoned because it would be too much of a breaking change without any real further notice.
According to Matz, he still wishes to enable frozen_string_literal by default in the future, but a reasonable migration plan is required.
The main issue is backward compatibility, flipping the switch immediately would break a lot of code, so there must be some deprecation period.
The usual the path forward for this kind of change is to emit deprecation warnings one of multiple versions in advance.
One example of that was the Ruby 2.7 keyword argument deprecation. It was quite verbose, and some users were initially annoyed, but I think the community pulled through it and I don't seem to hear much about it anymore.
So for frozen string literals, the first step would be to start warning when a string that would be frozen in the future is mutated.
Deprecation Warning Implementation¶
I implemented a quick proof of concept with @etienne (Étienne Barrié) in https://github.com/Shopify/ruby/pull/549
In short:
- Files with
# frozen_string_literal: trueor# frozen_string_literal: falsedon't change in behavior at all. - Files with no
# frozen_string_literalcomment are compiled to useputchilledstringopcode instead of regularputstring. - This opcode mark the string with a user flag, when these strings are mutated, a warning is issued.
Currently the proof of concept issue the warning at the mutation location, which in some case can make locating where the string was allocated a bit hard.
But it is possible to improve it so the message also include the location at which the literal string was allocated, and learning from the keyword argument warning experience,
we can record which warnings were already issued to avoid spamming users with duplicated warnings.
As currently implemented, there is almost no overhead. If we modify the implementation to record the literal location,
we'd incur a small memory overhead for each literal string in a file without an explicit frozen_string_literal pragma.
But I believe we could do it in a way that has no overhead if Warning[:deprecated] = false.
Timeline¶
The migration would happen in 3 steps, each step can potentially last multiple releases. e.g. R0 could be 3.4, R1 be 3.7 and R2 be 4.0.
I don't have a strong opinion on the pace.
- Release
R0: introduce the deprecation warning (only if deprecation warnings enabled). - Release
R1: make the deprecation warning show up regardless of verbosity level. - Release
R2: make string literals frozen by default.
Impact¶
Given that rubocop is quite popular in the community and it has enforced the usage of # frozen_string_literal: true for years now,
I suspect a large part of the actively maintained codebases in the wild wouldn't see any warnings.
And with recent versions of minitest enabling deprecation warnings by default (and potentially RSpec too),
the few that didn't migrate will likely be made compatible quickly.
The real problem of course are the less actively developed libraries and applications. For such cases, any codebase can remain compatible by setting RUBYOPT="--disable=frozen_string_literal",
and so even after R2 release. The flag would never be removed any legacy codebase can continue upgrading Ruby without changing a single line of cod by just flipping this flag.
Workflow for library maintainers¶
As a library maintainer, fixing the deprecation warnings can be as simple as prepending # frozen_string_literal: false at the top of all their source files, and this will keep working forever.
Alternatively they can of course make their code compatible with frozen string literals.
Code that is frozen string literal compatible doesn't need to explicitly declare it. Only code that need it turned of need to do so.
Workflow for application owners¶
For application owners, the workflow is the same than for libraries.
However if they depend on a gem that hasn't updated, or that they can't upgrade it, they can run their application with RUBYOPT="--disable=frozen_string_literal" and it will keep working forever.
Any user running into an incompatibility issue can set RUBYOPT="--disable=frozen_string_literal" forever, even in 4.x, the only thing changing is the default value.
And any application for which all dependencies have been made fully frozen string literal compatible can set RUBYOPT="--enable=frozen_string_literal" and start immediately removing magic comment from their codebase.
Updated by matheusrich (Matheus Richard) over 2 years ago
Given that rubocop is quite popular in the community and it has enforced the usage of # frozen_string_literal: true for years now,
I suspect a large part of the actively maintained codebases in the wild wouldn't see any warnings.
That's true, but because standardrb doesn't enforce it (and many folks have been defaulting to that), I've seen several codebases not consistently using the pragma.
Updated by mame (Yusuke Endoh) over 2 years ago
I think we should evaluate the value of # frozen_string_literal: true before making it the default.
The purpose of # frozen_string_literal: true is to make Ruby code fast and memory-saving. When it was introduced, no quantitative evaluation was available except for micro-benchmarks, because most code did not support frozen_string_literal.
Now that there are many gems supporting # frozen_string_literal: true. So we can evaluate it. For example, how much would the performance degrade if we removed # frozen_string_literal: true from all code used in yjit-bench?
If the degradation is large enough, then making # frozen_string_literal: true the default may be worth serious consideration. If the degradation is negligible, it would be reasonable for Rubocop to stop enforcing # frozen_string_literal: true and remove the magic comments from existing gems as well.
Updated by zverok (Victor Shepelev) over 2 years ago
The purpose of
# frozen_string_literal: trueis to make Ruby code fast and memory-saving. When it was introduced, no quantitative evaluation was available except for micro-benchmarks, because most code did not support frozen_string_literal.
I believe that whatever the initial intention, the "frozen string literals" concept being adopted by many codebases is also a cultural thing, not only performance-related.
The "string literals are frozen by default" changes the way we program, if even slightly. That’s actually a long-standing topic to have more impactful freezing (of the constants, for example), but frozen string literals at the very least prevent trivial errors like
Updated by byroot (Jean Boussier) over 2 years ago
I think we should evaluate the value of # frozen_string_literal: true before making it the default.
So it's not 100% reliable because I ran it locally rather than on a benchmarking server, as evidenced by some strange effect on benchmarks that don't normally deal with strings (e.g. setivar_object), but here are the results:
mutable: ruby 3.4.0dev (2024-01-24T08:24:16Z disable-frozen-str.. a39d5eae1e) [arm64-darwin23]
frozen: ruby 3.4.0dev (2024-01-24T08:24:16Z disable-frozen-str.. bb0cee8dab) [arm64-darwin23]
-------------- ------------ ---------- ----------- ---------- -------------- --------------
bench mutable (ms) stddev (%) frozen (ms) stddev (%) frozen 1st itr mutable/frozen
activerecord 32.9 4.0 32.7 5.3 1.06 1.01
chunky-png 578.0 1.1 555.2 1.0 1.01 1.04
erubi-rails 1124.2 2.4 1158.7 1.7 0.93 0.97
hexapdf 1703.9 2.7 1670.2 2.2 1.04 1.02
liquid-c 34.3 5.7 34.1 5.0 1.02 1.01
liquid-compile 39.8 4.2 38.0 5.3 0.99 1.05
liquid-render 97.2 3.4 97.7 3.5 1.01 0.99
lobsters 637.8 5.2 614.9 4.2 1.11 1.04
mail 82.7 3.7 82.1 3.7 1.00 1.01
psych-load 1500.8 0.8 1487.6 0.5 0.99 1.01
railsbench 1116.3 2.1 1099.8 1.6 1.02 1.01
rubocop 112.0 3.8 111.2 3.7 1.03 1.01
ruby-lsp 79.5 2.6 80.3 2.4 0.92 0.99
sequel 36.4 4.4 36.5 3.3 0.95 1.00
binarytrees 238.4 1.9 238.8 1.7 0.99 1.00
blurhash 281.9 1.4 282.5 1.7 0.99 1.00
erubi 166.1 2.0 171.5 2.5 0.97 0.97
etanni 198.7 2.7 200.5 1.8 1.04 0.99
fannkuchredux 2065.3 0.7 2078.5 0.5 0.99 0.99
fluentd 1192.0 1.0 1202.7 0.7 1.03 0.99
graphql 2323.6 0.7 2340.5 0.5 0.98 0.99
graphql-native 385.3 1.5 384.2 2.1 1.00 1.00
lee 739.3 1.8 750.1 1.2 0.98 0.99
matmul 1494.2 0.8 1497.8 0.9 1.01 1.00
nbody 71.1 2.3 71.3 5.1 1.00 1.00
nqueens 169.8 1.6 167.6 2.0 1.00 1.01
optcarrot 4202.7 0.6 4216.8 0.5 1.00 1.00
rack 56.0 3.3 53.9 3.9 1.06 1.04
ruby-json 1976.3 0.9 1992.4 0.3 1.00 0.99
rubykon 6971.3 0.8 7053.4 0.4 1.00 0.99
sudoku 1836.1 0.4 1836.3 0.3 1.00 1.00
tinygql 441.4 1.0 447.4 1.2 0.96 0.99
30k_ifelse 1459.0 8.6 1429.1 4.6 0.96 1.02
30k_methods 3331.8 3.7 3264.0 1.1 1.03 1.02
cfunc_itself 81.5 1.8 81.2 2.9 1.02 1.00
fib 187.0 1.6 188.8 1.5 0.94 0.99
getivar 65.1 2.2 65.7 2.4 1.01 0.99
keyword_args 141.6 2.3 140.9 2.1 1.00 1.01
respond_to 184.9 1.9 185.7 1.1 0.96 1.00
setivar 39.2 2.9 39.2 2.4 1.03 1.00
setivar_object 72.7 2.5 77.5 2.1 0.95 0.94
setivar_young 72.5 1.9 77.5 0.9 0.93 0.94
str_concat 69.7 2.5 69.9 1.9 1.02 1.00
throw 14.9 5.0 14.8 4.5 1.04 1.01
-------------- ------------ ---------- ----------- ---------- -------------- --------------
Legend:
- frozen 1st itr: ratio of mutable/frozen time for the first benchmarking iteration.
- mutable/frozen: ratio of mutable/frozen time. Higher is better for frozen. Above 1 represents a speedup.
I used the following patch to disable frozen string literals globally: https://github.com/ruby/ruby/compare/master...Shopify:ruby:disable-frozen-string-literal, and ran the suite with ruby run_benchmarks.rb --chruby 'mutable::head-mutable-strings;frozen::head'.
Most of these benchmarks don't really deal with string, but one that I think is the most close to reality and to Ruby bread and butter is lobsters and it seem to be quite positive.
I also didn't enable YJIT, I suspect the difference would be bigger if I did, as the extra GC pressure would be relatively bigger.
I'll see about using the yjit-perf benchmark server to run these more scientifically.
I also want to note that I have to explictly freeze RUBY_DESCRIPTION in my patch, otherwise it would cause a Ractor issue, which is another argument for frozen strings, as they ease the necessary work necessary to make code Ractor compatible.
Updated by byroot (Jean Boussier) over 2 years ago
but because standardrb doesn't enforce it (and many folks have been defaulting to that), I've seen several codebases not consistently using the pragma.
Yes, I didn't extend on my motivations for bringing this now, but It's in big part influenced by standardrb and some other communities starting to discourage the use of frozen string literals. I suspect many people are starting to be fed up with that magic comment and we probably reached peak usage of it.
Updated by duerst (Martin Dürst) over 2 years ago
frozen_string_literal can lead to more efficient code, but it's also part of a programming style (usually called functional programming). Functional programming often leads to cleaner code, but it may require some additional programming effort. There's also a fundamental conflict between object-oriented programming (objects are generally mutable) and functional programming, although Ruby is pretty good at integrating these concepts.
My main issue with this proposal is that I think it's probably the right thing for most big code bases, but it may not be the right thing for quick-and-dirty small scripts. And Ruby is used, and should continue to be usable, for both kinds of code.
Maybe what could help is a declaration on a higher level, e.g. per gem or so rather than per source file. (That's just an idea, with many open questions: Where would the setting go? How would the interpreter pick it up? How would people become aware of it? ...)
Updated by Eregon (Benoit Daloze) over 2 years ago
+1 to this issue proposal, it sounds like a good plan.
And agreed with @zverok (Victor Shepelev), having frozen literals by default is not only faster and less wasteful in memory usage, but also a safer model, where "String used like buffers/mutated Strings" are more explicit, which certainly seems good for readability and avoiding to accidentally mutate a String which should not be mutated.