Finance and Economics Discussion Series: 2017-107 Screen Reader version ^♣

What's the Story? A New Perspective on the Value of Economic Forecasts

Steven A. Sharpe, Nitish R. Sinha, and Christopher A. Hollrah*

First draft: August 30, 2017

Abstract:

We apply textual analysis tools to measure the degree of optimism versus pessimism of the text that describes Federal Reserve Board forecasts published in the Greenbook. We then examine whether this measure of sentiment, or Greenbook text "Tonality", has incremental power for predicting the economy, specifically, unemployment, GDP growth, and inflation up to four quarters ahead; we also test whether Tonality helps predict monetary policy and stock returns. Tonality is found to have significant and substantive directional predictive power for the GDP growth and the change in unemployment over the subsequent four-quarter horizon, particularly since 1990. Higher (more optimistic) Tonality presages higher than forecast GDP growth and lower unemployment. Higher Tonality is also found to help predict tighter monetary policy up to four quarters ahead. Finally, we find that Tonality has substantial positive and significant power for predicting 3-month-ahead and 6-month ahead stock market returns.

JEL codes: C53, E17, E27, E37, E52, G40.

Keywords: Text Analysis, Economic Forecasts, Monetary Policy.

I. Introduction

For some time, a variety of researchers and market participants have questioned the value of economic forecasts, highlighting what seems to be a fairly dismal record. Nonetheless, substantial resources continue to be devoted to producing detailed economic forecasts. For instance, the Blue Chip Survey of Economic Indicators collects monthly updates of U.S. economic forecasts from over 50 of the "top analysts," the large majority of which are associated with private-sector profit-driven firms. The Blue Chip Financial Forecasts survey polls a similar set of analysts/firms on their interest rate and currency value forecasts, despite probably even less compelling evidence for success in predicting financial prices. Similarly, eight times a year (prior to each meeting of the FOMC committee), the staff at the Federal Reserve Board provide a detailed forecast of the U.S. economy (staff forecast). In December 2010, the most recently available public staff forecast, the document containing the staff forecast was over 100 pages long, with tables detailing forecasts for about 50 U.S. macroeconomic data series, plus dozens of additional series detailing forecasts of the federal budget, credit flows across sectors, plus GDP and inflation for major foreign countries and regions. In this paper, we provide a new perspective on the value of forecasts, which might explain why consumers of economic forecasts - financial market participants and policy makers - continue to pay for them.

In the academic literature, economic forecasts by the Federal Reserve staff as well as those from the private sector and academia have been evaluated for their predictive content, for evidence of bias, as well as for their comparative merit.¹ Such studies focus almost exclusively on the track record of quantitative point estimates of inflation and/or GDP growth, which are usually interpreted as either modal or mean predictions. Consequently, these studies ignore a major element of the forecasters' product, the narratives in which those quantitative forecasts are embedded. Such narratives tend to give a flavor of the range of plausible outcomes or characterize the direction of the most likely risks to forecasts, and in some cases explicitly consider alternative scenarios. This shortcoming of traditional research on forecast efficacy is not surprising, as quantitative forecasts have been conveniently catalogued for decades. However, it is plausible that policymakers and investors who pay for these forecasts draw significant value from the narratives that accompany individual forecasts, and new methods of text analysis offer the opportunity to explore this angle.

Our study breaks new ground by applying tools from the emerging literature on textual analysis in an attempt to evaluate a key dimension of the information conveyed in the narratives that accompany forecasts. To do so, we focus on Federal Reserve Board forecasts published in the Greenbook. In particular, we quantify the degree of optimism versus pessimism embedded in the Greenbook text, which we call the "Tonality" of the text, based upon counts of words that have been classified as positive or negative. The starting point for that classification is the Harvard Psycho-social dictionary, which is then fine-tuned by excluding words that have special meaning in an economic forecasting context, such as "demean" and "interest." Not surprisingly, the resulting measure of Greenbook text sentiment is strongly correlated with the point forecasts for economic variables in the Greenbook, specifically, forecasts for GDP growth, unemployment and inflation.

We then examine whether the resulting measure of optimism has incremental power, over and above numerical forecasts, for predicting key macroeconomic quantities--namely unemployment, GDP growth, and inflation. We consider horizons ranging from one quarter to four quarters ahead. In short, we find that Tonality has significant and often substantive directional predictive power, which is particularly sizable for the cumulative change in unemployment and GDP growth over the subsequent four-quarter horizon.

In light of the predictive power of Tonality for economic activity, the analysis then proceeds to test the logical corollary, which seems particularly relevant given the identity of the forecaster: does text Tonality help to predict shocks to monetary policy? For this test, we use two alternative measures of monetary policy expectations: (i) the Fed staff's own internal funds rate forecasts and, (ii) the consensus Blue Chip Financial Forecasts Fed funds rate forecasts. Indeed, using either measure as the benchmark forecast, we find that Tonality has significant marginal predictive power for monetary policy. In particular, a more optimistic tone presages a higher than anticipated Fed funds rate in the near-term and up to four quarters ahead.

Finally, we examine a couple implications of Tonality for asset prices. First, we consider whether Tonality can help explain the response of interest rates and stock prices to the monetary policy announcements that follow. Second, we examine whether Tonality can predict asset price changes beyond the announcement window. Tonality appears to have substantial asset-price effects in both cases. First, over the 30-minute announcement window, higher Tonality is found to presage an increase in Fed funds futures rates, as well as short and intermediate-term Treasury yields. This effect is estimated while controlling for the highly influential effect of current monetary policy surprises. The finding suggests that, at least to some extent, the sentiment reflected in Greenbook text, which tends to presage future monetary policy actions, seems to be conveyed in the Fed's current policy announcement.

Tests for forward-looking predictive content in Tonality turn up what perhaps are even more interesting results. We find that Tonality has substantial positive and statistically significant power for predicting stock market returns both for 3-month-ahead as well as 6-month ahead. Indeed, that predictability also holds up in out-of-sample tests. In particular, our estimates suggest that a mean-variance investor with unit risk aversion and real-time knowledge of Tonality would have earned a 12.8 percent higher annual return by adjusting quarterly her allocation to stocks over our sample period. Since the conditioning variable, Tonality, is not publicly observable, we interpret this finding as suggesting that the stronger economy predicted by Tonality represents future news to investors. News of a stronger economy raises expected profits and dividends and lowers investors' risk premiums.

While contributing to the literature on the efficacy of economic forecasts, our study also contributes to the relatively new and burgeoning line of research in economics that draws insights from treating text as a new source of data. Among the most widely cited text-as-data studies in economics is a study by Baker, Bloom and Davis (2016), who create measures of government economic and monetary policy uncertainty by measuring the usage of language in newspaper articles on the subject. The approach used in our analysis also has close parallels to recent studies that examine how the tone of newspaper articles helps explain or predict stock market returns (Tetlock 2007), or how the text in company earnings reports or equity analyst updates helps explain the company's stock price responses to earnings forecast revisions (Asquith, Mikhail and Au 2005).

Perhaps even more closely related is a recent study by Carvalho, Hsu and Nechio (2016). They quantify the sentiment in FOMC communication, which is then used to explain interest rate reactions to FOMC communication before and during the zero lower bound period. They find that an increase in Fed communications surprise is associated with lower increase in near-dated government bond yields but similar increase in five and ten year government bond yields before versus during the zero lower bound period. Finally, also in a similar vein to our study, Shapiro, Sudhof and Wilson (2017) find that sentiment gleaned from the text of newspaper articles outperforms other sentiment indicators, such as the University of Michigan index of consumer sentiment, in predicting various macroeconomic series including output, unemployment and S&P 500 returns.

In section II, we describe how we measure Tonality and explore how it co-varies with Fed staff's key quantitative forecasts. In section III, we examine whether Tonality can predict macroeconomic conditions. In section IV, we examine the relationship between Tonality and monetary policy. Section V explores relationship between Tonality and asset prices. Section VI examines two extensions - information content of positive and negative Tonality and Tonality as a predictor of recessions. Finally, section VII concludes.

II. Measurement of Tonality in Greenbook Text

A. Measuring Tonality

Prior to every scheduled FOMC meeting, the staff puts together its outlook for the U.S. economy along with its numerical forecast in a document called the Greenbook (now the Tealbook). Greenbook forecasts were published monthly up until 1980; thereafter, the frequency was reduced to eight per year. Since August 1974, the Greenbook was organized as Greenbook Part 1, the summary and outlook, which outlined the forecast, and Greenbook Part 2, which described recent developments. Our analysis focuses on the text of Greenbook Part 1, from 1973-2009. Prior to 1974, Greenbook was published as a single volume document which contained both the recent developments and outlook for domestic economic activity as well as international developments; we extract the Recent Developments and Outlook for Domestic Economic Activity as the source of text.

We construct an index that quantifies the optimism and pessimism of the Greenbook text, which we refer to as "Tonality." Tonality is equal to the difference between the weighted sum of positive and negative words from our word list. To classify words as "positive" or "negative," we create a custom dictionary of 231 positive words and 102 negative words.² To derive our dictionary, we adopt the initial classification of positive and negative words in the widely used Harvard psycho-social dictionary³ but then exclude words that have a different connotation in the forecasting context. For example, in contrast to the psycho-social dictionary, we do not consider the words "demean" or "hedge" as negative. Positive words in our dictionary include terms like "enthusiasm," "abundant," "enhance," and "successful." On the other hand, negative words include "unrest," "fragile," "trouble," and "gloomy." Our approach is most similar to Tetlock (2007) and Loughran and McDonald (2011), who examine word frequency without trying to gauge the context in which words are used. Like Tetlock (2007), we use the Harvard IV Psychosocial dictionary to classify words; and, like Loughran and McDonald (2011), we use weighted word counts and we cull from the list any words that have domain-specific connotation in economic forecasts.

By using the whole document to quantify the overall degree of optimism, irrespective of how words are grouped, we are choosing not to use more elaborate methods of text analysis that would, for instance, attempt to identify double negatives or text specific to the economic indicators whose forecasts we evaluate. Such approach would require a good deal of additional judgment, for instance, on how to quantify "nearby" words in text space. It would also necessitate excluding a lot of information such as the descriptors of the many other economic variables that are related to the specific forecasts on which we focus.

The Tonality index of a document compares the number of positive and negative words in its text, where a word's frequency of appearance in any given Greenbook is normalized by its average past frequency. Specifically, the weight for each word is equal to its current-document frequency (tf) multiplied by the inverse document frequency (idf) from the previous 40 Greenbooks, a weighting scheme commonly known as tf-idf.⁴ The tf-idf weighing scheme is based on the intuition that infrequently used words are especially informative and so receive relatively high weight in the index. By the same token, very frequently used words are discounted. Common application of tf-idf scheme would have used the inverse document frequency over all the Greenbooks, instead we chose a moving window of 40 past Greenbooks (roughly five years) to account for changes in writing style of Greenbook over time while ensuring that we do not introduce any look-ahead bias. Finally, the Tonality index is standardized to have a zero mean and a standard deviation equal to one. We adapt the Python machine learning library Scikit (Pedregosa, et al. 2012) for tf-idf scoring of Greenbooks.

Figure 1 shows two side-by-side word clouds for the 50 most prominent positive words in Greenbooks during two periods, 1994-1998 and 2005-2009. The word size is proportional to its contribution to Tonality, that is, its contribution to the sum of tf-idf weights during the five-year window. The word cloud for positive words during 1994-1998 is slightly bigger than that for 2005-2009. Word choices between these two periods that are roughly ten years apart are similar, suggesting there is not a lot of language drift, whereby many words simply fall out of favor and are replaced by new ones. The most important positive word in both periods is "upward", followed closely by "positive." However, the word "favorable" is a more prominent word during 1994-1998, as is the word "moderation."

Figure 1

Note: Word cloud for fifty most positive words in the Greenbook. The word cloud on the plot on left side shows fifty most positive words used in the Greenbook during the period Jan 1994 and Dec 1998. The word cloud on the right side shows fifty most positive words during the period Jan 2005 and Dec 2009. The size of individual word in a word cloud is proportional to its contribution in the calculation of Tonality during the plotted time-window.

Accessible version

Figure 2 shows two side-by-side word clouds for the 50 most prominent negative words in Greenbooks during the same two periods. The word cloud for negative words during 1994-1998 is smaller than that in 2005-2009. The most prominent negative word in both samples is "negative", followed by "sluggish." However, negative words are simply more prominent in the latter sample as indicated by the larger word sizes in the right-hand-side word cloud. For example, the word "adverse" is somewhat more prominent in 2005-2009 period. Similarly the word "recession" is much smaller in the 1994-1998 period than it is in the 2005-2009 period, perhaps not surprising since the later period includes the "Great Recession."

Figure 2

Note: Word cloud for fifty most negative words in the Greenbook. The word cloud on the plot on left side shows fifty most negative words used in the Greenbook during the period Jan 1994 and Dec 1998. The word cloud on the right side shows fifty most negative words during the period Jan 2005 and Dec 2009. The size of individual word in a word cloud is proportional to its contribution in the calculation of Tonality during the plotted time-window.

Accessible version

Figure 3 shows the Tonality index plotted over the full sample period, with positive levels indicated in green and negative levels indicated in red. As one might expect, Tonality appears to be procyclical, with the large majority of observations during recessions in negative territory, and a mixture of positive and negative observations during expansionary periods. Among the most deeply negative readings of Tonality are observations in the year leading up to and during the Great recession, and also during the 1974-75 recession. The most noticeable run of highly positive reading was during the mid-1990s. Despite these cyclical tendencies, Tonality also appears to be quite volatile, exhibiting a lot of high-frequency movement that is often quickly reversed. To some extent, these fluctuations might reflect noise in our proxy for sentiment.

Figure 3

Accessible version

In order to examine whether the high frequency movements reflect more noise than signal, we produce a measure of what we call "Trend" Tonality, by constructing an exponentially weighted moving-average of Tonality. For that calculation, the weighting parameter--the rate of decay of weight placed on lagged observations--is chosen to maximize the fit (minimize the mean squared distance) between the smoothed series and our raw Tonality measure. This yields the same measure of trend as that from estimating a Kalman Filter under the assumption that "Trend" Tonality followed a random walk. We define deviations of the Tonality from the Trend Tonality as the "Shock" component of Tonality. Figure 4 shows the resulting times series plot for "Trend Tonality", shown by the black line, along with (total) Tonality. Not surprisingly, the cyclical pattern in this smoothed measure of sentiment stands out more clearly.

Figure 4

Accessible version

B. Relation of Tonality to the Current Greenbook Point Forecasts

To explore possible links between the forecast text sentiment and the Fed staff quantitative forecast, we begin by examining simple correlations between Tonality and forecasts for three key economic performance variables: inflation, unemployment, and GDP growth. The first two are the two components of the Fed's dual directive. The third, GDP growth, also is a compelling candidate, being among the most frequently cited summary statistic of economic performance. For all three variables, we consider three metrics. The first is current economic conditions, specifically, forecasts of current-quarter inflation, GDP growth, and the unemployment rate. Second, we construct gauges for the outlook four quarters ahead: cumulative inflation and GDP growth over the next four quarters, and the four-quarter change in the unemployment rate. Since the sentiment embedded in the narrative may be influenced both by the state of the outlook as well as the nature of recent revisions, we also construct forecast revisions, relative to the previous Greenbook, for each outlook measure.⁵

The correlations between each of these forecast metrics and text Tonality--both raw Tonality and Trend Tonality are shown in Table 1. By either measure, the signs of the correlations between Tonality and our metrics of expected economic performance are consistent with what one might intuitively expect--negative for measures of inflation and unemployment and positive for measures of GDP growth. In most cases, those correlations tend to be somewhat stronger for Trend Tonality than raw Tonality. For each economic variable, the measure of (four-quarter) outlook is strongly related to Tonality. In contrast, Tonality does not appear to be correlated with revisions to the GDP or inflation outlooks.

We next explore the marginal contributions these forecast metrics have for "explaining" Tonality in a multivariate regression context. To facilitate this analysis, we omit forecast metrics that are uncorrelated with Tonality, and forecast metrics that appear redundant. We also omit clearly redundant measures-those that are very highly correlated with other measures, as gauged in Table 2. Accordingly, we omit GDP Outlook, which exhibits a correlation of 86 percent with Unemployment Outlook and 78 percent with Current GDP Growth; we also exclude Current Inflation, which exhibits a correlation of 84 percent with Inflation Outlook. Similarly we omit GDP revision which exhibits 70 percent correlation with unemployment revision. Regressions of Tonality determinants is explored in Table 3. For the full sample (1973 - 2009), shown in the first column, we find that, Tonality is negatively related to the Inflation Outlook, the Unemployment Outlook, and to the Unemployment Outlook revision. The forecast for current-quarter GDP has no marginal effect on Tonality. All told, the quantitative metrics account for about 22 percent of the variation of Tonality over the full sample.

Using the Bai and Perron (2003) test for multiple structural breaks, we test for structural breaks in this econometric relationship and find strong evidence for at least one break, which is estimated to have occurred in September, 1990. While there is some statistical support for a second break in December, 2000 (figure 5), the resulting improvement in BIC is not as large. Also, it is convenient for statistical analysis that the first break divides the sample roughly into halves, and dividing it further would increase the statistical barriers to uncovering significant results. The second and third columns show the Tonality regression estimates for the early (1973 to September 1990) and late (after September 1990) sub-periods, respectively.⁶

Figure 5

Note: Number of breakpoints and model improvement. The plot shows the decrease in Bayesian information criteria (BIC) and residual sum of square as we increase the number of breakpoints in the relationship between Tonality and current Greenbook point forecasts. The first breakpoint corresponds to September 1990 and the second breakpoint corresponds to December 2000). There is a U shaped relationship between BIC (shown on the left Y axis) and number of breakpoints, the optimum number of breakpoints is two. However, the second breakpoint does not lead to as much improvement in BIC (lower is better) as the first breakpoint.

Accessible version

Perhaps the most notable difference in the factors driving text Tonality in the two sub-periods is a change in the sign for the Inflation Outlook. Prior to September 1990, Inflation Outlook has a significant negative marginal effect on Tonality; whereas, in the later period, it has a positive marginal effect on Tonality. Although the later positive effect is a little puzzling, the overall result is consistent with the idea that the Federal Reserve was more concerned about fighting inflation earlier in our sample. On the other hand, while the Unemployment Outlook has a negative marginal effect on Tonality in both sub-periods, it appears to be quite a bit stronger in the later period. Interestingly, current GDP growth has a negative, though only marginally significant, effect on Tonality in the earlier regime; but has the more intuitive positive marginal effect in the later regime. Finally, our variables explain twice the fraction of variation in the later period (adjusted R-squared of 35%) as in the early period (adjusted R-squared of 16%).

III. Greenbook Tonality as an Economic Indicator

Given the strong contemporaneous connection between Tonality and key economic performance measures, our analysis turns to a central question of interest: does Tonality have marginal predictive power for those measures of economic performance? For instance, does Tonality contain information regarding future GDP growth that is not fully reflected in the GDP forecast itself? To gauge the predictive content of Tonality, we estimate regressions that test whether Tonality helps to predict the three key economic performance variables forecasted in Greenbook. We measure performance over 3 different horizons, the next quarter, two quarters ahead, and four quarters ahead. In each case, the dependent variable is the realized cumulative performance for the variable in question, and the explanatory variables are Tonality and the same-horizon Greenbook point forecast. In light of the structural change in how Tonality of Greenbook text related to inflation and GDP growth, regressions are estimated separately on the pre- and post-September 1990 subsamples.

The baseline econometric framework for our analysis is adopted from the extensive literature on forecast rationality and efficiency, beginning with studies such as Zarnowitz (1985) and Aggarwal, Mohanty and Song (1995), which examines whether economic forecasts embed systematic errors. The canonical approach involves regressing the realized value of the forecasted variable on the forecast and testing whether the coefficient on the forecast is unity and the intercept is zero. Following on this, "forecast efficiency" tests then examine whether adding other information variables to the regression helps predict the variable of interest.

In our analysis, this suggests the following basic specification:

$$\displaystyle {Realized}_{t+h}=\alpha+\gamma_h{Forecast}_{t,t+h}+\ \beta_h{Tonality}_t+\varepsilon_{t,h}\ $$

This represents an efficiency test for the Greenbook forecast because any information reflected in Tonality is presumably observable to the Fed staff making the forecast. We also incorporate other information available at time t when the forecast is produced, which in previous studies have been found to improve upon the forecast or help to predict forecast error. One of these is the revision to the forecast from the previous Greenbook, which is motivated by findings of "information rigidities" by Coibion and Gorodnichenko (2012), who document that forecasts by the Survey of Professional Forecasts tend to be only partially adjusted toward their mean-square-error minimizing value. We also control for recent stock market returns in the regressions predicting economic variables (Stock and Waston 2003) and interest rate term spreads in regressions predicting the Fed funds rate (Rudebusch and Williams 2009).

A. Predicting GDP

Baseline regressions that examine the predictive content of Tonality for future GDP growth are shown for the early and late sample periods in Table 4 panel A and panel B, respectively. The dependent variables are the realized 1 quarter-ahead GDP growth in the first column; and the realized cumulative GDP growth looking 2 and 4 quarters out, in the second and third columns respectively. The first set of three regressions examines the predictive content of the GDP growth forecast by itself. Tonality is added in the second set of regressions, and in the third set Tonality is broken down into its trend and shock components. Standard errors shown below coefficient estimates are corrected for autocorrelation for (2*k +1) lags for k quarter out forecast error regression using the automatic bandwidth selection procedure described in (Newey and West (1994).

For the early subsample (the pre-1990 sample), coefficient estimates on the staff point forecast range from 0.94 for the 1 quarter-ahead forecast to 0.82 for the four-quarter forecast, each of which is not significantly different from 1.0. The R-squared statistics range from 0.58 for 1 quarter-ahead GDP to 0.45 for 4-quarter growth. In columns 4-6, adding Tonality to these regressions does little to boost the R-squared statistics and Tonality is only marginally significant in the four-quarter forecast. Results are a bit more interesting when we break out the two components of Tonality: in this case, Trend Tonality has a large and significant coefficient for the 4-quarter forecast, while the adjusted R-squared in that case rises to 0.54 from 0.45 in the basic regression.

Results are considerably more interesting for the late period (Table 4 Panel B). Results for the initial benchmark regressions on forecast alone are fairly similar, though the coefficient on the 4-quarter forecast is lower at 0.76 and the adjusted R-squared in that is only 0.25. This is not because forecasts are less accurate than in the early subsample, as the residual standard error is quite a bit smaller in the later-period regression. Rather, this would seem to reflect the reduction in predictable variation that came with the dampening of the business cycle around the mid-1980s (the "great moderation"). Perhaps more interesting, we find that Tonality has a strongly significant positive coefficient (at the .01 percent level) for the two longer horizons, while the R-squared statistics rise notably, particularly for the increase from 0.25 to 0.36 at the 4-quarter horizon. At this horizon, the coefficient estimate implies that a one-standard deviation increase in Tonality boosts expected GDP growth by 71 basis points.

When we split Tonality into its trend and shock components, shown in columns 7-9, we find that Trend Tonality is the important component for predicting GDP growth 2 and 4 quarters out, and this specification boosts the adjusted R-squared in both those regressions. For the 1 quarter-ahead regression, in contrast, there is no material improvement in regression fit. Another telling observation is that the coefficient on the staff forecast declines when Tonality is added to the regression, and again in the Trend Tonality specification. This suggests that the consumer of the Greenbook should place only partial weight on, or fade, the Greenbook point forecast, while incorporating information from the narrative in Greenbook quantified by Tonality, particularly its trend value.

To test the robustness of these results and their interpretation, we add to the regressions two control variables discussed above--the revision to the GDP forecast and stock market appreciation since the last FOMC meeting. Results are shown in Table 5. Interestingly, here we find evidence that, in the early period, the recent forecast revision had higher ability to predict realized GDP growth over most horizons, with a coefficient ranging between 0.18 and 0.09, though statistically significant only for 2 quarter ahead GDP suggesting that the GDP forecast was somewhat "sticky" in the sense of Coibion and Gorodnichenko (2012). In the later period the coefficients on Revision are smaller, suggesting forecasts have become less sticky. In addition, these regressions indicate that recent stock returns also have positive marginal predictive power for GDP growth, particularly in the post-1990 period.

B. Predicting Unemployment

Results from estimating analogous regressions for predicting the change in unemployment rate are shown in Table 6 Panel A and Panel B. Overall, findings regarding the predictive effects of Tonality are quite similar to those for GDP, and even a bit stronger. In short, Trend Tonality has some predictive power for unemployment in the early period, though only for the 4-quarter horizon regression (last column), where the adjusted R-squared rises to 0.68 compared to 0.62 in the baseline specification that conditions only on the point forecast. In the later subsample, Tonality, and particularly Trend Tonality, contributes to the predictive power for all three horizons. The contribution at the 4-quarter horizon is quite substantial, with the R-squared rising from 0.49 to 0.61. At the same time, the coefficient on the point forecast for unemployment drops from an oversized 1.53 in the baseline regression (3rd column) to 0.91 in the specification that includes Trend Tonality (last column). A one-standard deviation (0.72) increase in Trend Tonality reduces expected unemployment by about 1/2 percentage points.

Adding the control variables (Table 7) has essentially no effect on the Tonality coefficients. Moreover, except for the current-quarter forecast (not shown) in the early period, the forecast revision has no marginal predictive power for realized unemployment. Stock returns have some predictive power, which is statistically more convincing in the early period.

C. Predicting Inflation

Results from estimating the regressions for (cumulative) inflation forecast errors are shown in Table 8. Overall, Tonality appears to have little predictive power for inflation forecast errors. Qualitatively, the early (Panel A) and late (Panel B) periods look quite different in that Tonality has negative coefficients in the early period and positive coefficients in the late period. This seems consistent with the interpretation that higher inflation was of greater independent concern in the 1970s and 1980s; however, in only one case--the 1 quarter-ahead forecast in early period--does the coefficient on trend Tonality display some statistical significance. Although not shown, results from regressions with controls added do not alter these conclusions.

Though somewhat tangential to focus of this paper, it is interesting to note the very small coefficient estimates on the Staff Forecast in the longer-horizon regressions for the later period. For instance, the coefficient estimate of 0.13 for the four-quarter forecast (Panel B, column 3) implies that that forecast itself has no predictive power for actual inflation over the four quarters ahead; together with the large positive intercept estimate, these estimates suggests that forecast errors would be significantly reduced for the longer horizon forecasts if the forecast had called for a constant inflation rate equal to the expected average rate. This echoes findings by Atkeson and Ohanian (2001) and Stock and Watson (2007) that inflation has become harder to forecast since the mid-1980s.

D. Greenbook Tonality and Blue Chip Forecasts

So far, our findings indicate that Tonality of the Greenbook narrative has predictive value for GDP growth and unemployment, conditional on the Greenbook forecast. One question this raises is whether our findings reflect any particular built-in complementarity between the point forecast and the narrative. For instance, are there biases in the Greenbook point forecasts induced by some complementary communication built into the forecast narrative? While we cannot test this directly, it is possible to examine whether the predictive content of Tonality holds up when we instead condition on economic forecasts outside the Fed. If so, this would suggest that, even by itself, the information content measured by Tonality would be valuable not only as a complement to the Greenbook point forecast but also to other economic or financial market participants.

We use the Blue Chip consensus economic forecasts to conduct such an exercise. Of course, doing so requires contending with an imperfect match in the timing of when the Blue Chip and the Greenbook forecasts are published. We match as follows: When the Greenbook forecast is published on or before the 15th of the month, then that Greenbook (its Tonality) is married with the most recent (previous-month) Blue Chip forecast; otherwise, it is married with the upcoming (end-of-current-month) Blue Chip forecast. While the analysis was conducted using all three forecast horizons and on both sub-periods, for brevity, we focus on the for the 2 and 4-quarter forecast horizon only in the post-1990 sample, displayed in Table 9, but conclusions are the same for shorter forecast horizons.

The dependent variables in the first two columns are realized 2 and 4-quarter GDP growth, while explanatory variables are the Blue Chip 4-quarter GDP growth forecast and the components of Greenbook Tonality; regressions are estimated with the additional controls, though we get very similar results without controls as well. All variables are measured at or around the time of the Greenbook published. The second and third pairs of columns show analogous regressions for the four-quarter change in unemployment and the four-quarter inflation rate. In each case, the results are remarkably similar to the analogous regressions that conditioned on the Greenbook forecast. In particular, coefficients on Trend Tonality are very close in magnitude. Moreover, when conditioning on the Blue Chip forecast, Trend Tonality is again a statistically strong positive predictor of future GDP growth, a strong negative predictor for future unemployment, and a modestly positive predictor of future inflation. These results suggest that the predictive content in the Tonality of the Greenbook forecast narrative does not seem to owe to a unique complementarity between Greenbook point forecasts and Tonality.

IV. Tonality as a Predictor of Monetary Policy

Given that Tonality is helpful for predicting economic performance up to four quarters ahead, a reasonable corollary hypothesis would seem to be that Tonality has predictive power for monetary policy over a similar horizon. In particular, in the post-1990 period, higher Tonality tends to signal stronger future economic activity relative to economic point forecasts, whether those forecasts are by Fed staff or the private sector. As a consequence, one might expect higher Tonality to predicate higher policy rates, and perhaps even higher-than-forecast policy rates. By the same token, Tonality might help to predict movements in the prices of assets whose values are affected by macroeconomic news or Federal Reserve policy.⁷

The logic of the hypothesis that Tonality would predict surprises in the Fed funds rate seems straightforward, but there are potential differences between the analysis that conditions on Greenbook and private forecasts of the funds rate. To the extent that Blue Chip consensus forecasts of interest rate policy are connected to Blue Chip consensus forecasts for the economy (such as through a perceived Taylor rule), then positive economic surprises presaged by Tonality would, in turn, presage positive surprises in the path of policy rates.

The logic for such a connection is somewhat murkier in the case of funds rate forecast surprises relative to Greenbook, particularly in light of arguments by Reifschneider and Tulip (2017). They report that the Greenbook traditionally has taken a more "neutral" approach to the Fed funds rate forecast, that it has tended to "condition on [funds rate] paths that modestly rose or fell over time in a manner that signaled the staff's assessment ...[of the required] adjustment in policy." This suggests that Greenbook funds rate forecasts will tend to appear timid relative to a prescriptive forecast, an additional reason that Tonality might help to predict surprises in the funds rate, relative to the Greenbook forecast.

Our baseline funds rate forecast regressions that condition on Greenbook point forecasts for the funds rate are shown in Table 10. In these regressions, the realized change in the funds rate is regressed on the Greenbook forecast of the change in funds rate (over the same horizon) and Tonality. As shown in the first three columns, which include solely the Staff Forecast, coefficients on the point forecast are highly significant and are also quite a bit larger than 1.0, particularly at longer horizons. This seems consistent with the claim by Reifschneider and Tulip (2017) that Greenbook funds rate forecasts tend to be timid. As hypothesized, when Tonality is added to regressions in the subsequent three columns, its coefficients are positive and significant at all three horizons; thus, Tonality helps forecast the funds rate. Improvement to the regression fit is modest, however, with the R-squared rising most at the 4-quarter horizon, from 0.39 to 0.42. At the same time, the coefficient on the Staff Forecast remains significantly higher than 1.0. Finally, in contrast to our results for GDP and Unemployment, splitting Tonality into its two components (the final three columns) does not improve regression fit, as the coefficients on the Trend and Shock components are not statistically distinguishable.

Analogous tests that condition on Blue Chip funds rate forecasts are shown in Table 11, with results that are remarkably similar. For all three horizons, the coefficient on the forecast in the basic specification is again substantially and significantly higher than 1.0. Interestingly, though, Blue Chip forecasts appear to be somewhat more correlated with realized changes in the fund rate compared to the Greenbook Fed funds forecasts, as indicated by the somewhat higher R-squared statistics. Nonetheless, similar to the regressions that condition on the Greenbook forecast, when Tonality is added, it has statistically significant positive coefficients. Also similar, even after the addition of Tonality, the coefficient on the forecast remains significantly higher than 1; and the decomposition of Tonality into Trend and Shock does not boost predictive power, except in the 4-quarter horizon regression.

The previous tests of Tonality's forecasting power for the Fed funds rate were conditioned on either Greenbook or Blue Chip point forecasts for the funds rate. Table 12 runs similar tests but with additional variables that might help forecast changes in the funds rate. As with the analysis of economic forecasts, we control for forecast rigidity using the revision in the funds rate forecast relative to the previous Greenbook forecast (or, in the case of Blue Chip, relative to the Blue Chip forecast nearest to the previous Greenbook). We also add a measure of the market expectation for the change in the funds rate over a comparable horizon. This allows us to test whether economists' forecasts of the funds rate efficiently reflect market expectations and, at the same time, will help assess the extent to which Tonality's predictive power is related to market expectations. Following Gurkaynak, Sack and Swanson (2007), we gauge market expectations as the spread between Fed funds futures contract rates and the current funds rate, using the futures contract maturity that best approximates the forecast horizon.⁸

As shown in the first three columns, the coefficients on the Greenbook funds rate forecast drop dramatically relative to the regressions that did not include futures rates. Thus, the results unequivocally imply that the Greenbook funds rate forecast does not reflect the market's information about the likely path for the funds rate. What is more, the coefficient on Revision is positive and significant, suggesting that the staff forecast for the funds rate is "sticky". Even so, Trend Tonality remains a significant predictor of the future funds rate, with coefficients little effected, suggesting that Tonality contains information not reflected in the futures rates or forecast revisions.

The remaining columns show the results when regressions are conditioned on the Blue Chip forecast. Here again, we find that the Fed funds futures rates help predict the realized funds rate at all three horizons, though with somewhat smaller coefficients. Also, coefficients on the forecast are only a bit attenuated relative to the regressions that did not include the futures. These results suggests that, while Blue Chip forecasts do not reflect all the information about subsequent policy signaled by markets, they reflect substantially more than the Greenbook forecasts--again, consistent with the (Reifschneider and Tulip 2017) perspective on the Greenbook funds rate projection. Nonetheless, we again find that coefficients on Trend Tonality remain significant and little affected by the addition of futures rates. Therefore, as surmised earlier, it appears that Greenbook Tonality has marginal predictive power for monetary policy as well as for real economic variables.

V. Tonality and Asset Prices

In light of the predictive power of Tonality for future monetary policy as well as economic activity, these findings naturally beg the question of whether the sentiment reflected in Greenbook Tonality also helps to predict changes in asset prices such as market interest rates and stock prices. We first examine this from a short-term perspective, specifically, by testing whether Greenbook Tonality helps explain the market reaction to post-meeting FOMC policy announcements. If Tonality does explain some of the market reaction, it would suggest that information reflected in Tonality is conveyed in those policymaker communications (the week after Greenbook completion). These tests look for effects on interest-bearing securities of various maturities, as well as on stock returns.

We then briefly turn to consider whether Tonality has predictive power for asset price movements in the months subsequent to the current-meeting monetary policy announcement. If the information content of Tonality (for future GDP, unemployment, or the funds rate) is not fully reflected in asset prices after the FOMC meeting policy announcement, then Tonality might help predict subsequent asset price changes. Here, we only consider a brief foray into tests of stock return predictability that employ a straightforward extension of that literature. Indeed, given that we already have shown Tonality helps predict innovations to Fed funds rates up to four quarters ahead, at least in the post-1990 sample, the implications for bond return forecasting seem potentially quite rich, and deserving of careful attention; thus we reserve this topic for future study.

A. Post-FOMC Announcement Effects

To the extent that Tonality provides a signal about the trajectory of future monetary policy, then communications that follow the subsequent FOMC meeting could convey that sentiment and thus color the financial market response to those communications. Since February 1994, the FOMC has explicitly announced changes in the target Fed funds rate following the FOMC meeting in a press release that, over time, has increasingly conveyed the context for decisions or considerations for future decisions. We consider whether market reactions to the post-FOMC meeting policy announcements can by explained, in part, by the Tonality of the recently-produced Greenbook, despite its being unobservable to the market.

Although financial market participants pay close attention to every nuance in FOMC announcements, the market price reactions to information embedded in that message could be swamped by the market reaction to the current decision regarding the Fed funds rate--the "monetary policy surprise"--conveyed in the policy announcement. Following Gurkaynak, Sack and Swanson (2005), we measure the monetary policy surprise at announcement as the event-window change in the current-month Fed funds futures rate, adjusted for the number of days remaining in that month. We also use the narrow window of 30 minutes around the press releases, 10 minutes before until 20 minutes after, to measure the market reaction.

We first test whether Greenbook Tonality helps to explain the response to the FOMC announcement by Fed funds futures rates on three different futures contracts beyond the current-month contract. In particular, we examine the announcement effect on futures rates on contracts expiring around the subsequent FOMC meeting, the meeting after that, and the fourth meeting scheduled 6 months out. Changes in these futures prices convey the market's interpretation of the news for the near-term path of policy. We then test whether Tonality also helps to explain the event-window response to post-FOMC meeting communications by Treasury bond yields and the S&P 500 index. In particular, to examine whether incremental information embedded in Tonality is conveyed to financial markets during announcements, asset price changes over the event window are regressed on Tonality, while controlling for the effect of the monetary policy surprise:

$$\displaystyle {Market\ price\ change}_t=\gamma \times {Monetary\ Policy\ Surprise}_t+\ \beta\ \times Tonality+\varepsilon_t\ \ $$

Regressions that explore the changes to the Fed funds futures prices on contracts that expire in the months following upcoming FOMC meetings are presented in the Table 13. For each futures contract, we first estimate the effect of the monetary policy surprise on its own, and then subsequently add Tonality and finally its two components. Consistent with Gurkaynak, Sack and Swanson (2005), the current monetary policy surprise has a large and highly significant effect on the Fed funds futures rate for each of the contracts expiring over the next several months. The coefficient of 0.81 on the nearest contract implies that a 10 basis point positive surprise to the current funds rate boosts that futures rate 8 basis points. For the contracts further out, the effect is a bit smaller and though still strongly significant for contracts further out.

Regarding the tests of principal interest, we find that higher Greenbook Tonality predicates a significant positive incremental reaction by Fed funds futures rates to the monetary policy announcement for contracts at all three horizons. Breaking Tonality into its two components, it is interesting that we find that, unlike results for predicting economic variables, both the trend and the shock components of Tonality contribute to this effect. In fact, for the nearest contract, which expires around the next meeting, all the explanatory power appears to reside in the Tonality shock. This suggests that, in evaluating the relevance of the announcement for the subsequent FOMC meeting, the market seems to be mostly attuned to innovations in the tone of the Greenbook, even though they are not directly observable. Also somewhat interesting, we find that the contribution of Tonality (or its components) to the regression R-squared increases as we look to funds rate contracts maturing three meetings (six months) out.

Table 14 explores the sensitivity of bond yields and stock prices to Tonality during the 30 minute announcement window, 10 minutes before and 20 minutes after the FOMC announcement. The dependent variable in the first two columns is the announcement-window change in yield for on-the-run 6-month Treasury bills. As in previous table, regressors include the policy surprise and either Tonality or its two components. The next four columns show the same specifications for 2-year and 5-year Treasury bond yields. Consistent with the effects on Fed funds futures, both the policy shock and higher Tonality are associated with substantial and statistically significant increases in 6-month and 2-year Treasury yields. In contrast, the policy shock has no marginal effect on the 5-year Treasury yield, but higher Tonality again presages an increase in that yield over the announcement window.

The last two columns examine the stock market return during the FOMC policy announcement window. As shown, monetary policy has a substantial negative effect on stock returns, and the magnitude of the coefficient implying that a 10 basis point shock lowers stock prices about a third of a percent. On the other hand, higher Tonality does not presage a differential effect on stock returns.⁹

B. Tonality and Future Stock Returns

We next consider whether Greenbook Tonality has predictive power for stock prices beyond the current-Greenbook FOMC meeting announcement. These tests look for predictive power over roughly a 3-month horizon and a 6-month horizon, each beginning with closing prices on the current-Greenbook FOMC announcement day. For observations after 1980, the endpoints of the two prediction periods correspond to the FOMC announcement days that follow the second prospective meeting (about three months hence), and the fourth prospective meeting (six months hence). Before then, meetings were monthly, so the two prediction periods end with the announcements after the third and sixth prospective meetings. Regressions use the full sample, as there is no evidence of a structural break at 1990 or anywhere else.

Table 15 shows coefficient estimates from in-sample regressions predicting 3-month and 6-month S&P 500 index returns (price appreciation), for three alternative specifications. Shown below each specification is the in-sample R-squared as well as an out-of-sample R-squared, the latter simulated beginning in March 1980 (with 80 observations reserved to estimate the initial historical relationship). To establish a benchmark for comparison to previous research, the first pair of regressions focus on three predictors that have frequently been found to have at least in-sample predictive power for stock returns--the 3-month Treasury yield (short rate) (Campbell 1987), the dividend yield (Ball 1978), and the consumption, wealth, income ratio (cay) (Lettau and Ludvigson 2001).¹⁰ The second pair of regressions then adds Trend Tonality to the set of conditioning variables, and the third pair conditions only on the Trend Tonality.

In the benchmark specification, we find that the conventional conditioning variables have a substantial degree of in-sample predictability for our sample period, particularly for the 6-month returns, which has an R-squared of 8.4 percent. On the other hand, the out-of-sample R-squared statistics are quite negative for both 3- and 6-month returns, a contrast that mirrors findings of Welch and Goyal (2008). The third and fourth columns show the results when the two Tonality variables are added to the regression. At both horizons, Trend Tonality is highly significant, with a positive coefficient, in each case boosting the in-sample R-squared substantially. In particular, the in-sample R-squared increases to 8.7 percent for the 3-month horizon and 18.9 percent for the 6-month horizon. As with the benchmark estimates, the out-of-sample R-squared is much lower in both cases, though it remains sizable for the 6-month horizon.

The results for the final pair of specifications, which condition only on Trend Tonality show again show positive predictive power for stock returns at both horizons, with larger coefficients and higher statistical significance for the longer horizon. The size of these effects seem fairly substantial. A change in Trend Tonality of one--which amounts to roughly 1.5 standard deviations in this variable--presages about 4.1 percent higher stock return over the subsequent 6 months (or 4-meeting period). The in-sample R-squared statistics for the 3-month and 6-month horizons are 2.6, and 5.4 percent, respectively; while quite a bit lower than the combination specification, these are still quite respectable compared with most stock return predictive regressions in the literature for example, (Welch and Goyal 2008). More interesting, however, is the result that out-of-sample R² statistics are 3.0, and 5.3 percent for stock returns cumulated over the 3-month and 6-month horizons, respectively, which are economically substantial and quite similar to the in-sample values.

In interpreting these results, we do not seek to draw direct implications for the literature on time-varying expected returns, as Greenbook Tonality was unobservable to market participants and presumably only vaguely observable to even the authors of the documents. Coupled with our results with regard to economic forecasts, the natural interpretation would seem to be that the future performance of the economy predicted by Tonality represents future news to investors. News of a stronger economy presumably would translate to news of stronger corporate cash flows and dividends as well a decline in risk premiums.

In any case, it would be interesting to know how much an investor could have improved upon their investment performance if they had had access to Greenbook (and the algorithms used to extract Tonality!) in real time. To quantify the gains from using real-time knowledge of Tonality for portfolio allocation, we use Campbell and Thompson (2007) framework, which gauges economic gain from observing a mean zero signal T_t about the expected return on risky asset. In our case, that signal is equal to Trend Tonality multiplied by its coefficient in return prediction regression. The risky asset return can then be expressed as the sum of unconditional expected return on the risky asset (m), the signal (T_t), and a random shock (e) with mean zero and variance $$ \sigma_e^2$$. Thus, the proportion of total variance of the risky asset return explained by the variance of the signal is $$ R^2 = \sigma_T^2/(\sigma_T^2 + \sigma_e^2)$$. Letting $$ S = (\mu - r_f / ((\sigma_T^2 + \sigma_e^2))^{1/2}$$ represent the Sharpe ratio of the risky asset when no signal is observed, and g represent relative risk-aversion, then the gain in expected return from observing the signal is equal to¹¹

$$\displaystyle \frac{R^2}{\left(1-R^2\right)}\ \frac{(1+\ S^2)}{\gamma}$$

To simulate the investor experience, we use an "out-of-sample" R², shown in the table and cited above, which measures the gain from the knowledge of Trend Tonality at time t, given and the past statistical relationship between Tonality and investment-period stock returns. Letting $$ \widehat{r_t}\ $$ denote the expected return in t conditional on Trend Tonality, and $$ \overline{r_t}$$ denote unconditional expected return, then it is calculated as

$$\displaystyle R^2_{OS}=1-\ \frac{\ {\Sigma }^T_{t=81}{(r_t-\ \widehat{r_t}\ )}^2}{\ {\Sigma }^T_{t=81}{(r_t-\overline{r_t})}^2}$$

If Tonality is a good predictor of future equity returns, actual return is on average closer to the predicted value than to the unconditional average value; thus resulting in a higher out-of-sample R².

To calculate the gain in returns, we use 2-meeting ahead "out of sample" R²of 3 percent for our numerical example, as it implies investment horizon of roughly 3 months, a middle of the range investment horizon. The 3-month Sharpe ratio for stocks over 1927-2009 is 0.18, so plugging in S² = 0.032525 (3.25%) to the gain formula, yields the result that a mean-variance investor with risk aversion parameter $$ \gamma$$=1 can use Tonality to increase the average three-month return of her equity portfolio by 3.2 percentage points or 12.8 percentage points annually. Similarly, the for 6-month horizon Trend Tonality leads to 11.7 percentage points annual gain. Clearly, the potential gain from real-time knowledge of Tonality offers substantial improvement. Of course, this calculation ignores transactions costs, which would probably be non-trivial given that this strategy would have required shorting the equity market during some of the period. The gain would also be smaller for an investor with higher risk aversion.

VI. Extensions

This section considers a couple of angles to provide a bit more color on how sentiment in the Greenbook text might be used for forecasting the economy. The first question we consider is whether there is different information in negative words than in positive words. For instance, Loughran and McDonald (2011) find that in annual reports of U.S. firms, negative words are more informative than positive words. In our construction, Tonality is equal to the amount of positive Tonality minus the amount of negative Tonality; thus, it can easily be decoupled into two separate components. For the exercise in Table 16, we estimate a decomposition for each of the two signed components of Tonality into trend and shock components, yielding four variables, Trend Positivity, Positivity Shock, Trend Negativity and Negativity Shock. We then estimate regressions predicting 2-, and 4-quarter GDP growth and Unemployment change and stock prices on the four components.

Table 16 shows that, for realized GDP growth, the predictive information is almost exclusively coming from Trend Positivity. For the current quarter, the hypothesis that the positive and negative trend components have equal and opposite coefficients can be rejected with 10 percent confidence, and for two and four quarters ahead the hypothesis can be rejected at 5 and 1 percent confidence, respectively. A different picture emerges for the unemployment forecast, where we find that, at least for the longer two horizons, both the positive and negative component of Tonality each contribute with the expected sign. High Positivity predicts a lower unemployment rate relative to the staff forecast; and High Negativity predicts a higher unemployment rate. Furthermore, we cannot reject the hypothesis that Positivity and Negativity have equal and opposite signs for the two horizons. For stock price changes, we cannot reject the hypothesis that the two pieces have equal and opposite effects, but only Trend Positivity continues to show statistical significance at each horizon. Results for stock price predictability from Positivity are very similar when the full sample is used to estimate the predictability.

The second dimension along which we briefly consider generalizing is the measure of economic conditions that might be predicted by Tonality--specifically, we look at whether Tonality could have helped to predict recessions, at least in-sample. For this question, we refocus on the full sample in order to avoid over-fitting, a risk given the very small number of recessions in each sub-period. We estimate probit models for recession 3-months, 6-months and 12-months ahead, using the NBER definition of recessions for the U.S. economy from 1973-2009, results from which are shown in Table 17.

In the initial specification for each horizon, we include the staff forecast for unemployment change and GDP growth rate over the matching horizon (1, 2, or 4 quarters ahead) and an indicator variable for whether or not the economy was in recession 2 months prior to the month in question. In short, we find that Trend Tonality is a significant predictor of recessions at all three horizons. When we control for the term spread (10 year minus 1 year Treasury yield) and the GZ (credit) spread (Gilchrist and Zakrajsek 2012), two well-established financial market based predictors of recessions, the magnitude of Tonality's predictive power declines but is it remains a statistically significant predictor.

VII. Summary, Interpretation, and Conclusions

The predictive contribution of Greenbook Tonality for unemployment and GDP growth, when conditioning on the Greenbook forecast for those variables, suggests that an important element of economic forecasting is found in the accompanying narrative. The information embedded in the text appears to be more broadly valuable than simply a complement to the Greenbook forecast, given that we find a similar contribution to prediction when conditioning on Blue Chip forecasts. The analysis also indicates that very little if any of the predictive ability of Tonality seems to reflect either stickiness in the forecast or information reflected in asset prices.

The fact that Tonality predicts monetary policy surprises in the futures months indicates that Tonality conveys policy-relevant information. Our finding that Tonality explains some of the response of interest rates beyond the current Fed funds spot rate to monetary policy announcements suggests that monetary policy makers absorb information from the Greenbook beyond the point forecasts and communicate some of the information to market participants. The finding that Tonality predicts equity prices over the subsequent six months is notable but perhaps should not be surprising given its ability to predict economic growth. That results suggests that equity prices do not contemporaneously impound all the information that has been aggregated into the forecast narrative.

The evidence presented in this paper argues for examining forecast effectiveness while augmenting other information forecasters are relaying along with the quantitative forecasts. Doing so will require preserving (and in some cases) obtaining the narrative accompanying the forecasts. While the paper shows that the narrative of economic forecast is informative in itself, it leaves an important question unanswered - is the narrative of other economic forecasters similarly informative or is the Federal Reserve's staff forecast special in this regard.

The paper uses a relatively coarse measure of textual information. Deeper and more targeted textual analysis could lead to more insight into the economic forecasting process.

VIII. References

Aggarwal, Raj, Sunil Mohanty, and Frank Song. 1995. "Are Survey Forecasts of Macroeconomic Variables Rational?" The Journal of Business 68 (1): 99-119.

Asquith, Paul, Michael B. Mikhail, and Andrea S. Au. 2005. "Information content of equity analyst reports." Journal of Financial Economics 75 (2): 245-282.

Atkeson, Andrew, and Lee E. Ohanian. 2001. "Are Phillips curves useful for forecasting inflation?" Federal Reserve Bank of Minneapolis. Quarterly Review 2-11.

Bai, Jushan, and Pierre Perron. 2003. "Computation and analysis of multiple structural change models." Journal of Applied Economterics 18 (1): 1-22.

Baker, Scott R., Nicholas Bloom, and Steven J. Davis. 2016. "Measuring economic policy uncertainty." The Quarterly Journal of Economics 131 (4): 1593-1636.

Ball, Ray. 1978. "Anomalies in Relationship between securities' yields and yield-surrogates." Journal of Financial Economics 6 (2-3): 103-126.

Campbell, John Y. 1987. "Stock returns and the term structure." Journal of Financial Economics 18 (2): 373-399.

Campbell, John Y., and Samuel B. Thompson. 2007. "Predicting excess stock returns out of sample: Can anything beat the historical average?" The Review of Financial Studies (21): 1509-1531.

Carvalho, Carlos, Eric Hsu, and Fernanda Nechio. 2016. "Measuring the effect of the zero lower bound on monetary policy." Federal Reserve Bank of San Francisco Working Paper 1-32.

Coibion, Olivier, and Yuriy Gorodnichenko. 2012. "What can survey forecasts tell us about information rigidities?" Journal of Political Economy 120 (1): 116-159.

Croushore, Dean, and Tom Stark. 2001. "A real-time data set for macroeconomists." Journal of Econometrics 105 (1): 111-130.

Da, Zhi, Joseph Engelberg, and Pengjie Gao. 2014. "The sum of all FEARS." The Review of Financial Studies 28 (1): 1-32.

D'Agostino, Antonello, and Karl Whelan. 2008. "Federal Reserve Information during the Great Moderation." Journal of the European Economic Association 6 (2-3): 609-620.

Gilchrist, Simon, and Egon Zakrajsek. 2012. "Credit spreads and business cycle fluctuations." The American Economic Review 102 (4): 1692-1720.

Gurkaynak, Refet S., Brian P. Sack, and Eric T. Swanson. 2007. "Market-based measures of monetary policy expectations." Journal of Business & Economic Statistics 25 (2): 201-212.

Gurkaynak, Refet S., Brian Sack, and Eric Swanson. 2005. "The sensitivity of long-term interest rates to economic news: evidence and implications for macroeconomic models." The American Economic Review 95 (1): 425-436.

Lettau, Martin, and Sydney Ludvigson. 2001. "Consumption, Aggregate Wealth, and Expected Stock Returns." The Journal of Finance 56 (3): 815-849.

Loughran, Tim, and and Bill McDonald. 2011. "When a liability is not a liability? Textual analysis, dictionaries, and 10-Ks." The Journal of Finance 66 (1): 35-65.

Newey, Whitney K., and Kenneth D. West. 1994. "Automatic lag selection in covariance matrix estimation." The Review of Economic Studies 61 (4): 631-653.

Pedregosa, F., G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, et al. 2012. "Scikit-learn: Machine Learning in Python." Journal of Machine Learning Research 2085-2830.

Reifschneider, David, and Peter Tulip. 2017. "Gauging the Uncertainty of the Economic Outlook Using Historical Forecasting Errors: The Federal Reserve's Approach." Finance and Economics Discussion Series 2017-020. Washington: Board of Governors of the Federal Reserve System 1-46.

Romer, Christina D., and David H. Romer. 2000. "Federal Reserve information and the behavior of interest rates." The American Economic Review 90: 429-457.

Rudebusch, Glenn D., and John C. Williams. 2009. "Forecasting recessions: the puzzle of the enduring power of the yield curve." Journal of Business & Economic Statistics 27 (4): 492-503.

Shapiro, Adam H., Moritz Sudhof, and Daniel J. Wilson. 2017. "Measuring News Sentiment." Federal Reserve Bank of San Francisco Working Paper 1-32.

Sinclair, Tara M., Fred Joutz, and Herman O. Stekler. 2010. "Can the Fed predict the state of the economy?" Economics Letters 108 (1): 28-32.Stock, James H., and Mark W. Waston. 2003. "Forecasting Output and Inflation: The Role of Asset Prices." Journal of Economic Literature 788-829.

Stock, James H., and Mark W. Watson. 2007. "Why has US inflation become harder to forecast?" Journal of Money, Credit and Banking 39 (s1): 3-33.Tetlock, Paul C. 2007. "Giving content to investor sentiment: The role of media in the stock market." The Journal of Finance 62 (3): 1139-1168.

Welch, Ivo, and Amit Goyal. 2008. "A Comprehensive Look at The Empirical Performance of Equity Premium Prediction." The Review of Financial Studies 21 (4): 1455-1508.Zarnowitz, Victor. 1985. "Rational Expectations and macroeconomic forecasts." Journal of Business & Economic Statistics 3 (4): 293-311.

Data Appendix:

In this appendix we provide methodology and source for constructing our dataset. For each set of variables - Tonality, Economic (outcome) variables, Federal funds rate variables, Forecast revisions, Monetary Policy announcement variables, Asset prices and Recession indicators we outline our methodology and source data.

1. Tonality Variables

All measures of Tonality are built using text of Greenbook Part 1 from 1973 to 2009. Of this text, we specifically use the Recent Developments and Outlook for Domestic Economic Activity portion. For the dictionary, we used the Harvard psycho-social dictionary as a base, but exclude words that have special meaning in an economic forecasting context, which leaves us with 231 positive and 102 negative words, which are listed below.

List of 231 positive words

assurance, confident, exuberant, joy, prominent, Satisfactory, unlimited, assure, constancy, facilitate, liberal, promise, Satisfy, upbeat, attain, constructive, faith, lucrative, prompt, Sound, upgrade, attractive, cooperate, favor, manageable, proper, Soundness, uplift, auspicious, coordinate, favorable, mediate, prosperity, Spectacular, upside, backing, credible, feasible, mend, rally, Stabilize, upward, befitting, decent, fervor, mindful, readily, Stable, valid, beneficial, definitive, filial, moderation, reassure, Stable, viable, beneficiary, deserve, flatter, onward, receptive, Steadiness, victorious, benefit, desirable, flourish, opportunity, reconcile, Steady, virtuous, benign, discern, fond, optimism, refine, Stimulate, vitality, better, distinction, foster, optimistic, reinstate, Stimulation, warm, bloom, distinguish, friendly, outrun, relaxation, Subscribe, welcome, bolster, durability, gain, outstanding, reliable, Succeed, boom, eager, generous, overcome, relief, Success, boost, earnest, genuine, paramount, relieve, Successful, bountiful, ease, good, particular, remarkable, Suffice, bright, easy, happy, patience, remarkably, Suit, buoyant, encourage, heal, patient, repair, Support, calm, encouragement, healthy, peaceful, rescue, Supportive, celebrate, endorse, helpful, persuasive, resolve, Surge, coherent, energetic, hope, pleasant, resolved, Surpass, comeback, engage, hopeful, please, respectable, Sweeten, comfort, enhance, hospitable, pleased, respite, Sympathetic, comfortable, enhancement, imperative, plentiful, restoration, Sympathy, commend, enjoy, impetus, plenty, restore, Synthesis, compensate, enrichment, impress, positive, revival, Temperate, composure, enthusiasm, impressive, potent, revive, Thorough, concession, enthusiastic, improve, precious, ripe, Tolerant, concur, envision, improvement, pretty, rosy, tranquil, conducive, excellent, inspire, progress, salutary, tremendous, confide, exuberance, irresistible, progressive, sanguine, undoubtedly

List of 102 negative words

adverse, dim, feeble, mishap, struggle, afflict, disappoint, feverish, negative, suffer, alarming, disappointment, fragile, nervousness, terrorism, apprehension, disaster, gloom, offensive, threat, apprehensive, discomfort, gloomy, painful, tragedy, awkward, discouragement, grim, paltry, tragic, bad, dismal, harsh, pessimistic, trouble, badly, disrupt, havoc, plague, turmoil, bitter, disruption, hit, plight, unattractive, bleak, dissatisfied, horrible, poor, undermine, bug, distort, hurt, recession, undesirable, burdensome, distortion, illegal, sank, uneasiness, corrosive, distress, insecurity, scandal, uneasy, danger, doldrums, insidious, scare, unfavorable, daunting, downbeat, instability, sequester, unforeseen, deadlock, emergency, interfere, sluggish, unprofitable, deficient, erode, jeopardize, slump, unrest, depress, fail, jeopardy, sour, violent, depression, failure, lack, sputter, war, destruction, fake, languish, stagnant, devastation, falter, loss, standstill

Tonality is the number of positive and negative words in a text using a tf-idf weighting scheme from the previous 40 Greenbooks normalized to have mean 0 and standard deviation 1.

Positivity and Negativity are the normalized number of positive and negative words respectively using the same tf-idf weighting as Tonality.

Trend versions of Tonality variables are the exponentially weighted moving averages (EWMA) of the normalized Tonality variables with the weighting parameter chosen to maximize fit. The trend measure is fitted over two periods divided at the beginning of 1980, when the frequency of observations changes from 12 to 8 times a year. They are then appended together.

Tonality Shock is equal to Tonality variable - Trend variable.

2. Economic Variables

Historical realized values

The realized values ("actuals") for the economic indicators are real gross domestic product (RGDP), unemployment and inflation as gauged by the consumer price index (CPI) are drawn from the Philadelphia Fed's real-time data set (Croushore and Stark 2001). For GDP, we use the third monthly estimate ("first final") published by the BEA. For CPI and unemployment we use the initial monthly release values, compiled into the quarterly values. We transform the real time data vintages as RGDP growth, CPI growth, and change in unemployment rate. Fed staff forecasted GNP instead of GDP till 1990 and GNP deflator instead of CPI until 1980, hence we use GNP growth and GNP deflator growth accordingly.

The base value for the GDP growth rate is the GDP from the previous quarter at the time of the publication of the Greenbook. Act_RGDP_-1 is the value of RGDP from the previous quarter and RGDP_i is the value of RGDP i quarters into the future. We then compute the i quarters ahead cumulative GDP growth as following:

Act_RGDP_growth_i = 100 * ((RGDP_i / RGDP_-1)- 1)

Similarly, the unemployment change, we use the quarter prior to the Greenbook publication as base value. Act_Unemployment_-1 is the value of Unemployment from the previous quarter and Unemployment_i is the value of Unemployment i quarters into the future. We then compute the i quarters ahead unemployment change as following:

Act_Unemployment_change_i = Unemployment_i - Unemployment_-1

Growth in CPI is instead calculated using the contemporaneous CPI. Act_CPI₀ is the value of CPI from the current quarter and CPI_i is the value of CPI i quarters into the future. We then compute the i quarters ahead cumulative GPI growth as following:

Act_CPI_growth_i = 100 * ((Act_CPI_i / Act_CPI₀)- 1)

Staff Forecasts

All data for staff forecasts of RGDP, unemployment and CPI are from the Greenbook forecast dataset published by Federal Reserve Bank of Philadelphia. We use the forecasts for the previous quarter through four quarters ahead. Forecasts are aligned by the quarter to which the Greenbook is released. With the exception of unemployment rate, data is reported as annualized quarter over quarter percent growth, which we convert to quarterly growth before calculating cumulative growth rates.

Staff_RDGP₀ is the staff's projection of the growth from the previous quarter to the current quarter of RGDP. Staff_RGDP_i is equal to the projected Q/Q growth i quarters into the future. We then compute the i quarters ahead cumulative GDP growth as following:

Staff_RGDP_growth_i = $$ \prod^i_{k=0}{{Staff\_RGDP}_k}$$

Staff_Unemployment_-1 is the staff's projection for the unemployment rate in the previous quarter and Staff_Unemployment_i is equal to the staff's projection for the unemployment rate i quarters ahead. We then compute the i quarters ahead unemployment change as following:

Staff_Unemployment_change_i = Staff_Unemployment_i - Staff_Unemployment_-1

Staff_CPI₀ is the staff's projection for the change in CPI from the previous quarter to the current quarter. Staff_CPI_i is equal to the projected Q/Q growth i quarters into the future. We then compute the i quarters ahead cumulative CPI growth as following:

Staff_CPI_growth_i = $$ \prod^i_{k=1}{{Staff\_CPI}_k}$$

Blue Chip Forecasts

The Blue Chip forecasts for RGDP, unemployment and CPI are from the consensus estimates from the Blue Chip Economic Indicators publication from 1990 until 2009. The forecast periods are aligned by the month of the Blue Chip public release. In order to match Blue Chip forecasts to Greenbook release dates, the 15th of the month is used as a cutoff. If the Greenbook release date is on or before the 15th of the month, the Blue Chip forecast will be from the same month. In the other case, the next month's Blue Chip forecast will be used. In the event the next month is also the next quarter, one less forecast period is used in order to preserve a constant forecast quarter. After making this adjustment, Blue Chip growth and change variables are constructed in analogous fashion to the variables for the staff forecast.

BC_RGDP_growth_i = $$ \prod^i_{k=0}{{BC\_RGDP}_k}$$

BC_Unemployment_change_i = BC_Unemployment_i - BC_Unemployment_-1

BC_CPI_growth_i = $$ \prod^i_{k=1}{{BC\_CPI}_k}$$

3. Federal Fund Rate Variables

Actuals

Until December 16th 2008, we use the target Fed funds rate. Thereafter we use the midpoint of the upper and lower range of the target Federal funds rate. Since the forecasts predict the average rate, we use the average target rate over the entire quarter.

Act_FedFunds_-1 is equal to the average Fed funds rate in the previous quarter. Act_FedFunds_iis the average rate i quarters into the future. We define the change in Fed funds rate as follows:

Act_FedFunds_change_i = Act_FedFunds_i - Act_FedFunds_-1

Staff Forecasts

Staff projections for the Fed funds rate are from the financial assumptions dataset maintained by the Philadelphia Fed from 1990-2009.

Staff_FedFunds_-1 is equal to the staff's forecast for the previous quarter. Staff_FedFunds_i is equal to the staff's forecast i quarters into the future. We define the change in the Fed funds rate as follows:

Staff_FedFunds_change_i = Staff_FedFunds_i - Staff_FedFunds_-1

Blue Chip Forecast

Blue Chip projections for the Fed funds rate are the consensus estimates from the Blue Chip Financial Forecasts publication from 1990 until 2009. As with economic indicator variables, the Blue Chip forecast is matched to the current Greenbook based on whether or not the Greenbook release date was on or before the 15th of the month. We define the Blue Chip Fed funds variables in the same manner as the staff variables.

BC_FedFunds_change_i = BC_FedFunds_i - BC_FedFunds_-1

Term Spreads

The term spreads variables are calculated following (Gurkaynak, Sack and Swanson 2007) as the spread between Fed funds futures contract rates and the current Fed funds rate. The Fed funds future rate is chosen to align most closely with the forecast period given the availability of the contract. In the case of 1-quarter ahead, the 3-month contract is used and for the 2- and 4- quarter ahead forecasts, the 6-month contract is used. The spread is taken at the Greenbook release date when matched with the Staff Forecast and as close to the 23rd of the month when matched with the Blue Chip Forecast. In the case of Blue Chip Forecasts, the term spread is taken from the month prior when the Greenbook release date is on or before the 15th of the month. In case of the first month of each quarter, the future contract with an additional month of expiration is used to ensure it falls within the target quarter.

FedFutures_i is equal to the Fed funds futures contract aligned with the forecast i quarters into the future. Curr_FedFunds is the current Fed funds rate. We define the term spread as follows:

Term_Spread_i = FedFutures_i - Curr_FedFunds

4. Revisions

We create revision variables for both the Staff and Blue Chip forecasts. Revisions are defined as the difference between the current forecast and the previous forecast for the same period. In the case that the Greenbook release date is in the first month of the quarter, the forecast from the period before will use one additional forecast period in order to maintain the quarterly alignment. For example, in January the revision for a 1-quarter ahead forecast will be calculated as the current 1-quarter ahead forecast minus the December meeting's 2-quarter ahead forecast. We define the revision for the i quarter ahead projection at meeting t as follows:

Revision_t,i = Forecast_t,i - Forecast_t-1,i

5. Monetary Policy Announcement Variables

All announcement variables are calculated as the difference between the market quotes 20 minutes after the FOMC announcement and 10 minutes before the FOMC. The data used spans 1994-2009 and includes fed funds futures contracts expiring closest to 1-, 2- and 4- meetings ahead, 6-month, 2-year and 5-year treasury yields and the S&P 500 return.

FedFutures₁ is the equal to the Fed Funds futures 1 meeting ahead and is always the 2-month future.

FedFutures₂ is the equal to the Fed Funds futures 2 meetings ahead and is either the 3-month future or the 4-month future depending on which would expire closest to the meeting.

FedFutures₄ is the equal to the Fed Funds futures 3 meetings ahead and is always the 6-month future.

Policy Shock is equal to the change in the Target Federal Funds rate as a result of the FOMC decision.

6. Asset Price Variables

We calculate return on the S&P 500 index as the return between the current and previous Greenbooks. We also calculate the return on S&P 500 from the closing price on day of current meeting to 2- and 4- meetings ahead.

SPret_i,j is equal to the return of the S&P 500 from the ith to the jth Greenbook.

Short_Rate is the 3-month Treasury bill yield

Div_Yield is the ratio of the 12-month trailing dividends in the month of the Greenbook and the S&P 500 index value in the prior month

CAY is the consumption, wealth, income ratio of the previous quarter as defined by (Lettau and Ludvigson 2001).

7. Recession Variables

Recessions are defined using the NBER's recession dates. All data uses the full sample of 1973-2009.

Recession_iis a dummy equal to 1 if the United States is in a recession i months ahead

Prev_Recession is a dummy equal to 1 if the United States was in a recession 2 months ago.

Term_Spread_Recession is equal to the difference between the 10-year treasury yield and the 1-year treasury yield.

GZ_Spread is defined in Gilchrist and Zakrajsek (2012).

Footnotes

* All authors are at the Federal Reserve Board, 20th Street and Constitution Avenue, NW, Washington DC 20551. Sharpe (Steve.A.Sharpe@frb.gov) is a Deputy Associate Director, Sinha (Nitish.R.Sinha@frb.gov) is a Senior Economist at the Federal Reserve Board and Hollrah (Christopher.A.Hollrah@frb.gov) is a Senior Research assistant. Our views do not necessarily reflect those of the Federal Reserve System or its Board of Governors. We are very grateful for the research assistance provided by Toby Hollis, Taryn Ohashi, and Stephen Paolillo. Also, many thanks to Jeremy Rudd for his help in developing the wordlists. Return to Text

1. For example, Romer and Romer (2000) show the Federal Reserve Greenbook forecasts are superior to private sector forecasts. D'Agostino and Whelan (2008) and Sinclair, Joutz and Stekler (2010) note that the superiority of Fed's forecast has faded recently. Return to Text

2. For the list of positive and negative words, see the data appendix. Return to Text

3. Tetlock (2007) used Harvard-Psychosocial dictionary to quantify the sentiment in financial news. Da, Engelberg and Gao (2014) use Google searches on select words from this dictionary to quantify fear among U.S. investors. Return to Text

4. In the information retrieval and text analysis literature the tf-idf weighing scheme is a commonly used metric to gauge the importance of a word in a collection of documents (or a corpus). Loughran and McDonald (2011) first used tf-idf weight in the finance literature to quantify SEC filings by U.S. firms. Return to Text

5. Revisions are measured as changes to the outlook only 3 quarters out. For most observations, constructing revisions to the 4-quarter outlook would require having the lagged value of the 5-quarter outlook, which is frequently unavailable. Return to Text

6. If we were to incorporate a second break as indicated by the Bai-Perron test, the two later sub periods (September 1990 to December 2000 and after December 2000) would be qualitatively similar, differing from each other mostly by size of the negative effect of the unemployment rate outlook on Tonality. Return to Text

7. We focus only on the post-1990 period (the "late sample") for two reasons: first, this is where Tonality was found to have robust predictive power for economic variables, and second, because Greenbook and Blue Chip forecasts of the Fed funds rate are only available beginning in 1983 and 1984, respectively. Return to Text

8. For the 1- and 2-quarter-ahead funds rate forecast, market expectations should be well approximated using the futures contract maturing 3 and 6 months ahead, respectively. For the 4-quarter-ahead forecast, we again use the 6-month ahead futures rate, the latest-maturing reliably-traded contract. Alternatively, using the 4-quarter-ahead Eurodollar futures rate for the latter does not materially change the results. Return to Text

9. In unreported regression, we examine whether the information reflected in Tonality "leaks" to the public between the time the Greenbook is finalized and the time of the FOMC announcement by regressing the pre-FOMC drift on Tonality. The pre-FOMC drift is the return on S&P index in the 24 hour window before the FOMC announcement. We find that the monetary policy surprise does not help explain pre-FOMC drift in stock prices; at the same time, Tonality is found to have a negative effect which is significant at the 1 percent level and explains about 6.5 percent of the variation in the pre-FOMC drift. At the same time, neither the monetary policy surprise nor the Tonality help to explain changes in Treasury yields during the pre-FOMC drift period. Return to Text

10. The cay data at quarterly frequency is downloaded from Martin Lettau's website. Return to Text

11. In absence of the signal T_t , an investor will invest an amount inversely proportional to the risk $$ (\sigma_T^2 + \sigma_e^2)$$. That investor's expected return on the portfolio will be $$ (1/\gamma)*S^2 + r_f$$, where, again, S is the risky asset's Sharpe ratio. With the knowledge of signal T_t , the investor's risk declines to simply $$ \sigma_e^2$$ and the investor buys more or less depending on the signal from T_t about the expected excess return $$ (\mu + T_{t} - r_{f} )$$. The expected return for the investor with real-time knowledge of Tonality is $$ (1/\gamma)* (S^2 + R^2)/ (1 - R^2) + r_{f}$$. Return to Text

^♣ This version is optimized for use by screen readers. Descriptions for all mathematical expressions are provided in LaTex format. Return to Text