No meaningful thresholds in the CESI

Update on May 17 — Welcome ISI readers. I can tell from the bump in readership that Dennis must have mentioned this piece. Thanks, D.

I document that moves of the CESI from significantly below zero to above zero have not led predictably to equity market weakness. Today, you will care about the symmetrical question of whether meaningful declines below zero lead.  There is no evidence here on that, but it is indeed symmetrical, logically and on the evidence, such as it is.  I don’t have a strong view on market here, fwiw. Been agnostic going on a couple years now. But that is separate.

—————————–

Yesterday I mentioned that I would take a critical look at the Citi Economic Surprise Index (CESI), which has recently “broken” above its neutral line (of zero) and thereby attracted some attention from journalists and analysts.

I follow up on that today. To avoid any suspense, my main point here is that there is no logical reason to believe that the CESI should lead the equity market or anything else for that matter.  Moreover, the historical record shows that meaningful rises of the surprise index have tended (marginally) to be followed by negative to below-average returns, at least on the somewhat arbitrary (but not data mined) test I chose to run.

I assume that record is mostly a fluke. Statistically, it is not significant, and I mention it only to lean into the commonly-expressed view that the Citi index somehow leads.  So far as I can tell, the CESI is pretty much pointless, except as a crude summary description of the recent past.  That may be helpful in some circumstances, but it seems very misleading to imply that there is some sort of secret sauce implicit in the index’s time series properties.  The Citi index does have time series properties, but they are a function of its construction and predict nothing about the underlying data surprise process.

Before turning to that, though, I want to be clear that the point of this post is not to be downbeat on the market. I shifted effectively to agnostic on the market a couple years ago, because the visible or compelling part of the rally seemed to have passed and yet there was no evidence that equities were necessarily overvalued.  Instead, they seemed roughly fairly valued to deliver the subpar returns implied by slow growth, low risk-free yields and a reduced equity premium associated with a low-vol economic environment. Some have called this world the Not So Great Moderation.

In the event, my shift to agnostic was quite premature, as returns have been closer to par than subpar, especially over the past year. As the propagandists at CNBC tell us, “the Trump rally roars on!” But for lack of a better view, I am sticking with my boring, agnostic view.  There is nothing worse than knowing your neighbours are getting rich!  I don’t want to compound the pain by confusing being pissed and confused with being bearish.

Ok, so getting down to business here, I have a couple minor logical quibbles with the Citi index, even as a backward-looking measure of economic surprise. But the main problem with the CESI index is not so much how it is built as how it is interpreted.  It may be roughly what it purports to be.  But it certainly does not do what people claim it does.  To show why, let’s start with how it is built, go to some of the weaknesses in its construction, and then go over how it is abused, before concluding on some evidence.

The CESI is a measure of whether the growth and inflation data have collectively been surprising to the upside or downside over a 3-month rolling window.  To build the index, analysts at Citi start by measuring the extent to which select individual data releases have been beating or missing the street-economist consensus.   The individual beats or misses are measured in terms of historical standard deviations and then weighted according to each release’s typical instantaneous impact on dollar exchange rates.  The CESI is then computed as a 3-month weighted moving average of the individual surprises, where the time weights follow a geometric decay.

There are a few things to note about this approach:

  • The index blends growth and inflation indicators, which is appropriate if the purpose is to measure the aggregate influence on monetary policy and through that channel exchange rates. But the index does not measure just economic growth surprise. It is an amalgam, in contrast with an otherwise-largely-similar index produced by Goldman.
  • If Wall Street economists are unbiased and not terribly unlucky, then the index should be strongly mean-reverting and have an average of zero. Empirically this is the case. Since 2003, the index has passed through zero more than 100 times. *  Its average value over this period has been -1 against a standard deviation of 40. In other words, the average has not been significantly different from zero.
  • It is difficult to know with certainty if a rising (or falling) value reflects the data recently beating (or missing) or mere base effects. For example, the index could easily rise from -100 to -50, simply in response to the recent data missing by less. And it could make the same move in response to the data beating. Absent access to the underlying daily observations, it is impossible to know.

Subject to those caveats, I don’t hate how this index is built.  If it has risen from -100 to +100, then we can be reasonably confident that the growth data have gone from missing to beating.  What is much more irksome, to me anyway, is the way the index is interpreted by analysts and journalists who radically overstate its importance and imply that the index has useful time series properties.

There are three particularly bone-headed claims about the index that are made from time to time.  The first you might hear from “contrarians” who might point out that the index has made an extreme and cannot go much higher or lower.  For example, when the index is at +150, somebody will invariably say that its next major move has to be down.  The implication is often that equity prices (or bond yields) will follow.

The irksome thing about such claims is that they are both true and highly misleading.  As mentioned, the index is mean reverting by construction.  Your best guess for the index at any point more than three months from now should always be that it will be zero, even if you know nothing about the nature of the coming data surprises, positive or negative.

So yeah, if it is high, then it is likely to fall. And if it is low, then it is likely to rise. Given that the truth of that implies nothing about upcoming data surprise, it is irrelevant to the market.  This mistake has not been common recently, because the index has not been near an extreme. But I point it out for completeness. You may notice some guy falling for it in the future.

A closely related claim you occasionally here is that the index has made a clear bottom or top and is likely to continue its move towards zero – and so get bearish or bullish, etc.  This claim is made by people who resemble the mean reverters, but like to see some “confirmation.” It is like they are trying to applying technical analysis – in an inappropriate context.

The final claim, which we have heard recently, is that a move through zero is important because once the index goes from negative to positive it tends to stay in the new regime for a while.  This claim has the unique disadvantage of being both false and trivial. As you can see from the chart below, whipsaws in the index are not the least uncommon.  Moreover, big movements in the index far above or far below zero tend to persist simply because the index is built as a moving average of underlying (daily) data surprise. **

screen-shot-2016-12-08-at-11-52-45-am

Back when I was back at the Mother Corp, I remember occasionally leaning into claims that the CESI had broken down below zero and that that was bearish. I remember dutifully testing and finding no evidence that breakdowns were followed by unusual equity market weakness or lack of strength.  Now, of course, with the equity market at a record high, as the “Trump rally roars on”, you are more often to hear that a breakout is bullish.

To test this, we could just measure what results in the equity market from all movements of the CESI from negative into positive territory. But that would be awkward because there have been so many moves through zero and – more to the point – because the people making claims for the CESI tend to get excited when the index has moved from significantly below zero to above zero. So we need to narrow down the list by arbitrarily defining what “significantly” below zero might mean.  You can choose any number you like. I chose one: -50.

Since 2003, the CESI has risen from -50 to above zero 12 times.  Three month returns in the wake of such moves have actually been negative, which is a minor accomplishment, given that the average three-month return since 2003 has been 1.7%. Six month returns have been positive, although less than average. So empirically, there is no evidence this thing does anything helpful, at least as I measure it.

screen-shot-2016-12-08-at-11-36-20-am

Last-episode six month price return is estimated.

And why should it?  Data surprise does not predict data surprise, despite the false impression of that provided by a naïve reading of an economic surprise index whose time series “properties” are driven entirely by base effects.  Predicting data surprise is very hard, and it would be made that much harder if you were to start with the ridiculous notion that the past to present values of the Citi index tell you anything remotely interesting about upcoming underlying data surprise.

* I choose my words carefully there.  The exact number of crosses is 103, as I count them. But sometimes the thing lands at exactly zero, which means I have to use compound IF statements, which means in turn that there is a high risk of a Rogaine and Braveheart style coding error.  The astute among you might want to check my work. But yeah, it went through zero “more than” a hundred times. I am pretty sure about that.

** This raises an interesting question.  Is there some sense in which underlying daily data surprise predicts itself.  For example do five days of generally positive surprise predict a sixth? To test this, we would need a measure of underlying daily data surprise, not the output of an algorithm taking a weighted 3-month moving average.  I am skeptical, but could be convinced by evidence. Nobody making claims for the CESI ever offers any of that. And nor could they.  Goldman makes their underlying daily data surprise available to clients. Somebody might want to test that. What I attack here are the claims made for the CESI, most of which are absurd. Perhaps I sound like a shill for Goldman. I do believe they do the best macro research.  But more to the point, I don’t like being snowed.