Archive

Tag Archives: time series predictability

In terms of the main issue with Cochrane’s(2008) paper, we are asking : Which (and how much) of dividend growth or returns is forecastable. Given the relationship between [a] returns,[b]dividend growth and [c] dividend yields, a null hypothesis in which returns are not forecastable must also account for dividend growth that is forecastable.Hence the need to evaluate the joint distribution of return and dividend growth coefficients in simulated datasets.

So far there are three tests with respect to return predictability :

  1. the one-period return regression coefficient b(r)
  2. the one-period dividend growth regression coefficient b(d)
  3. the long-horizon return regression coefficient (b_lrun) [provides same results as long-horizon dividend growth regression but is more convenient to look at]

In terms of the results for the single period case :

  • The return forecasting coefficient,taken alone, is not significant.Evidence against the null is weak since only about 19% of simulated return forecasts exceed the sample counterpart. and only around 11% of simulated t-statistics are more extreme than that of the sample.
  • Absence of dividend growth forecastability offers much more evidence against the null than the presence of return forecastability,lowering probability values to the single digit range.

In terms of the long horizon case :

  • The long horizon return and dividend growth forecasts generate coefficients that are larger than their sample counterparts only 1-2% of the time (1.43% in my case) complementing the single period dividend growth forecast results.

 


 

[6] Once we have summarised both single-period and long horizon results ,we can look at the source of power regarding single-period dividend growth and long horizon tests, as well as examine the effect of varying dividend yield autocorrelation on short- and long run percentage values.

To investigate the source of the greater power afforded the [a] single period dividend growth estimates and [b] long horizon estimates,Cochrane (2008)  plots the joint distribution of return regression coefficients b(r) and dividend yield regression coefficient φ. I have also included the small sample distribution for the long horizon return coefficient estimates for the case where φ=0.941 and φ=0.99,the sample and fixed cases respectively.A simple call to the JointPlot() function with the fourth parameter set to ‘long horizon’ suffices :

#Long horizon results
windows()
par(mfrow=c(2,2))
JointPlot(mcs.sample,5000,initial.reg,'long horizon')
JointPlot(mcs.fixed,5000,initial.reg,'long horizon')

ll

Resulting in the following set of plots :

hj

Focusing on the joint distribution for which φ =0.941 :

  1. The triangle marks the null hypothesis and the circle marks the estimated coefficients
  2. The golden line marks the line  br = 1 − ρφ + bd_smpl. Points above and to the right are samples in which bd exceeds its sample value
  3. The black line is defined by br /(1 − ρφ) = br_smpl /(1 − ρ φ_smpl).Points above and to the right are draws in which b(r)_lrun exceeds the sample value.
  4. Percentages are probabilities of belonging to the associated quadrant.
  5. A high return coefficient by itself is not uncommon (just under 20% of the time).
  6. b(r) and φ estimates are negatively correlated across samples
  7. We never see b(r) larger than in the data ALONG with φ larger than in the data.The top right quadrant is empty.

These plots basically capture why single period dividend growth tests and long-horizon regression results give so many fewer rejections under the null than the single-period return test.While the single period return test falls in the 10-20% range probability values,the single-period dividend growth and the long-run tests fall in the 1-2% range. It is the strong negative correlation of estimates b(r) and φ across simulated samples that underlies the power of long-horizon return and dividend-growth tests.

The final part of this post shall address the effects of varying φ (the dividend yield autocorrelation) on the percentage values of simulated estimates being more extreme than their sample counterparts. A quick glance at the joint distribution plots above clearly suggest that as φ  rises from 0.941 to 0.99,the cloud of points rises and more of them cross the sample estimates of [a] the single period dividend growth variable and [b] the long horizon return variable.

In the following code we specify a vector of phis (from 0.9 to 1.01),calculate the half life of each member of that vector,create a list object to hold our monte carlo simulation results for each value of phi and save the resulting list in another .Rdata file.I only run 5000 trials to save time.As before, the MonteCarlo function() returns a list object with two elements,the first being a summary of the inputs and the second being a collection of simulation results. The mcs.list variable below only stores the second element.

 

#######Varying phi###################################################
# Vector of phis
phi.vec <- c(0.9,0.941,0.96,0.98,0.99,1.00,1.01)
half.life <- log(0.5)/log(phi.vec)
mcs.list<-list()
numT.temp <- 5000

#Save data [run once]
for(i in 1:length(phi.vec)){
	mcs.list[[i]] <- MonteCarloSimulation(num.trials=numT.temp,num.obs=100,phi=phi.vec[i],rho=0.9638,error.cov=res.cov)[[2]]
}
save(mcs.list,file='Dog2.Rdata')
#####################################################################

ll

Now that we have saved a list object with 7 elements,each corresponding to the monte carlo results of a particular value of φ ,let us load the data,and apply to each element of that list a custom function that [a] extracts coefficients/estimates whose values exceed their sample counterparts and [b] express these as a fraction of the number of simulated samples (5000).The TableMaker() function once again is used to tabulate the results.

 

################Table of phis########################################
#Load and Plot
load('Dog2.Rdata')

p.br <-unlist(lapply(mcs.list,function(g)length(which(g[,'b[r]']>betas[4]))/numT.temp*100))
p.bd <- unlist(lapply(mcs.list,function(g)length(which(g[,'b[d]']>betas[5]))/numT.temp*100))
p.lbr <- unlist(lapply(mcs.list,function(g)length(which(g[,'b[long.r]']>lr.beta))/numT.temp*100))
p.lbd <- unlist(lapply(mcs.list,function(g)length(which(g[,'b[long.d]']>(lr.beta-1)))/numT.temp*100))

tbl <- matrix(c(phi.vec,p.br,p.bd,p.lbr,p.lbd,half.life),ncol=6)
colnames(tbl) <- c('phi','b[r]','b[d]','long b[r]','long b[d]','half life')

col.names <- colnames(tbl)
reg.table = tbl

est.tab <- round(reg.table,5)
est.tab <- apply(est.tab, 2, rev)
est.tab <- cbind(c(' '),est.tab)
TableMaker(row.h=1,est.tab,c('Effects of phi',col.names),strip=F,strip.col=c('green','blue'),col.cut=0.05,alpha=0.7,border.col='lightgrey',text.col='black',header.bcol='gold',header.tcol='black',title=c('The effects of dividend yield autocorrelation\n5000 Montecarlo trials per phi'))
#####################################################################

km

 

As φ rises :

  • The probability of the single-period return coefficient exceeding its sample value is little changed.
  • The probability of the single-period dividend growth coefficient exceeding its sample value increases quite strongly.
  • The probability values of long run coefficients are identical with respect to each other and similar to the single-period dividend growth case. The latter result corroborates the overlap between the two areas carved out by the coefficients in the joint distribution plot from before.
  • To 1.01, nothing dramatic happens to the coefficients but the half life which should be infinite (as per the paper) takes on a strange number in my case (not sure what i have done wrong here).

The most important point here is that in all cases,that is to say for all reasonable value of φ ,the single-period dividend-growth and long-run regression tests have a great deal more power than the single-period return regression in rejecting the notion that returns are unpredictable.

Continuing from the previous post,we now turn to the distribution of coefficients and t-statistics associated with the return and dividend growth regressions across simulated datasets. Following the outline at the beginning of this set of posts  :

[4] Once we have collected regression results across 20000 simulated datasets,we can look at the probability of observing b(r) and b(d) pairs as well as t-statistic pairs that are more extreme than original sample estimates.

Each Monte Carlo trial involves running the single period forecasting regressions of the VAR. I use the custom JoinPlot() function to plot the joint distributions of return and dividend growth coefficients as well as respective t-statistics. Implementation is quite simple :

#Single Period results
windows()
par(mfrow=c(2,2))
JointPlot(mcs.sample,5000,initial.reg,'single period')
JointPlot(mcs.fixed,5000,initial.reg,'single period')

While the he mcs.sample captures simulated results using original sample estimates (1927-2004),the mcs.fixed case uses phi=0.99 as per the paper. This yields the following set of plots,with the first row showing the joint distributions for return/div growth coefficients and t-stats using original sample estimates in the simulation, and the second row capturing the same information but for the fixed case of phi=0.99 in the simulation :

fftAlthough we have simulated 20000 datasets,i only plot the results for 5000 simulations (for clarity and to keep the optician at bay). The black lines which meet at the red circle represent original sample estimates.The blue circles represent simulated results.The percentages in each quadrant capture the portion of simulated results more extreme than sample estimates. The green triangle marks the location of the null hypothesis (e.g. b(r)=0 and b(d) =-0.1 in the case of the original sample).

The original null hypothesis is that returns are unpredictable and that dividend growth is predictable,or more compactly that b(r)=0 and b(d)=-0.1 as per the identity. The null hypothesis is meant to help us answer the question : which of and how much of each of return and dividend growth is forecastable. As far as I understand it (and correct me if i am wrong),evidence against the null hypothesis should consist of the following (accounting for statistical significance) :

  1. return coefficient from simulated data > return coefficient from sample data.
  2. dividend growth coefficient from simulated data > dividend growth coefficient from sample data.

In the light of the above and the joint distribution plots for the case of our original sample (first row of plots) :

  1. If we ONLY look at the marginal distribution of the simulated return coefficients (br) we would find weak evidence against the null hypothesis since the monte carlo draw produces a coefficient larger than the sample estimate around 19% of the time (top right + bottom right quadrant).Coefficients larger the original sample estimate are to be seen even if the true underlying coefficient is zero.
  2. If we look at the joint distribution,most of the events with more extreme return coefficients are paired with dividend-growth forecast coefficients that are more negative than seen in the data (bottom right quadrant). In these events,both dividend growth AND returns are forecastable. Here prices are moving partially on forecasts of future dividend growth and in the right direction.
  3. Dividend growth fails to be forecastable in only 1.27% of the samples generated under the null (top right quadrant). The absence of dividend-growth forecastability offers stronger evidence against the null.

If I understand this correctly,the dividend-growth variable is the dog that did not bark, given that it is the absence of its forecastability that provides a stronger evidence against the null hypothesis than the presence of return forecastability.The absence of dividend growth forecastability can be thought of as evidence that return forecastability is economically significant.In other words,if we wanted to prove that returns are forecastable in the short run,the lack of dividend growth forecastability is more compelling than the presence of return forecastability for achieving that purpose.

 


[5] Once we have looked at the single period results,we can proceed with the long-horizon regression coefficients ( which are mercifully not new estimates, but rather calculated from the single period estimates of the original sample 1927-2004)

Long horizon estimates and tests provide a further avenue for delivering economically significant return forecasts.Conveniently,long horizon regression coefficients are not new estimates but rather derived from existing single-period results :

juu

Once the long-run return coefficient has been calculated,the corresponding dividend growth coefficient can be derived from :

kkl

These results reinforce the notion of economically significant return forecastability.The dividend yield volatility is almost exactly accounted for by return forecasts(b_longrun=1) with essentially no contribution from dividend-growth forecasts (b_longrun=0). In our case :

#Identitical div/lrun results
rho <- mcs.sample[[1]]['rho']
phi <- mcs.sample[[1]]['phi']
lr.beta <- betas[4]/(1-rho*phi)
ld.beta <- lr.beta-1
betas.lr <- c(lr.beta,ld.beta)

numT <- mcs.sample[[1]]['num.trials']
pr <- length(which(mcs.sample[[2]][,'b[long.r]']>lr.beta))/numT*100
pd <- length(which(mcs.sample[[2]][,'b[long.d]']>(ld.beta)))/numT*100
pvals.lr <- c(pr,pd)

#Table
col.names <- c('Long-run coeff','%p values')
row.names <- c('Return','Dividend growth')
reg.table <- cbind(betas.lr,pvals.lr)
est.tab <- apply(reg.table, 2, rev)
est.tab <- cbind(rev(row.names),est.tab)
windows()
TableMaker(row.h=1,est.tab,c('Results',col.names),strip=F,strip.col=c('green','blue'),col.cut=0.05,alpha=0.7,border.col='lightgrey',text.col='black',header.bcol='lightblue',header.tcol='black',title=c('Long horizon estimates'))

This results in the following table :

hhh

 

Although the t-statistics and standard errors are missing from this table (because i do not know how there are determined),they would,according to the paper, be exactly identical. Hence one advantage of using long-horizon regression coefficients is that they obviate the need to choose between return and dividend growth tests (as was the case in single period regressions). The last column shows our conventional probability values,essentially telling us : how many forecasts are greater than the sample value under the unforecastable null hypothesis.There is a 1.43% chance of seeing a long-run forecast more extreme than seen in the data. Ultimately,the long-run return AND dividend growth regressions give essentially the same strong rejections as the short-run dividend-growth regression. As a consequence, we can tabulate the small-sample distribution of the test in a conventional histogram rather than a two dimensional plot.

#histogram
windows()
par(mfrow=c(1,2))
lr<-hist(plot=FALSE,mcs.sample[[2]][,'b[long.r]'],breaks=100)
	  plot(xlim=c(-3,3),lr,freq=FALSE,col=ifelse(lr$breaks<=lr.beta,'lightgrey','red'),ylab='',xlab='long_b[r]',cex.main=0.85,cex.lab=0.75,cex.axis=0.75,cex=0.65,main=paste('phi=',phi))
	  abline(v=lr.beta,col='red')
	  text(x=2,y=0.85,paste('Data:',round(lr.beta,3),'\nProb :',pr),col='black',cex=0.6)

ld<-hist(plot=FALSE,mcs.sample[[2]][,'b[long.d]'],breaks=100)
	  plot(xlim=c(-3,3),ld,freq=FALSE,col=ifelse(ld$breaks<=(lr.beta-1),'lightgrey','red'),ylab='',xlab='long_b[d]',cex.main=0.85,cex.lab=0.75,cex.axis=0.75,cex=0.65,main=paste('phi=',phi))
	  abline(v=(lr.beta-1),col='red')
	  text(x=1,y=0.85,paste('Data:',round((lr.beta-1),3),'\n Prob :',pd),col='black',cex=0.6)

 

I think this reinforces the point of long run estimates being similar :

mnj

In this post I will attempt to replicate some basic results from Cochrane’s(2008) paper ,-‘The Dog that did not bark : a defense of return predictability’ – ; as usual with the posts found on this blog,I do not claim to understand many of the nuances of the paper or the mathematics/statistical procedures behind it (in fact most of it is completely beyond me), it is nevertheless interesting to see how much one can tinker with and replicate using online resources for free.

The present value identity sketched in previous posts establishes the following relationship between [1] dividend yield ,[2] dividend growth and [3] return variables. If the state of the world were such that observed dividend yields are constant over time,then neither return nor dividend growth variables are predictable. Since the state of the world is such that observed dividend yields vary over time,it must be the case that at least one of dividend growth or return is/are predictable. Simple regressions that were run in previous posts establish several main findings (which i shall repeat lest i forget them again) :

  1. The dividend price ratio predicts returns but NOT dividend growth.
  2. Long-horizon return regressions provide stronger evidence of return predictability than single period counterparts.
  3. Returns which should not be forecastable are and dividend growth which should be forecastable isn’t.
  4. On average,price dividend ratios are moving in response to expected return news rather than cash flow news.
  5. Time varying discount rates explain virtually all the variance of dividend yield while dividend growth forecasts explain essentially none of it.

As has been repeated quite often in these posts on time series predictability: If returns are not predictable, dividend growth must be predictable, to generate the observed variation in dividend yields.The most basic (but nonetheless important) result of Cochrane’s paper that we are going to replicate is the finding that [1] the absence of dividend growth predictability gives stronger evidence than does the presence of return predictability and that [2] long-horizon return forecasts complement single period results. There will be some discrepancies between Cochrane and our results which are probably due to [1] coding errors on my part, [2] not using real (inflation adjusted) variables since they are a hassle to find online (…at least to me), and [3] the use of 20000 monte carlo trials instead of the 50000 used in the paper. Also,I wrote the code almost a year ago and have only come around to blogging about it now,hence many of the issues that may have cropped up at that time are oblivious to me at the moment. This part of the post simply outlines some of the intuitions and procedures behind Cochrane’s paper, as well as the dataset and custom functions used in the code. The subsequent post shall deal with the results obtained and perhaps compare them somewhat to those of the paper.

[Data]

The data used in the code comes from the Chicago Booth advanced investments seminar, which contains [1]Return, [2]Dividend-yield,[3] Dividend-growth and [4] T-bill rate variables. The data file I have goes from 1926 to 2011,the one from the linked site goes to 2012. We only need data from 1927 to 2004, so it doesn’t really matter.After converting the txt file to a csv file in Excel,let’s load it into R,transform some of the variables to the ones we need and subset the data to the date range used in the paper.
ll

#####################################################################################
# Load and extract annual data
#####################################################################################

#Import csv file
orig.csv  &lt;- read.csv('ps1_data.csv',header=T)
orig.ts &lt;- ts(data=orig.csv,start=c(1926),end=c(2011),frequency=1)
orig.extract &lt;- window(orig.ts,start=c(1926),end=c(2004),frequency=1)
orig.extract &lt;- cbind(orig.extract,orig.extract[,1]-orig.extract[,4],log(1+orig.extract[,1]),log(orig.extract[,2]),log(1+orig.extract[,3]))
colnames(orig.extract) &lt;- c('Idx.ret','Div.yield','Div.growth','Tbill.ret','Exc.ret','log.ret','log.div.yield','log.div.growth')

kkk

[Functions]

Functions include: SimulateVAR(),MonteCarloSimulation(),JointPlot(),RegExtractor(),TableMaker(),of which the last two are general helper functions to extract results from regression objects and display tabular information from matrices and such respectively. I don’t know why the code,when copied into wordpress,becomes oddly misaligned at times.
ll

#####################################################################################
#Extract Regression Values
#####################################################################################
RegExtractor &lt;- function(x,type)
{
 if(type=='est'){
  output &lt;- t(sapply(x,function(v)coef(summary(v))[,1]))
 }
 else if (type=='stderr'){
  output &lt;- t(sapply(x,function(v)coef(summary(v))[,2]))
 }
 else if(type=='tval'){
  output &lt;- t(sapply(x,function(v)coef(summary(v))[,3]))
 }
 else if(type=='pval'){
  output &lt;- t(sapply(x,function(v)coef(summary(v))[,4]))
 }
 else if(type=='rsq'){
  output &lt;- t(sapply(x,function(v)(summary(v))$r.squared))
 }
 else if(type=='res'){
  output &lt;- sapply(x,function(v)(resid(v)))
 }
return(output)
}
#####################################################################################


#####################################################################################
# Simulate the VAR
#####################################################################################

SimulateVAR &lt;- function(num.obs,phi,rho,error.cov){
 	dp.sd = round(sqrt(var(error.cov))[2,2],3)
	dg.sd = round(sqrt(var(error.cov))[1,1],2)
	dp.shock = rnorm(num.obs+1,mean=0,sd=dp.sd)
	dg.shock = rnorm(num.obs,mean=0,sd=dg.sd)
	sim.data = matrix(data=0,nrow=num.obs,ncol=3,dimnames=list(NULL,c('sim.divyield','sim.dgrowth','sim.returns')))

	ifelse(phi&lt;1,dp.beg&lt;-rnorm(1,mean=0,sd=dp.sd^2/(1-phi^2)),dp.beg&lt;-0)
  
  sim.data[1,'sim.divyield'] &lt;- phi*dp.beg+dp.shock[2]
  sim.data[1,'sim.dgrowth'] &lt;- ((rho*phi)-1)*dp.beg+dg.shock[1]
	
	for(i in 2:num.obs){
  	sim.data[i,'sim.divyield']&lt;- phi*sim.data[(i-1),'sim.divyield']+dp.shock[i+1]
  	sim.data[i,'sim.dgrowth'] &lt;- ((rho*phi)-1)*sim.data[(i-1),'sim.divyield']+dg.shock[i]
  }
 	sim.data[,'sim.returns'] &lt;- dg.shock-(rho*dp.shock[2:(num.obs+1)])
	return(data.frame(sim.data))
}
#####################################################################################


#####################################################################################
# MonteCarlo
#####################################################################################

MonteCarloSimulation &lt;-function(num.trials,num.obs,phi,rho,error.cov){
	coeff.names &lt;- c('b[r]','b[d]','phi')
	montecarlo.results &lt;- matrix(data=0,nrow=num.trials,ncol=8,dimnames=list(NULL,c(coeff.names,paste(coeff.names,'_tstat',sep=''),c('b[long.r]','b[long.d]'))))
  inputs &lt;- c(num.trials=num.trials,num.obs=num.obs,phi=phi,rho=rho,error.cov=error.cov)

	for(i in 1:num.trials){
		simulated.data &lt;- SimulateVAR(num.obs,phi,rho,error.cov)
		sim.reg &lt;- list(
			dyn$lm(sim.returns~lag(sim.divyield,-1),data=ts(simulated.data)),
			dyn$lm(sim.dgrowth~lag(sim.divyield,-1),data=ts(simulated.data)),
			dyn$lm(sim.divyield~lag(sim.divyield,-1),data=ts(simulated.data))
		)
	  montecarlo.results[i,1:3] &lt;- (RegExtractor(sim.reg,'est')[,2])
		montecarlo.results[i,4:6] &lt;- (RegExtractor(sim.reg,'tval')[,2])
  	montecarlo.results[i,7] &lt;- montecarlo.results[i,1]/(1-rho*montecarlo.results[i,3])
  	montecarlo.results[i,8] &lt;- montecarlo.results[i,7]-1
		
		print(paste('Monte Carlo Simulation :',i))
	}
return(list(inputs,montecarlo.results))
}
#####################################################################################


######################################################################################
# Joint distribution plot
######################################################################################

JointPlot &lt;- function(mc.object,num.points,initial.reg,type){
	
  #Define variables
	betas &lt;- matrix(RegExtractor(initial.reg,'est')[,2],ncol=1)
  tvals &lt;- matrix(RegExtractor(initial.reg,'tval')[,2],ncol=1)
  num.trials &lt;- mc.object[[1]]['num.trials']
	rho &lt;- mc.object[[1]]['rho']
	phi &lt;- betas[6]
  lr.beta &lt;- betas[4]/(1-rho*phi)
	
	if(type=='single period'){
		
		bottom.right &lt;- length(mc.object[[2]][which(mc.object[[2]][,'b[r]']&gt;betas[4] &amp; mc.object[[2]][,'b[d]']&lt;betas[5],)])/num.trials*100
	  top.right &lt;- length(mc.object[[2]][which(mc.object[[2]][,'b[r]']&gt;betas[4] &amp; mc.object[[2]][,'b[d]']&gt;betas[5],)])/num.trials*100
	  bottom.left &lt;- length(mc.object[[2]][which(mc.object[[2]][,'b[r]']&lt;betas[4] &amp; mc.object[[2]][,'b[d]']&lt;betas[5],)])/num.trials*100
	  top.left &lt;- length(mc.object[[2]][which(mc.object[[2]][,'b[r]']&lt;betas[4] &amp; mc.object[[2]][,'b[d]']&gt;betas[5],)])/num.trials*100
	
		plot(ylim=c(min(mc.object[[2]][1:num.points,'b[d]'])-0.15,max(mc.object[[2]][1:num.points,'b[d]'])+0.15),xlim=c(min(mc.object[[2]][1:num.points,'b[r]'])-0.15,max(mc.object[[2]][1:num.points,'b[r]'])+0.15),x=mc.object[[2]][1:num.points,'b[r]'],y=mc.object[[2]][1:num.points,'b[d]'],ylab='b[d]',xlab='b[r]',cex.main=0.85,cex.lab=0.75,cex.axis=0.75,cex=0.65,main=paste('Coefficients\nphi=',mc.object[[1]]['phi']),col='blue',pch=21)
  	abline(h=betas[5])
  	abline(v=betas[4])
		points(x=betas[4],y=betas[5],col='red',cex=1,pch=19)
		points(x=0,y=mc.object[[1]]['rho']*mc.object[[1]]['phi']-1+0,cex=1,pch=17,col='green')
	
		text(x=max(mc.object[[2]][1:num.points,'b[r]']),y=min(mc.object[[2]][1:num.points,'b[d]'])-0.1,paste(bottom.right,'%'),cex=0.75)
		text(x=max(mc.object[[2]][1:num.points,'b[r]']),y=max(mc.object[[2]][1:num.points,'b[d]'])+0.1,paste(top.right,'%'),cex=0.75)
		text(x=min(mc.object[[2]][1:num.points,'b[r]']),y=min(mc.object[[2]][1:num.points,'b[d]'])-0.1,paste(bottom.left,'%'),cex=0.75)
		text(x=min(mc.object[[2]][1:num.points,'b[r]']),y=max(mc.object[[2]][1:num.points,'b[d]'])+0.1,paste(top.left,'%'),cex=0.75)

		bottom.right &lt;- length(mc.object[[2]][which(mc.object[[2]][,'b[r]_tstat']&gt;tvals[4] &amp; mc.object[[2]][,'b[d]_tstat']&lt;tvals[5],)])/num.trials*100
		top.right &lt;- length(mc.object[[2]][which(mc.object[[2]][,'b[r]_tstat']&gt;tvals[4] &amp; mc.object[[2]][,'b[d]_tstat']&gt;tvals[5],)])/num.trials*100
		bottom.left &lt;- length(mc.object[[2]][which(mc.object[[2]][,'b[r]_tstat']&lt;tvals[4] &amp; mc.object[[2]][,'b[d]_tstat']&lt;tvals[5],)])/num.trials*100
		top.left &lt;- length(mc.object[[2]][which(mc.object[[2]][,'b[r]_tstat']&lt;tvals[4] &amp; mc.object[[2]][,'b[d]_tstat']&gt;tvals[5],)])/num.trials*100
	
		plot(ylim=c(min(mc.object[[2]][1:num.points,'b[d]_tstat'])-1,max(mc.object[[2]][1:num.points,'b[d]_tstat'])+1),xlim=c(min(mc.object[[2]][1:num.points,'b[r]_tstat'])-1,max(mc.object[[2]][1:num.points,'b[r]_tstat'])+1),x=mc.object[[2]][1:num.points,'b[r]_tstat'],y=mc.object[[2]][1:num.points,'b[d]_tstat'],ylab='tstat,b[d]',xlab='tstat,b[r]',cex.main=0.85,cex.lab=0.75,cex.axis=0.75,cex=0.65,main=paste('T-statistics\nphi=',mc.object[[1]]['phi']),col='blue',pch=21)
  	abline(h=tvals[5])
  	abline(v=tvals[4])
		points(x=tvals[4],y=tvals[5],col='red',cex=1,pch=19)

		text(x=max(mc.object[[2]][1:num.points,'b[r]_tstat']),y=min(mc.object[[2]][1:num.points,'b[d]_tstat'])-0.1,paste(bottom.right,'%'),cex=0.75)
		text(x=max(mc.object[[2]][1:num.points,'b[r]_tstat']),y=max(mc.object[[2]][1:num.points,'b[d]_tstat'])+0.1,paste(top.right,'%'),cex=0.75)
		text(x=min(mc.object[[2]][1:num.points,'b[r]_tstat']),y=min(mc.object[[2]][1:num.points,'b[d]_tstat'])-0.1,paste(bottom.left,'%'),cex=0.75)
		text(x=min(mc.object[[2]][1:num.points,'b[r]_tstat']),y=max(mc.object[[2]][1:num.points,'b[d]_tstat'])+0.1,paste(top.left,'%'),cex=0.75)

} else if(type=='long horizon'){
     
	  h&lt;-hist(plot=FALSE,mc.object[[2]][,'b[long.r]'],breaks=100)
	  plot(xlim=c(-3,3),h,freq=FALSE,col=ifelse(h$breaks&lt;=lr.beta,'lightgrey','red'),ylab='',xlab='b(r)/(1-rho*phi)',cex.main=0.85,cex.lab=0.75,cex.axis=0.75,cex=0.65,main=paste('phi=',mc.object[[1]]['phi']))
	  abline(v=lr.beta,col='red')
	  text(x=lr.beta+0.3,y=0.45,'Data',col='black',cex=0.75)

		bottom.right &lt;- length(mc.object[[2]][which(mc.object[[2]][,'b[r]']&gt;betas[4] &amp; mc.object[[2]][,'phi']&lt;phi,)])/num.trials*100
	  top.right &lt;- length(mc.object[[2]][which(mc.object[[2]][,'b[r]']&gt;betas[4] &amp; mc.object[[2]][,'phi']&gt;phi,)])/num.trials*100
	  bottom.left &lt;- length(mc.object[[2]][which(mc.object[[2]][,'b[r]']&lt;betas[4] &amp; mc.object[[2]][,'phi']&lt;phi,)])/num.trials*100
	  top.left &lt;- length(mc.object[[2]][which(mc.object[[2]][,'b[r]']&lt;betas[4] &amp; mc.object[[2]][,'phi']&gt;phi,)])/num.trials*100
	
    plot(ylim=c(min(mc.object[[2]][1:num.points,'phi'])-0.1,max(mc.object[[2]][1:num.points,'phi'])+0.1),xlim=c(min(mc.object[[2]][1:num.points,'b[r]'])-0.1,max(mc.object[[2]][1:num.points,'b[r]'])+0.1),x=mc.object[[2]][1:num.points,'b[r]'],y=mc.object[[2]][1:num.points,'phi'],ylab='phi',xlab='b[r]',cex.main=0.85,cex.lab=0.75,cex.axis=0.75,cex=0.65,main=paste('phi=',mc.object[[1]]['phi']),col='blue',pch=21)
    abline(h=phi)
  	abline(v=betas[4])
		points(x=betas[4],y=phi,col='red',cex=1,pch=19)
		points(x=0,y=mc.object[[1]]['phi'],cex=1,pch=17,col='green')
	  lines(x=1-rho*mc.object[[2]][,'phi']+betas[5],y=mc.object[[2]][,'phi'],lty='dotted',lwd=0.75,col='gold')
	  lines(x=betas[4]*(1-rho*mc.object[[2]][,'phi'])/(1-phi*rho),col='black',y=mc.object[[2]][,'phi'],lwd=0.75)
  
	  text(x=max(mc.object[[2]][1:num.points,'b[r]']),y=min(mc.object[[2]][1:num.points,'phi'])-0.06,paste(bottom.right,'%'),cex=0.75)
		text(x=max(mc.object[[2]][1:num.points,'b[r]']),y=max(mc.object[[2]][1:num.points,'phi'])+0.06,paste(top.right,'%'),cex=0.75)
		text(x=min(mc.object[[2]][1:num.points,'b[r]']),y=min(mc.object[[2]][1:num.points,'phi'])-0.06,paste(bottom.left,'%'),cex=0.75)
		text(x=min(mc.object[[2]][1:num.points,'b[r]']),y=max(mc.object[[2]][1:num.points,'phi'])+0.06,paste(top.left,'%'),cex=0.75)

}
}

################################################################################
#Table Maker
################################################################################

TableMaker &lt;- function(row.h,core,header,strip,strip.col,col.cut,alpha,border.col,text.col,header.bcol,header.tcol,title)
{
	rows &lt;- nrow(core)+1
	cols &lt;- ncol(core)
  colours &lt;- adjustcolor(col=strip.col,alpha.f=alpha)
  idx &lt;- 1
	
	plot.new()
  plot.window(xlim=c(0,cols*11),ylim=c(0,rows*7))
  mtext(title,side=3,cex=0.7)
	
	for(i in 1:cols){
		for(j in 1:rows){
			if(j==1){
       rect((i-1)*11,rows*7-7,11*i,rows*7,density=NA,col=header.bcol,lty=1,border=border.col)
       text(((i-1)*11+11*i)/2,((rows*7-7)+rows*7)/2,header[i],cex=0.8,col=header.tcol,font=1)
			}
			else if((j&gt;1) &amp;&amp; (row.h==0))
      {
       rect((i-1)*11,(j-1)*7-7,11*i,(j-1)*7,density=NA,col=ifelse(strip,ifelse(core[j-1,i]&lt;col.cut,colours[1],colours[2]),'white'),lwd=1.0,border=border.col)
       text(((i-1)*11+11*i)/2,(((j-1)*7-7)+(j-1)*7)/2,core[j-1,i],col='black',cex=0.75,font=1)
      }
			else if((j&gt;1) &amp;&amp; (row.h!=0) &amp;&amp; (i==row.h[idx]))
      {
       rect((i-1)*11,(j-1)*7-7,11*i,(j-1)*7,density=NA,col=border.col,lwd=1.0,border=border.col)
       text(((i-1)*11+11*i)/2,(((j-1)*7-7)+(j-1)*7)/2,core[j-1,i],col='black',cex=0.75,font=1)
			}
			else if((j&gt;1) &amp;&amp; (row.h!=0) &amp;&amp; (i!=row.h[idx]))
      {
       rect((i-1)*11,(j-1)*7-7,11*i,(j-1)*7,density=NA,col=ifelse(strip,ifelse(core[j-1,i]&lt;col.cut,colours[1],colours[2]),'white'),lwd=1.0,border=border.col)
       text(((i-1)*11+11*i)/2,(((j-1)*7-7)+(j-1)*7)/2,core[j-1,i],col='black',cex=0.75,font=1)
      }
		}
     ifelse(idx&lt;length(row.h),idx&lt;-idx+1,idx&lt;-length(row.h))

	}
}
#####################################################################################
#####################################################################################
#@@@@@@@@@@@@@@@@@@@@@@@@@@@Function Ends@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@#

ll

[Intuition & Procedure]

Cochrane starts with the conventional VAR representation that links log returns,dividend yield and dividend growth variables as seen before:

klk

The Campbell-Shiller (1988) linearisation of returns is then used to link regression coefficients and errors of the VAR such that one can infer the data,coefficients and errors for any one equation from those of the other two :

cv1

cv2

We are interested in learning something about the return predictability coefficient b(r) on the basis of what we can say about phi. In this framework,to generate a coherent null hypothesis with b(r)=0,one must assume a negative b(d) and then address the absence of this coefficient in the data. Essentially this hypothesis will be tested by simulating the VAR under the assumption of no return predictability,b(r)=0 and dividend growth predictability,b(d)=negative. The simulation procedure is as follows (taken from the original paper and other sites such as this presentation) :

[1] Obtain original sample estimates (1927-2004) and the error covariance matrix by running the three VAR regressions. This step provides the inputs needed for the subsequent simulation.

[2] Each of the 20000 monte carlo trials (50000 in the paper) generates a dataset simulated according to the following (shamelessly snipped from the presentation) :

cs

[3] Once we have 20000 simulated datasets,we run each of the VAR regressions for each simulated dataset and collect regression estimates.

[4] Once we have collected regression results across 20000 simulated datasets,we can look at the probability of observing b(r) and b(d) pairs as well as t-statistic pairs that are more extreme than original sample estimates.

[5] Once we have looked at the single period results,we can proceed with the long-horizon regression coefficients ( which are mercifully not new estimates, but rather calculated from the single period estimates of the original sample 1927-2004)

[6] Once we have summarised both single-period and long horizon results ,we can look at the source of power regarding single-period dividend growth coefficients and long horizon tests, as well as examine the effect of varying dividend yield autocorrelation on short- and long run percentage values.

Present value logic suggests that current prices are linked to a trader’s expectations about future returns and dividends. Regression results prove that if expected returns rise,this drives down prices and on average we see the returns that the trader expected in the first place. Impulse response calculations can provide answers to questions like :‘how long do changes in expected returns last?’ Essentially, impulse response functions help illustrate the expected future path of a variable across time for a given unexpected shock to the system at a particular time(t). To illustrate how shocks today reflect changes in expected dividend growth,returns (and other variables) going forward,one would simulate the system of forecasts considered in previous posts

klk

While it may be tempting to shock each of the dependent variables separately,the shocks are related in such a way that there are really only two shocks with the third one following along :

kji

We shall apply [1] a shock to dividend growth with no change in dividend yield, and [2] a shock to dividend yields with no change in dividend growth or:

kmo

The impulse-response function is obtained by simply simulating the system forward through time in response to each of these shocks. The following code illustrates the base case sketched above along with the the extended case that includes the consumption-to-wealth ratio. While the base case is simulated manually using a for loop,the extended case makes use of the VAR function to automatically fit a vector autoregressive model to the data and the irf function to extract impulse responses from the fitted object. Cumulative paths are also drawn.

k

###################################################################################
# Impulse Response
###################################################################################

#Regression on single period lagged variables
idx.temp <- cbind(log.ret,lag(log.dp,-1))
idx.reg<-lm(idx.temp[,1]~idx.temp[,2])
ret.coeff <- coef(idx.reg)[2]

dp.temp <- cbind(log.dp,lag(log.dp,-1))
dp.reg <- lm(dp.temp[,1]~dp.temp[,2])
dp.coeff <- coef(dp.reg)[2]

dg.temp <- cbind(log.dg,lag(log.dp,-1))
dg.reg <- lm(dg.temp[,1]~dg.temp[,2])
dg.coeff <- coef(dg.reg)[2]

#Simulate dividend yield shock
dpt<-ddt<-rt<-chg.pt<-matrix(0,nrow=25,ncol=1)
dpt[2] = 1
ddt[2] =0
chg.pt[2] =-dpt[2]+0+ddt[2]
rt[2] = -rho

for(i in 3:25){
 dpt[i] = dp.coeff*dpt[i-1]
 ddt[i] = dg.coeff*dpt[i-1]
 rt[i] = ret.coeff*dpt[i-1]
 chg.pt[i] = -dpt[i]+dpt[i-1]+ddt[i]
}

#Cumulate
cumddt=cumsum(ddt)
cumrt=cumsum(rt)
cumpt=cumsum(chg.pt)

#simulate dividend growth shock
rt.d<-dpt.d<-ddt.d<-chg.ptd<-matrix(0,nrow=25,ncol=1)
rt.d[2]=1
dpt.d[2]=0
ddt.d[2]=1
chg.ptd[2] =-dpt.d[2]+0+ddt.d[2]

for(i in 3:25){
 dpt.d[i]=dp.coeff*dpt.d[i-1]
 ddt.d[i]=dg.coeff*dpt.d[i-1]
 rt.d[i]=ret.coeff*dpt.d[i-1]
 chg.ptd[i] = -dpt.d[i]+dpt.d[i-1]+ddt.d[i]
}

#Cumulate
cumddt.d=cumsum(ddt.d)
cumdpt.d=cumsum(dpt.d)
cumrt.d=cumsum(rt.d)
cumpt.d=cumsum(chg.ptd)

#Simulate cay shock
var.data=cbind(log.ret,log.dp,log.dg,cay)
vv=VAR(var.data)
imp=irf(vv,response=c('log.ret'),n.ahead=24)
cay.resp=imp$irf$cay
dg.resp=imp$irf$log.dg

#Cumulate
cumrt.c=cumsum(cay.resp)
cumddt.c=cumsum(dg.resp)

#Plot Impulse responses for both shocks
windows()
par(mfrow=c(2,3),mai=c(0.3,0.35,0.5,0.15))
plot(xaxt='n',rt.d,type='l',col='blue',lwd=2,ylab='',xlab='',cex.main=0.8,cex.lab=0.75,cex.axis=0.75,main='Response to \ndividend growth shock')
axis(1, at=0:25, labels=-1:24, padj=1,cex.axis=0.75)
points(rt.d,pch=17,cex=0.95,col='darkblue')
lines(ddt.d,col='red',lwd=2,lty=2)
lines(chg.ptd,col='green',lwd=2,lty=2)
abline(h=0)
abline(v=2,lty='dashed')
legend(bty='n',y.intersp=1,'right',fill=c('blue','red','green'),legend=c('Return','Dividend','Price'),ncol=1,bg='white',cex=0.75)

par(mai=c(0.3,0.35,0.5,0.15))
plot(xaxt='n',ylim=c(min(cay.resp),max(rt)),rt,type='l',col='blue',lwd=2,ylab='',xlab='',cex.main=0.8,cex.lab=0.75,cex.axis=0.75,main='Response to \ndividend yield shock')
axis(1, at=0:25, labels=-1:24, padj=1,cex.axis=0.75)
points(rt,pch=17,cex=0.95,col='darkblue')
lines(ddt,col='red',lwd=2,lty=2)
#lines(chg.pt,col='green',lwd=2,lty=2)
abline(h=0)
abline(v=2,lty='dashed')
legend(bty='n',y.intersp=1,'right',fill=c('blue','red'),legend=c('Return','Dividend'),ncol=1,bg='white',cex=0.75)

par(mai=c(0.3,0.35,0.5,0.15))
plot(xaxt='n',cay.resp,type='l',col='blue',lwd=2,ylab='',xlab='',cex.main=0.8,cex.lab=0.75,cex.axis=0.75,main='Response to \ncay shock (VAR)')
axis(1, at=0:25, labels=-1:24, padj=1,cex.axis=0.75)
points(cay.resp,pch=17,cex=0.95,col='darkblue')
lines(dg.resp,col='red',lwd=2,lty=2)
abline(h=0)
abline(v=2,lty='dashed')
legend(bty='n',y.intersp=1,'right',fill=c('blue','red'),legend=c('Return','Dividend'),ncol=1,bg='white',cex=0.75)

par(mai=c(0.35,0.35,0.5,0.15))
plot(xaxt='n',cumrt.d,type='l',col='blue',lwd=2,ylab='',xlab='',cex.main=0.8,cex.lab=0.75,cex.axis=0.75,main='Cumulative Response to \ndividend growth shock')
axis(1, at=0:25, labels=-1:24, padj=1,cex.axis=0.75)
points(cumrt.d,pch=17,cex=0.95,col='darkblue')
lines(cumddt.d,col='red',lwd=2,lty=2)
lines(cumpt.d,col='green',lwd=2,lty=3)
abline(v=2,lty='dashed')
abline(h=0)
legend(bty='n',y.intersp=1,'right',fill=c('blue','red','green'),legend=c('Return','Dividend','Price'),ncol=1,bg='white',cex=0.75)

par(mai=c(0.35,0.35,0.5,0.15))
plot(xaxt='n',cumrt,type='l',col='blue',lwd=2,ylab='',xlab='',cex.main=0.8,cex.lab=0.75,cex.axis=0.75,main='Cumulative Response to \ndividend yield shock')
axis(1, at=0:25, labels=-1:24, padj=1,cex.axis=0.75)
points(cumrt,pch=17,cex=0.95,col='darkblue')
lines(cumddt,col='red',lwd=2,lty=2)
lines(cumpt,col='green',lwd=2,lty=3)
abline(h=0)
abline(v=2,lty='dashed')
legend(bty='n',y.intersp=1,'right',fill=c('blue','red','green'),legend=c('Return','Dividend','Price'),ncol=1,bg='white',cex=0.75)

par(mai=c(0.35,0.35,0.5,0.15))
plot(xaxt='n',cumrt.c,type='l',col='blue',lwd=2,ylab='',xlab='',cex.main=0.8,cex.lab=0.75,cex.axis=0.75,main='Cumulative Response to \ncay shock (VAR)')
axis(1, at=0:25, labels=-1:24, padj=1,cex.axis=0.75)
points(cumrt.c,pch=17,cex=0.95,col='darkblue')
lines(cumddt.c,col='red',lwd=2,lty=2)
abline(h=0)
abline(v=2,lty='dashed')
legend(bty='n',y.intersp=1,'right',fill=c('blue','red'),legend=c('Price','Dividend'),ncol=1,bg='white',cex=0.75)
###################################################################################

k

ppp

l

[Dividend growth shock]

The dividend growth shock represents a movement in dividends with no change in dividend yields such that prices would rise 1% with a 1% increase in dividends. Hence a dividend growth shock can be interpreted as a cashflow shock with no change in expected returns. A dividend growth shock at time t=1 of 1% with no change in dividend yields implies that prices just rise proportionately to dividends. This price spike we see in the top left plot corresponds to cashflow news only. A high return with no change in dividend yield isolates the event as cashflow news with no implication at all about future returns (which remain flat after the initial shock).

k

[Dividend yield shock]

The dividend yield shock represents a movement in the dividend yield with no change in dividend growth, so it must have been the prices that went down for a positive shock. If dividend growth is unpredictable (i.e b[d]=0 in the set of regressions) and prices change with no change in current dividends,then prices must be changing in the absence of news about future dividends too. If prices are changing with no news about future dividends then they must be changing on the basis of discount rate news. Hence a dividend yield shock can be thought of as a pure discount rate shock when dividends are unpredictable. In typical regressions (in ours aswell) dividends are not completely unpredictable (i.e.b[d] is small but not 0) such that dividend yield shocks combine a small amount of dividend growth news with a large amount of discount rate news.

The initial dip in prices is accompanied by a huge negative return shock (goes off the bottom of the graph). Given that it is the independent variable in all regressions of the base case,a shock to the dividend yield at time t=1 is expected to have ripple effects over time on other variables in the system. After disastrous first period returns,subsequent positive returns bring cumulative prices/return on to a positive path again.

Impulse response functions help separate a price hike that corresponds to cashflow news from one motivated by discount rate news. In the first scenario, a dividend growth shock occurs without a change in dividend yields and the initial price/return hikes revert to their original values in the second period. In the second case, a dividend yield shock occurs without a change in dividend growth and the initial price/return hikes melt away slowly over time.

l

[Consumption/Wealth ratio shock]

A shock to cay is followed firstly by a hike in expected returns in the short-term (this corresponds to the short term predictability alluded to in the previous post), secondly by a rapid decline in returns (much faster than the expected returns that follow a dividend yield shock), and finally by a reversal in returns.

kk

As usual,the Chicago lecture is the main reference used in this and other posts. Although there are some discrepancies between the plot in the lecture and mine (data differences?coding error?) the main point seems valid. 

Campbell & Shiller’s(1988) present value identity provides a convenient framework for understanding time series predictability and addressing the new issues alluded to towards the end of the previous post. As usual, i will not profess to understand every step of the derivation or the requisite mathematics behind it, but I shall provide (copy) a brief sketch that is relevant to the post. Starting with a simple return decomposition and taking logs:

xa

xb

The Taylor expansion of the final equation yields:

xc

Iterating forward and ruling out explosive behaviour of stock prices:

xd

From this relation,it is clear that prices are high relative to fundamentals if investors today :

[a] expect dividends to rise in the future.This is sensible since high dividend growth expectations give high prices (and p-d). 

[b] expect returns (discount rates) to be low in the future. This is sensible since low discount rates imply future cashflows are discounted at a lower rate to the present and hence provide higher current prices (and p-d).

Price-dividend ratios can move if and only if there is news about current dividends,future dividend growth or future returns. Hence if dividend growth and returns are totally unpredictable (i.e. same for every time t), then it must follow from the identity that p(t)-d(t) be constant over time.

If prices are supposed to capture how the world looks going forward,then constant expectations of fundamentals going forward must generate constant prices today. Since empirically p(t)-d(t) is not constant, it must be true that one of the two components on the right-hand-side of the identity be predictable. To determine which of these two channels is ultimately responsible for the observed variation of the p-d ratio, it is convenient to link predictability to variability via a set of regressions. Reformulating the identity to :

bb

and regressing each component of the identity on the dividend yield would result in the following set of regressions:

fg

The coefficients must be such that:

kk

The set of regressions might give one the illusion that the right hand side (dividend yield) is causally related to the dependent variable on the left (whatever it may be).Contrary to typical statistical interpretation,these forecasting regressions are estimated in an attempt to understand how the right hand variable is formed. Traders that observe some news about expected cash flows or discount rates going forward (LHS) would incorporate this information in current prices (RHS). Therefore, these regressions are not explanatory in nature, rather they seek to answer the question ‘how much of the return can one know ahead of time’? The R-squared measure provides an answer to this question.

The long run return & dividend growth forecasting regression coefficients must add to one.If dividend yields vary,they must forecast long-run returns or dividend growth. Since covariances are the numerators of regression betas,multiplying both sides by the variance of the dividend yield ratio (denominator of the coefficient) implies :

nb

Clearly,the price ratio only varies if it forecasts long-run dividend growth or long run returns.If both coefficients were equal to zero this would imply that the price ratio can neither forecast dividend growth nor returns and dp(t) would not vary over time.Since dp(t) clearly varies over time it must forecast at least one of returns or dividend growth. Returning to the original identity, variation in dividend yields must correspond to changing investor expectations of dividend growth or returns.

Towards the end of the previous post, I alluded to the fact that the initial tests & results concerning stock return predictability had been overturned in the 80s.The simple forecasting regression of return on lagged returns used in the first generation of tests engendered the view that beta=0,returns followed a random walk and that one cannot beat the market.

The second generation of time series tests overturned these remarks by focussing on new variables (price ratios in particular) across longer horizons, concluding that expected returns vary over time in ways not captured by the classic random walk.

To illustrate how the introduction of longer horizons and new variables overturned previously cherished views of stock-return behaviour over time, consider the following regression :

km

The dependent variable is the CSRP value weighted excess return compounded over k horizons such that for k=5,the compounded stock return is given by the following with each R(t) being defined as (1+Return) :

xds

and the corresponding excess return is given by :

xaa

The following code illustrates the results of regressing compounded returns across 15 years on the dividend price ratio and the drastic variation in the R-squared measure aswell as the slope coefficient across horizons.

k

#####################################################################################
# New views & facts
#####################################################################################
#Return regressions~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

dep.list<-list()
new.reg <- list()
dep.list[[1]]<-matrix(c(idx.ret,tbill,exc),nrow=length(idx.ret))
for(i in 2:15){
  dep.list[[i]]=CustomCompound(idx.ret,tbill,i)
}

png(file="actf%02d.png",width=550,height=500)
for(i in 1:15){
  temp=cbind(dep.list[[i]][,3],lag(dp,-1)[1:nrow(dep.list[[i]])])
  new.reg[[i]]=lm(temp[,1]~temp[,2])
 plot(temp[,1],ylab='Returns',xlab='',cex.main=0.85,cex.lab=0.75,cex.axis=0.75,main=paste('Actual versus forecast\nHorizon',i,' years'),col='darkgreen',type='l',lwd=2)
	lines(fitted(new.reg[[i]]),col='blue')
	legend(bty='n',y.intersp=1,'bottomleft',fill=c('darkgreen','blue'),legend=c('Actual','Fitted'),ncol=1,bg='white',cex=0.75)
}
dev.off()
shell("convert -delay 15 *.png actf.gif")

#Extract regression values
estim <-RegExtractor(new.reg,'est')
pvals <-matrix(RegExtractor(new.reg,'pval'),ncol=2)
tvals <-matrix(RegExtractor(new.reg,'tval'),ncol=2)
rsq <-matrix(RegExtractor(new.reg,'rsq'),ncol=1)

#Variation relative to mean level
std.vals=sapply(new.reg,function(v) sd(fitted(v)))
mean.vals=sapply(new.reg,function(v) mean(fitted(v)))

#Plot & Table
row.names<- paste('Compound ',1:15,' Years')
col.names<-c('Intercept','Beta','t-stats','p-vals','R2','E(R)','Std(Fitted)')
reg.table=cbind(estim,(tvals[,2]),(pvals[,2]),rsq,mean.vals,std.vals)

windows()
layout(matrix(c(1,1,2),nrow=3))
barplot(t(rsq*100),ylab='R-squared(%)',xlab='horizon',cex.main=0.85,cex.lab=0.75,cex.axis=0.75,main='Comparing R-squared across horizons\nReturn regressions',col='gold',names.arg=1:15)
est.tab <- round(reg.table,5)
est.tab <- apply(est.tab, 2, rev)
est.tab <- cbind(rev(row.names),est.tab)
par(mai=c(0.15,0.25,0.25,0.15))
TableMaker(row.h=1,est.tab,c('Multiple Periods',col.names),strip=F,strip.col=c('green','blue'),col.cut=0.05,alpha=0.7,border.col='lightgrey',text.col='black',header.bcol='gold',header.tcol='black',title=c('Regression of returns on lagged DP\nAnnual data for 1952-2008'))

l
An animated gif that shows how the predicted and actual values vary for each horizon of the regression :

mmThe correlation is evident,high dividend yields(low prices) mean high subsequent returns and low dividend yields(high prices) mean low subsequent returns. A more useful plot that illustrates how r-squared and regression results vary with the horizon : 

df

R-squared and slope coefficients increase with the regression horizon implying that forecastability get more economically interesting at longer horizons. From the Chicago lectures I would have expected that the slope remains statistically insignificant throughout, but I clearly obtained the opposite result with smaller p-values for longer horizons. Another good example of the ‘tinkering’ aspect of this blog where i may have made a mistake somewhere or glossed over some procedure that has been taken for granted but is unknown to me.

Looking at the parallel forecasts of dividend growth :

gtg

When prices are high, this should be a signal that investors expect dividends to be higher in the future, so on average one should observe higher dividend growth in the years following high prices (low dividend yields) and expect a negative slope coefficient. To determine whether this is the case, I ran dividend growth forecasting regressions for 15 horizons as before :

k

#Dividend growth ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
dep.list<-list()
new.reg <- list()
dep.list[[1]]<-dg
temp<-NULL
for(i in 2:15){
  dep.list[[i]]=DivCompound(dg,i)
}

for(i in 1:15){
  temp=cbind(dep.list[[i]],lag(dp,-1)[1:length(dep.list[[i]])])
  new.reg[[i]]=lm(temp[,1]~temp[,2])
}

#Extract regression values
estim <-RegExtractor(new.reg,'est')
pvals <-matrix(RegExtractor(new.reg,'pval'),ncol=2)
tvals <-matrix(RegExtractor(new.reg,'tval'),ncol=2)
rsq <-matrix(RegExtractor(new.reg,'rsq'),ncol=1)

#Table
row.names<- paste('Compound ',1:15,' Years')
col.names<-c('Intercept','Beta','t-stats','p-vals','R2')
reg.table=cbind(estim,(tvals[,2]),(pvals[,2]),rsq)

windows()
layout(matrix(c(1,1,2),nrow=3))
barplot(ylim=c(0,35),t(rsq*100),ylab='R-squared(%)',xlab='horizon',cex.main=0.85,cex.lab=0.75,cex.axis=0.75,main='Comparing R-squared across horizons\nDividend growth regressions',col='lightblue',names.arg=1:15)
est.tab <- round(reg.table,5)
est.tab <- apply(est.tab, 2, rev)
est.tab <- cbind(rev(row.names),est.tab)
par(mai=c(0.15,0.25,0.40,0.15))
TableMaker(row.h=1,est.tab,c('Multiple Periods',col.names),strip=F,strip.col=c('green','blue'),col.cut=0.05,alpha=0.7,border.col='lightgrey',text.col='black',header.bcol='lightblue',header.tcol='black',title=c('Regression of dividend growth on lagged DP\nAnnual data for 1952-2008'))
#####################################################################################

k
hn
Keeping the scale of the plot as before illustrates just how appalling the R-squared is.The coefficient which should be negative (theoretically) is positive in our regression (empirically). We do not,contrary to expectations,observe higher dividend growth in the years following high prices.

The present value identity sketched above establishes that : [a] if price variation comes from news about dividend growth,then the price-dividend ratio should forecast dividend growth ; [b] if price variation comes from news about changing discount rates,then the price-dividend ratio should forecast returns.

Using the results obtained from the return and dividend growth regressions above, we would conclude that returns which should not be forecastable are and dividend growth which should be forecastable isn’t. On average, price dividend ratios are moving in response to expected return news rather than cash flow news. Time varying discount rates explain virtually all the variance of dividend yields while dividend growth forecasts explain essentially none of it. This is a clear contradiction of the view engendered by first generation tests.