45-734 PROBABILITY AND STATISTICS II (4th Mini AY1997-98)

Note Set 12 Handouts



  1. ACF and PACF Correlograms

    1. The first example is data generated from an AR(1) with parameter .5. Note the one big spike in the first position in the PACF diagram.
                            Correlogram of Y0
      ==============================================================
      Date: 04/18/98   Time: 15:09                                            
      Sample: 1 500                                                           
      Included observations: 100                                              
      ==============================================================
       Autocorrelation Partial Correlation  AC     PAC  Q-Stat Prob           
      ==============================================================
            . |***    |      . |***    |  1 0.401 0.401 16.566 0.000          
            . |*.     |      . | .     |  2 0.147-0.016 18.817 0.000          
            . | .     |      . | .     |  3 0.032-0.025 18.925 0.000          
            . | .     |      . | .     |  4-0.044-0.055 19.132 0.001          
            . | .     |      . |*.     |  5 0.058 0.118 19.488 0.002          
      ==============================================================
      
    2. In this example, the data are AR(1) but now the parameter is .9.
                            Correlogram of Y1
      ==============================================================
      Date: 04/18/98   Time: 15:22                                            
      Sample: 1 500                                                           
      Included observations: 100                                              
      ==============================================================
       Autocorrelation Partial Correlation  AC     PAC  Q-Stat Prob           
      ==============================================================
            . |****** |      . |****** |  1 0.752 0.752 58.201 0.000          
            . |****   |      . | .     |  2 0.588 0.052 94.132 0.000          
            . |****   |      . |*.     |  3 0.512 0.125 121.68 0.000          
            . |***    |      . | .     |  4 0.440 0.016 142.26 0.000          
            . |***    |      . | .     |  5 0.392 0.052 158.76 0.000          
      ==============================================================
      
    3. In this example the data are AR(2) with parameters .5 and .5.
                            Correlogram of Y2
      ==============================================================
      Date: 04/18/98   Time: 15:56                                            
      Sample: 1 500                                                           
      Included observations: 100                                              
      ==============================================================
       Autocorrelation Partial Correlation  AC     PAC  Q-Stat Prob           
      ==============================================================
            . |****** |      . |****** |  1 0.800 0.800 65.963 0.000          
            . |****** |      . |***    |  2 0.770 0.359 127.60 0.000          
            . |*****  |      . | .     |  3 0.671-0.035 174.96 0.000          
            . |*****  |      . | .     |  4 0.617 0.007 215.44 0.000          
            . |****   |      . | .     |  5 0.571 0.061 250.40 0.000          
      ==============================================================
      
    4. In this example, the data are AR(3) with parameters .8, -.6, and .6.
                            Correlogram of Y3
      ==============================================================
      Date: 04/18/98   Time: 15:58                                            
      Sample: 1 500                                                           
      Included observations: 100                                              
      ==============================================================
       Autocorrelation Partial Correlation  AC     PAC  Q-Stat Prob           
      ==============================================================
            . |*****  |      . |*****  |  1 0.612 0.612 38.583 0.000          
            . |**     |      .*| .     |  2 0.265-0.175 45.909 0.000          
            . |***    |      . |*****  |  3 0.451 0.606 67.341 0.000          
            . |*****  |      . |*.     |  4 0.653 0.188 112.70 0.000          
            . |***    |      .*| .     |  5 0.440-0.095 133.52 0.000          
      ==============================================================
      
    5. Now let us try and estimate a model based upon an examination of the correlograms. The next correlogram is for a series (t=500) with parameters .8, -.6, and .6 respectively. Fortunately for us, the PACF correlogram indicates the presence of 3 lags.

      IDENT(5) Y
                             Correlogram of Y
      ==============================================================
      Date: 04/18/98   Time: 16:01                                            
      Sample: 1 500                                                           
      Included observations: 500                                              
      ==============================================================
       Autocorrelation Partial Correlation  AC     PAC  Q-Stat Prob           
      ==============================================================
             .|****   |       .|****   |  1 0.542 0.542 147.56 0.000          
             .|*      |      **|.      |  2 0.145-0.210 158.13 0.000          
             .|***    |       .|*****  |  3 0.382 0.593 231.75 0.000          
             .|****   |       *|.      |  4 0.492-0.094 354.01 0.000          
             .|**     |       .|.      |  5 0.203-0.007 374.99 0.000          
      ==============================================================
      
      Lets estimate AR(1), AR(2), and AR(3) models for this time series to see how the coefficients behave.

      First the AR(1)--in EVIEWS issue the command:

      LS Y C AR(1)
      ============================================================
      LS // Dependent Variable is Y                                         
      Date: 04/18/98   Time: 16:03                                          
      Sample(adjusted): 2 500                                               
      Included observations: 499 after adjusting endpoints                  
      Convergence achieved after 3 iterations                               
      ============================================================
            Variable      CoefficienStd. Errort-Statistic  Prob.            
      ============================================================
               C          -0.261085   0.122331  -2.134245   0.0333          
             AR(1)         0.542895   0.037725   14.39070   0.0000          
      ============================================================
      R-squared            0.294127    Mean dependent var-0.263589          
      Adjusted R-squared   0.292706    S.D. dependent var 1.485259          
      S.E. of regression   1.249114    Akaike info criter 0.448870          
      Sum squared resid    775.4626    Schwarz criterion  0.465754          
      Log likelihood      -818.0433    F-statistic        207.0922          
      Durbin-Watson stat   1.774118    Prob(F-statistic)  0.000000          
      ============================================================
      Inverted AR Roots          .54                                        
      ============================================================
      
      In actual data work, you should always take a look at the residuals after each model that you try. Go into the "View" menu and select "Residual Tests", then select "Correlogram-Q-Statistics", then enter the number of lags (always choose a number larger than the number of parameters being estimated!).

      This is the correlogram with 5 lags (note that the Q-Statistic will be Chi-Square with 5-1 = 4 degrees of freedom):
       
                         Correlogram of Residuals
      ==============================================================
      Date: 04/18/98   Time: 16:06                                            
      Sample: 2 500                                                           
      Included observations: 499                                              
      Q-statistic probabilities adjusted for 1 ARMA term(s)
      ==============================================================
       Autocorrelation Partial Correlation  AC     PAC  Q-Stat Prob           
      ==============================================================
             .|*      |       .|*      |  1 0.113 0.113 6.4002                
           ***|.      |    ****|.      |  2-0.442-0.460 104.53 0.000          
             .|**     |       .|***    |  3 0.213 0.434 127.40 0.000          
             .|***    |       .|*      |  4 0.450 0.140 229.75 0.000          
             .|.      |       .|*      |  5-0.057 0.092 231.39 0.000          
      ==============================================================
      
      The residual correlogram for the AR(1) strongly indicates that adding terms for second and third lags is appropriate. Beginning with the AR(2):

      LS Y C AR(1) AR(2)
      ============================================================
      LS // Dependent Variable is Y                                         
      Date: 04/18/98   Time: 16:13                                          
      Sample(adjusted): 3 500                                               
      Included observations: 498 after adjusting endpoints                  
      Convergence achieved after 3 iterations                               
      ============================================================
            Variable      CoefficienStd. Errort-Statistic  Prob.            
      ============================================================
               C          -0.263769   0.099012  -2.664006   0.0080          
             AR(1)         0.655926   0.043956   14.92242   0.0000          
             AR(2)        -0.209900   0.044168  -4.752335   0.0000          
      ============================================================
      R-squared            0.324915    Mean dependent var-0.263974          
      Adjusted R-squared   0.322187    S.D. dependent var 1.486728          
      S.E. of regression   1.224014    Akaike info criter 0.410277          
      Sum squared resid    741.6141    Schwarz criterion  0.435642          
      Log likelihood      -805.7904    F-statistic        119.1204          
      Durbin-Watson stat   1.747068    Prob(F-statistic)  0.000000          
      ============================================================
      Inverted AR Roots      .33 -.   .33+.32i                              
      ============================================================
      
      The residual correlogram from the AR(2) model is somewhat ambiguous but clearly indicates that at least a third lag should be added to the model (here the Q-Statistic has 5-2 = 3 degrees of freedom).
                         Correlogram of Residuals
      ==============================================================
      Date: 04/18/98   Time: 16:14                                            
      Sample: 3 500                                                           
      Included observations: 498                                              
      Q-statistic probabilities adjusted for 2 ARMA term(s)
      ==============================================================
       Autocorrelation Partial Correlation  AC     PAC  Q-Stat Prob           
      ==============================================================
             .|*      |       .|*      |  1 0.124 0.124 7.7243                
            **|.      |     ***|.      |  2-0.303-0.324 53.926                
             .|**     |       .|****   |  3 0.318 0.463 104.68 0.000          
             .|***    |       .|**     |  4 0.449 0.246 206.21 0.000          
             .|.      |       .|*      |  5-0.034 0.089 206.81 0.000          
      ==============================================================
      
      LS Y C AR(1) AR(2) AR(3)
      ============================================================
      LS // Dependent Variable is Y                                         
      Date: 04/18/98   Time: 16:18                                          
      Sample(adjusted): 4 500                                               
      Included observations: 497 after adjusting endpoints                  
      Convergence achieved after 3 iterations                               
      ============================================================
            Variable      CoefficienStd. Errort-Statistic  Prob.            
      ============================================================
               C          -0.220391   0.203460  -1.083211   0.2792          
             AR(1)         0.780235   0.036020   21.66122   0.0000          
             AR(2)        -0.596923   0.042455  -14.05997   0.0000          
             AR(3)         0.600186   0.036243   16.55998   0.0000          
      ============================================================
      R-squared            0.566875    Mean dependent var-0.260287          
      Adjusted R-squared   0.564240    S.D. dependent var 1.485946          
      S.E. of regression   0.980905    Akaike info criter-0.030544          
      Sum squared resid    474.3520    Schwarz criterion  0.003328          
      Log likelihood      -693.6223    F-statistic        215.0801          
      Durbin-Watson stat   1.879158    Prob(F-statistic)  0.000000          
      ============================================================
      Inverted AR Roots          .8  -.05+.83i  -.05 -.83i                  
      ============================================================
      
      The correlogram of the residuals for the AR(3) model indicates that we have the correct model (here the Q-Statistic has 5-3 = 2 degrees of freedom).
                         Correlogram of Residuals
      ==============================================================
      Date: 04/18/98   Time: 16:19                                            
      Sample: 4 500                                                           
      Included observations: 497                                              
      Q-statistic probabilities adjusted for 3 ARMA term(s)
      ==============================================================
       Autocorrelation Partial Correlation  AC     PAC  Q-Stat Prob           
      ==============================================================
             .|.      |       .|.      |  1 0.058 0.058 1.6800                
             .|.      |       .|.      |  2 0.002-0.002 1.6815                
             .|.      |       .|.      |  3 0.046 0.047 2.7651                
             .|.      |       .|.      |  4-0.014-0.019 2.8634 0.091          
             .|.      |       .|.      |  5-0.053-0.051 4.2867 0.117          
      ==============================================================
      
      Out of curiousity, we can add AR(4) and AR(5) terms just to see if there are any effects. LS Y C AR(1) AR(2) AR(3) AR(4) AR(5)
      ============================================================
      LS // Dependent Variable is Y                                         
      Date: 04/18/98   Time: 16:23                                          
      Sample(adjusted): 6 500                                               
      Included observations: 495 after adjusting endpoints                  
      Convergence achieved after 3 iterations                               
      ============================================================
            Variable      CoefficienStd. Errort-Statistic  Prob.            
      ============================================================
               C          -0.255278   0.184129  -1.386414   0.1663          
             AR(1)         0.832183   0.045099   18.45256   0.0000          
             AR(2)        -0.638784   0.058633  -10.89461   0.0000          
             AR(3)         0.658227   0.058163   11.31699   0.0000          
             AR(4)        -0.079692   0.058473  -1.362884   0.1735          
             AR(5)        -0.010189   0.045168  -0.225574   0.8216          
      ============================================================
      R-squared            0.571043    Mean dependent var-0.267188          
      Adjusted R-squared   0.566657    S.D. dependent var 1.482234          
      S.E. of regression   0.975737    Akaike info criter-0.037078          
      Sum squared resid    465.5585    Schwarz criterion  0.013887          
      Log likelihood      -687.1978    F-statistic        130.1948          
      Durbin-Watson stat   1.991752    Prob(F-statistic)  0.000000          
      ============================================================
      Inverted AR Roots          .8       .23       -.08   -.08+.83i        
                            -.08 -.83i                                      
      ============================================================
      
      Here is the correlogram of the residuals using 10 legs (the degrees of freedom are 10-5 = 5). The AR(3) model is clearly the correct model.
                         Correlogram of Residuals
      ==============================================================
      Date: 04/18/98   Time: 16:24                                            
      Sample: 6 500                                                           
      Included observations: 495                                              
      Q-statistic probabilities adjusted for 5 ARMA term(s)
      ==============================================================
       Autocorrelation Partial Correlation  AC     PAC  Q-Stat Prob           
      ==============================================================
             .|.      |       .|.      |  1 0.003 0.003 0.0045                
             .|.      |       .|.      |  2-0.004-0.004 0.0121                
             .|.      |       .|.      |  3 0.014 0.014 0.1077                
             .|.      |       .|.      |  4 0.012 0.012 0.1814                
             .|.      |       .|.      |  5 0.007 0.007 0.2037                
             .|.      |       .|.      |  6-0.016-0.016 0.3263 0.568          
             .|.      |       .|.      |  7-0.052-0.052 1.6984 0.428          
             .|.      |       .|.      |  8-0.054-0.055 3.1814 0.364          
             .|.      |       .|.      |  9 0.013 0.013 3.2622 0.515          
             .|.      |       .|.      | 10 0.008 0.009 3.2923 0.655          
      ==============================================================