45-734 PROBABILITY AND STATISTICS II (4th Mini AY1997-98)

ARIMA Examples (Part 1)



I have created several ARIMA series for us to study. These examples are all stationary series so we do not need to do any differencing [ARIMA(p,0,q) examples]. Unfortunately, with ARIMA interpreting the ACFs and PACFs is much more tricky. Indeed, we have to proceed mostly by trial and error until we get a reasonable looking model. The best way to proceed is to start with the IDENT command and make a guess about the model, estimate the model, look at the residuals, and add more terms if the pattern of the spikes suggests it. This process is continued until the residuals are white noise.
  1. First Example

    IDENT(5) Y1
                          Correlogram of Y1
    ==============================================================
    Date: 04/21/98   Time: 16:42                                            
    Sample: 1 100                                                           
    Included observations: 100                                              
    ==============================================================
     Autocorrelation Partial Correlation  AC     PAC  Q-Stat Prob           
    ==============================================================
          . |***    |      . |***    |  1 0.444 0.444 20.306 0.000          
         ***| .     |  ******| .     |  2-0.431-0.782 39.652 0.000          
       *****| .     |      . |**     |  3-0.603 0.275 77.871 0.000          
          .*| .     |     ***| .     |  4-0.153-0.345 80.352 0.000          
          . |**     |      . |**     |  5 0.302 0.319 90.148 0.000          
    ==============================================================
    
    This pattern is somewhat ambiguous but clearly we have multiple terms present. Lets be conservative and start off by estimating up to a MA(2).
    ============================================================
    LS // Dependent Variable is Y1                                        
    Date: 04/21/98   Time: 16:44                                          
    Sample: 1 100                                                         
    Included observations: 100                                            
    Convergence achieved after 23 iterations                              
    ============================================================
          Variable      CoefficienStd. Errort-Statistic  Prob.            
    ============================================================
             C           0.067147   0.239710   0.280117   0.7800          
           MA(1)         0.835517   0.032102   26.02710   0.0000          
           MA(2)        -0.152868   0.002619  -58.37944   0.0000          
    ============================================================
    R-squared            0.574891    Mean dependent var 0.150504          
    Adjusted R-squared   0.566126    S.D. dependent var 2.163285          
    S.E. of regression   1.424937    Akaike info criter 0.737796          
    Sum squared resid    196.9532    Schwarz criterion  0.815951          
    Log likelihood      -175.7836    F-statistic        65.58840          
    Durbin-Watson stat   1.284884    Prob(F-statistic)  0.000000          
    ============================================================
    Inverted MA Roots          .1      -.99                               
    ============================================================
    
                       Correlogram of Residuals
    ==============================================================
    Date: 04/21/98   Time: 16:45                                            
    Sample: 1 100                                                           
    Included observations: 100                                              
    Q-statistic probabilities adjusted for 2 ARMA term(s)                   
    ==============================================================
     Autocorrelation Partial Correlation  AC     PAC  Q-Stat Prob           
    ==============================================================
          . |***    |      . |***    |  1 0.356 0.356 13.034                
         ***| .     |   *****| .     |  2-0.430-0.637 32.265                
        ****| .     |      .*| .     |  3-0.551-0.143 64.209 0.000          
          .*| .     |      . | .     |  4-0.100-0.032 65.265 0.000          
          . |**     |      .*| .     |  5 0.269-0.080 73.037 0.000          
    ==============================================================
    
    Clearly we are not done yet. The two spikes in the PACF graph suggest--now that we have MA(1) and MA(2) terms--that we should add AR(1) and AR(2) terms to the model.
    ============================================================
    LS // Dependent Variable is Y1                                        
    Date: 04/21/98   Time: 16:46                                          
    Sample(adjusted): 3 100                                               
    Included observations: 98 after adjusting endpoints                   
    Convergence achieved after 11 iterations                              
    ============================================================
          Variable      CoefficienStd. Errort-Statistic  Prob.            
    ============================================================
             C           0.162320   0.138851   1.169027   0.2454          
           AR(1)         0.702614   0.111219   6.317390   0.0000          
           AR(2)        -0.687793   0.077228  -8.905977   0.0000          
           MA(1)         0.603438   0.149147   4.045922   0.0001          
           MA(2)        -0.294895   0.146821  -2.008535   0.0475          
    ============================================================
    R-squared            0.784079    Mean dependent var 0.156682          
    Adjusted R-squared   0.774793    S.D. dependent var 2.182691          
    S.E. of regression   1.035819    Akaike info criter 0.120057          
    Sum squared resid    99.78159    Schwarz criterion  0.251943          
    Log likelihood      -139.9388    F-statistic        84.42848          
    Durbin-Watson stat   2.014543    Prob(F-statistic)  0.000000          
    ============================================================
    Inverted AR Roots      .35+.7   .35 -.75i                             
    Inverted MA Roots          .3      -.92                               
    ============================================================
    
    
    
                       Correlogram of Residuals
    ==============================================================
    Date: 04/21/98   Time: 16:47                                            
    Sample: 3 100                                                           
    Included observations: 98                                               
    Q-statistic probabilities adjusted for 4 ARMA term(s)                   
    ==============================================================
     Autocorrelation Partial Correlation  AC     PAC  Q-Stat Prob           
    ==============================================================
          . | .     |      . | .     |  1-0.009-0.009 0.0075                
          . | .     |      . | .     |  2-0.017-0.017 0.0365                
          . | .     |      . | .     |  3-0.014-0.015 0.0577                
          . |*.     |      . |*.     |  4 0.121 0.121 1.5919                
          .*| .     |      .*| .     |  5-0.136-0.136 3.5300 0.060          
          . | .     |      . | .     |  6 0.040 0.046 3.7045 0.157          
          . | .     |      . | .     |  7-0.018-0.021 3.7402 0.291          
          . | .     |      . | .     |  8 0.038 0.022 3.8937 0.421          
          .*| .     |      . | .     |  9-0.085-0.054 4.6849 0.456          
          .*| .     |      .*| .     | 10-0.137-0.170 6.7816 0.342          
    ==============================================================
    
    This looks pretty good. All the p-values (except for the 5-lag Q-Statistic which has only 1 degree of freedom) are reasonable.

  2. Second Example:

    IDENT(5) Y2
                          Correlogram of Y2
    ==============================================================
    Date: 04/21/98   Time: 16:51                                            
    Sample: 1 100                                                           
    Included observations: 100                                              
    ==============================================================
     Autocorrelation Partial Correlation  AC     PAC  Q-Stat Prob           
    ==============================================================
          . |****** |      . |****** |  1 0.728 0.728 54.619 0.000          
          . |*.     | *******| .     |  2 0.093-0.931 55.510 0.000          
        ****| .     |      **| .     |  3-0.563-0.247 88.888 0.000          
     *******| .     |      . |*.     |  4-0.891 0.093 173.25 0.000          
      ******| .     |      .*| .     |  5-0.745-0.167 232.86 0.000          
    ==============================================================
    
    The pattern is once again ambiguous. I am inclined to start once again by estimating MA terms. Given the absence of a spike at MA(2), I am going to first estimate MA(1) and MA(3) terms.
    ============================================================
    LS // Dependent Variable is Y2                                        
    Date: 04/21/98   Time: 16:52                                          
    Sample: 1 100                                                         
    Included observations: 100                                            
    Convergence achieved after 14 iterations                              
    ============================================================
          Variable      CoefficienStd. Errort-Statistic  Prob.            
    ============================================================
             C          -0.428425   0.387856  -1.104597   0.2721          
           MA(1)         0.958552   0.023630   40.56466   0.0000          
           MA(3)        -0.608096   0.000476  -1277.001   0.0000          
    ============================================================
    R-squared            0.749535    Mean dependent var-0.466657          
    Adjusted R-squared   0.744370    S.D. dependent var 5.677772          
    S.E. of regression   2.870672    Akaike info criter 2.138633          
    Sum squared resid    799.3532    Schwarz criterion  2.216788          
    Log likelihood      -245.8255    F-statistic        145.1396          
    Durbin-Watson stat   0.664071    Prob(F-statistic)  0.000000          
    ============================================================
    Inverted MA Roots          .6  -.79+.60i  -.79 -.60i                  
    ============================================================
    
    
    
                       Correlogram of Residuals
    ==============================================================
    Date: 04/21/98   Time: 16:53                                            
    Sample: 1 100                                                           
    Included observations: 100                                              
    Q-statistic probabilities adjusted for 2 ARMA term(s)                   
    ==============================================================
     Autocorrelation Partial Correlation  AC     PAC  Q-Stat Prob           
    ==============================================================
          . |*****  |      . |*****  |  1 0.655 0.655 44.146                
          . |*.     |    ****| .     |  2 0.128-0.525 45.860                
        ****| .     |  ******| .     |  3-0.514-0.721 73.657 0.000          
     *******| .     |     ***| .     |  4-0.843-0.391 149.21 0.000          
       *****| .     |      . |***    |  5-0.680 0.365 198.85 0.000          
    ==============================================================
    
    Given that we have estimated MA(1) and MA(3) terms, the PACFs indicate that we should try at least AR(1) and AR(3). Given the spike at the 2nd PACF lets add that in too.
    ============================================================
    LS // Dependent Variable is Y2                                        
    Date: 04/21/98   Time: 16:54                                          
    Sample(adjusted): 4 100                                               
    Included observations: 97 after adjusting endpoints                   
    Convergence achieved after 10 iterations                              
    ============================================================
          Variable      CoefficienStd. Errort-Statistic  Prob.            
    ============================================================
             C          -0.354521   0.157481  -2.251192   0.0268          
           AR(1)         0.543147   0.063835   8.508590   0.0000          
           AR(2)         0.358374   0.087511   4.095176   0.0001          
           AR(3)        -0.919271   0.063529  -14.47018   0.0000          
           MA(1)         0.868996   0.089440   9.715997   0.0000          
           MA(3)        -0.123682   0.083465  -1.481841   0.1418          
    ============================================================
    R-squared            0.976675    Mean dependent var-0.483771          
    Adjusted R-squared   0.975394    S.D. dependent var 5.763586          
    S.E. of regression   0.904100    Akaike info criter-0.141770          
    Sum squared resid    74.38316    Schwarz criterion  0.017490          
    Log likelihood      -124.7612    F-statistic        762.0856          
    Durbin-Watson stat   2.051030    Prob(F-statistic)  0.000000          
    ============================================================
    Inverted AR Roots      .73+.6   .73 -.68      -.92                    
    Inverted MA Roots          .3  -.60 -.17  -.60+.17i                   
    ============================================================
    
    
    
                       Correlogram of Residuals
    ==============================================================
    Date: 04/21/98   Time: 16:55                                            
    Sample: 4 100                                                           
    Included observations: 97                                               
    Q-statistic probabilities adjusted for 5 ARMA term(s)                   
    ==============================================================
     Autocorrelation Partial Correlation  AC     PAC  Q-Stat Prob           
    ==============================================================
          . | .     |      . | .     |  1-0.040-0.040 0.1581                
          . | .     |      . | .     |  2 0.036 0.034 0.2874                
          .*| .     |      .*| .     |  3-0.064-0.062 0.7108                
          . | .     |      . | .     |  4 0.057 0.051 1.0463                
          . |*.     |      . |*.     |  5 0.090 0.099 1.8878                
          . | .     |      . | .     |  6-0.043-0.044 2.0815 0.149          
          . |*.     |      . |*.     |  7 0.142 0.142 4.2181 0.121          
          . |*.     |      . |*.     |  8 0.101 0.127 5.3290 0.149          
          .*| .     |      .*| .     |  9-0.060-0.082 5.7267 0.221          
          . | .     |      . | .     | 10 0.002 0.006 5.7271 0.334          
    ==============================================================
    
    We have white noise and the MA(3) term is not significant. This suggests that we try dropping it and see if a simplier model works just as well.
    ============================================================
    LS // Dependent Variable is Y2                                        
    Date: 04/21/98   Time: 16:57                                          
    Sample(adjusted): 4 100                                               
    Included observations: 97 after adjusting endpoints                   
    Convergence achieved after 11 iterations                              
    ============================================================
          Variable      CoefficienStd. Errort-Statistic  Prob.            
    ============================================================
             C          -0.358680   0.162995  -2.200559   0.0303          
           AR(1)         0.523056   0.054485   9.600060   0.0000          
           AR(2)         0.384275   0.073637   5.218478   0.0000          
           AR(3)        -0.940735   0.052948  -17.76712   0.0000          
           MA(1)         0.818634   0.089641   9.132368   0.0000          
    ============================================================
    R-squared            0.975970    Mean dependent var-0.483771          
    Adjusted R-squared   0.974925    S.D. dependent var 5.763586          
    S.E. of regression   0.912671    Akaike info criter-0.132588          
    Sum squared resid    76.63316    Schwarz criterion  0.000129          
    Log likelihood      -126.2065    F-statistic        934.1233          
    Durbin-Watson stat   1.924115    Prob(F-statistic)  0.000000          
    ============================================================
    Inverted AR Roots      .73 -.   .73+.68i      -.94                    
    Inverted MA Roots         -.82                                        
    ============================================================
    
    
    
                       Correlogram of Residuals
    ==============================================================
    Date: 04/21/98   Time: 16:57                                            
    Sample: 4 100                                                           
    Included observations: 97                                               
    Q-statistic probabilities adjusted for 4 ARMA term(s)                   
    ==============================================================
     Autocorrelation Partial Correlation  AC     PAC  Q-Stat Prob           
    ==============================================================
          . | .     |      . | .     |  1 0.021 0.021 0.0461                
          . | .     |      . | .     |  2-0.029-0.029 0.1310                
          .*| .     |      .*| .     |  3-0.135-0.134 1.9965                
          . |*.     |      . |*.     |  4 0.122 0.129 3.5240                
          . | .     |      . | .     |  5 0.024 0.011 3.5859 0.058          
          . | .     |      . | .     |  6 0.000-0.013 3.5859 0.166          
          . |*.     |      . |*.     |  7 0.101 0.142 4.6840 0.196          
          . |*.     |      . |*.     |  8 0.133 0.120 6.6018 0.158          
          .*| .     |      .*| .     |  9-0.096-0.112 7.6122 0.179          
          . | .     |      . | .     | 10-0.018 0.031 7.6495 0.265          
    ==============================================================
    
    This looks pretty good. Only the Q-Statistic at 5 lags and 1 degree of freedom has a p-value below .1. The adjusted r-square is essentially unchanged when we drop the MA(3) term so this is our preferred model.

  3. Third Example

    IDENT(5) Y3
                          Correlogram of Y3
    ==============================================================
    Date: 04/21/98   Time: 17:09                                            
    Sample: 1 100                                                           
    Included observations: 100                                              
    ==============================================================
     Autocorrelation Partial Correlation  AC     PAC  Q-Stat Prob           
    ==============================================================
      ******| .     |  ******| .     |  1-0.723-0.723 53.804 0.000          
          . |*******|      . |****** |  2 0.897 0.785 137.60 0.000          
       *****| .     |      . |*.     |  3-0.672 0.190 185.08 0.000          
          . |****** |      . |**     |  4 0.835 0.209 259.19 0.000          
       *****| .     |      . | .     |  5-0.635 0.051 302.54 0.000          
    ==============================================================
    
    Given the big spikes in the first two positions of both plots lets try starting again with MA(1) and MA(2) and see where it leads us.
    ============================================================
    LS // Dependent Variable is Y3                                        
    Date: 04/21/98   Time: 17:10                                          
    Sample: 1 100                                                         
    Included observations: 100                                            
    Convergence achieved after 20 iterations                              
    ============================================================
          Variable      CoefficienStd. Errort-Statistic  Prob.            
    ============================================================
             C          -0.174127   0.154511  -1.126956   0.2625          
           MA(1)        -0.645419   0.055408  -11.64849   0.0000          
           MA(2)         0.720077   0.066320   10.85755   0.0000          
    ============================================================
    R-squared            0.640556    Mean dependent var-0.191290          
    Adjusted R-squared   0.633144    S.D. dependent var 2.389969          
    S.E. of regression   1.447571    Akaike info criter 0.769315          
    Sum squared resid    203.2598    Schwarz criterion  0.847470          
    Log likelihood      -177.3596    F-statistic        86.43050          
    Durbin-Watson stat   2.123049    Prob(F-statistic)  0.000000          
    ============================================================
    Inverted MA Roots      .32+.7   .32 -.78i                             
    ============================================================
    
    
    
                       Correlogram of Residuals
    ==============================================================
    Date: 04/21/98   Time: 17:10                                            
    Sample: 1 100                                                           
    Included observations: 100                                              
    Q-statistic probabilities adjusted for 2 ARMA term(s)                   
    ==============================================================
     Autocorrelation Partial Correlation  AC     PAC  Q-Stat Prob           
    ==============================================================
          .*| .     |      .*| .     |  1-0.062-0.062 0.4013                
          . |****   |      . |****   |  2 0.486 0.484 25.018                
          **| .     |     ***| .     |  3-0.290-0.320 33.867 0.000          
          . |*****  |      . |*****  |  4 0.645 0.629 78.089 0.000          
          . | .     |      . |*.     |  5-0.029 0.110 78.178 0.000          
    ==============================================================
    
    Given the big spike at the 2nd lag, lets try adding an AR(2) term.
    ============================================================
    LS // Dependent Variable is Y3                                        
    Date: 04/21/98   Time: 17:12                                          
    Sample(adjusted): 3 100                                               
    Included observations: 98 after adjusting endpoints                   
    Convergence achieved after 11 iterations                              
    ============================================================
          Variable      CoefficienStd. Errort-Statistic  Prob.            
    ============================================================
             C          -0.174180   0.792690  -0.219734   0.8266          
           AR(2)         0.929372   0.045696   20.33793   0.0000          
           MA(1)        -0.307308   0.103296  -2.975027   0.0037          
           MA(2)        -0.097869   0.113819  -0.859866   0.3921          
    ============================================================
    R-squared            0.857380    Mean dependent var-0.216506          
    Adjusted R-squared   0.852829    S.D. dependent var 2.404179          
    S.E. of regression   0.922313    Akaike info criter-0.121781          
    Sum squared resid    79.96218    Schwarz criterion -0.016272          
    Log likelihood      -129.0887    F-statistic        188.3652          
    Durbin-Watson stat   1.954877    Prob(F-statistic)  0.000000          
    ============================================================
    Inverted AR Roots          .9      -.96                               
    Inverted MA Roots          .5      -.19                               
    ============================================================
    
    
    
                       Correlogram of Residuals
    ==============================================================
    Date: 04/21/98   Time: 17:12                                            
    Sample: 3 100                                                           
    Included observations: 98                                               
    Q-statistic probabilities adjusted for 3 ARMA term(s)                   
    ==============================================================
     Autocorrelation Partial Correlation  AC     PAC  Q-Stat Prob           
    ==============================================================
          . | .     |      . | .     |  1 0.006 0.006 0.0039                
          . | .     |      . | .     |  2-0.044-0.044 0.2028                
          . | .     |      . | .     |  3 0.054 0.054 0.5012                
          . |*.     |      . |*.     |  4 0.110 0.107 1.7560 0.185          
          . |*.     |      . |*.     |  5 0.144 0.149 3.9264 0.140          
          . | .     |      . | .     |  6-0.015-0.007 3.9495 0.267          
          . | .     |      . | .     |  7-0.034-0.034 4.0709 0.396          
          . |*.     |      . |*.     |  8 0.128 0.102 5.8653 0.320          
          . | .     |      . | .     |  9-0.012-0.045 5.8800 0.437          
          . |*.     |      . |*.     | 10 0.073 0.071 6.4753 0.485          
    ==============================================================
    
    This gives us white noise. However, the MA(2) term is now insignificant. Lets try dropping it:
    ============================================================
    LS // Dependent Variable is Y3                                        
    Date: 04/21/98   Time: 17:13                                          
    Sample(adjusted): 3 100                                               
    Included observations: 98 after adjusting endpoints                   
    Convergence achieved after 7 iterations                               
    ============================================================
          Variable      CoefficienStd. Errort-Statistic  Prob.            
    ============================================================
             C          -0.183495   0.659115  -0.278396   0.7813          
           AR(2)         0.910669   0.045020   20.22829   0.0000          
           MA(1)        -0.371488   0.095248  -3.900233   0.0002          
    ============================================================
    R-squared            0.856361    Mean dependent var-0.216506          
    Adjusted R-squared   0.853337    S.D. dependent var 2.404179          
    S.E. of regression   0.920720    Akaike info criter-0.135064          
    Sum squared resid    80.53393    Schwarz criterion -0.055933          
    Log likelihood      -129.4378    F-statistic        283.1891          
    Durbin-Watson stat   1.871770    Prob(F-statistic)  0.000000          
    ============================================================
    Inverted AR Roots          .9      -.95                               
    Inverted MA Roots          .37                                        
    ============================================================
    
    
    
                       Correlogram of Residuals
    ==============================================================
    Date: 04/21/98   Time: 17:13                                            
    Sample: 3 100                                                           
    Included observations: 98                                               
    Q-statistic probabilities adjusted for 2 ARMA term(s)                   
    ==============================================================
     Autocorrelation Partial Correlation  AC     PAC  Q-Stat Prob           
    ==============================================================
          . | .     |      . | .     |  1 0.050 0.050 0.2477                
          .*| .     |      .*| .     |  2-0.107-0.109 1.4086                
          . | .     |      . | .     |  3 0.014 0.025 1.4281 0.232          
          . |*.     |      . |*.     |  4 0.130 0.118 3.1917 0.203          
          . |*.     |      . |*.     |  5 0.135 0.130 5.1263 0.163          
          . | .     |      . | .     |  6-0.016-0.003 5.1524 0.272          
          . | .     |      . | .     |  7-0.044-0.023 5.3645 0.373          
          . |*.     |      . |*.     |  8 0.130 0.115 7.1965 0.303          
          . | .     |      . | .     |  9 0.005-0.045 7.1993 0.408          
          . | .     |      . |*.     | 10 0.065 0.082 7.6734 0.466          
    ==============================================================
    
    Very often more than one model will fit a time series equally well. After considerable experimentation with various models I found that ARIMA(1,0,3) model worked very well for this series.
    ============================================================
    LS // Dependent Variable is Y3                                        
    Date: 04/21/98   Time: 17:18                                          
    Sample(adjusted): 5 100                                               
    Included observations: 96 after adjusting endpoints                   
    Convergence achieved after 12 iterations                              
    ============================================================
          Variable      CoefficienStd. Errort-Statistic  Prob.            
    ============================================================
             C          -0.188639   0.595493  -0.316778   0.7521          
           AR(4)         0.804694   0.072112   11.15901   0.0000          
           MA(1)        -0.292846   0.086185  -3.397900   0.0010          
           MA(2)         0.791667   0.072370   10.93920   0.0000          
           MA(3)        -0.270966   0.085920  -3.153713   0.0022          
    ============================================================
    R-squared            0.859192    Mean dependent var-0.226304          
    Adjusted R-squared   0.853003    S.D. dependent var 2.428292          
    S.E. of regression   0.931011    Akaike info criter-0.092289          
    Sum squared resid    78.87720    Schwarz criterion  0.041270          
    Log likelihood      -126.7882    F-statistic        138.8180          
    Durbin-Watson stat   2.069957    Prob(F-statistic)  0.000000          
    ============================================================
    Inverted AR Roots          .9   .00 -.95   .00+.95i      -.95         
    Inverted MA Roots          .3  -.02 -.90  -.02+.90i                   
    ============================================================
    
    
    
                       Correlogram of Residuals
    ==============================================================
    Date: 04/21/98   Time: 17:19                                            
    Sample: 5 100                                                           
    Included observations: 96                                               
    Q-statistic probabilities adjusted for 4 ARMA term(s)                   
    ==============================================================
     Autocorrelation Partial Correlation  AC     PAC  Q-Stat Prob           
    ==============================================================
          . | .     |      . | .     |  1-0.039-0.039 0.1513                
          . | .     |      . | .     |  2-0.011-0.013 0.1634                
          . | .     |      . | .     |  3 0.029 0.028 0.2505                
          . |*.     |      . |*.     |  4 0.072 0.074 0.7827                
          . |*.     |      . |*.     |  5 0.090 0.097 1.6197 0.203          
          . | .     |      . | .     |  6 0.000 0.009 1.6197 0.445          
          . | .     |      . | .     |  7-0.051-0.054 1.8963 0.594          
          . |*.     |      . |*.     |  8 0.087 0.072 2.7091 0.608          
          . | .     |      . | .     |  9-0.040-0.049 2.8817 0.718          
          . |*.     |      . |*.     | 10 0.145 0.141 5.1813 0.521          
    ==============================================================