Software

1. Time Series Forecasting with Recurrent Neural NetworksNN3 Competition Mahmoud Abou-NasrResearch & Advanced EngineeringFord Motor CompanyEmail: mabounas@ford.com

2. Software • NTOOL Software Package developed in FORD, used for training the networks.

3. RMLP Architecture • Typically 1-4R-2-1L • One input node • Four fully recurrent nonlinear (bipolar sigmoid) nodes in the first hidden layer • 2 nonlinear nodes in the second hidden layer • One linear output node

4. Training Details • EKF multi-stream training, with typically 25 streams. • Each trajectory/stream is of length P, where P is no longer than half the number of points N in the series. • The input for training the network is taken from the actual series for P-M points, and from the network output for the last M points (M is the number of points to be predicted). • Switching logic is internal to the network. • Typically training time: 2 minutes per network • MSE error function.

5. P varies depending on the length of the series. • For a short series: P is 35 or about 0.5 N • For a long series: P is 60 or about 0.4 N

6. First P-M Training Steps From Actual Series RMLP From Network Output Last M Training Steps RMLP

7. P-M=42 M=18 P=60 End Start N = 143 Typical Training Stream For a Long Series

8. Ensemble of Networks • Maximum of ten networks of the same architecture were used to form an ensemble . • The trained networks were embedded in one architecture, with an output averaging node. • The networks used in the ensemble were the only networks trained. (They were not selected from a larger universe of trained networks).