1 / 15

Meta-controlled Boltzmann Machine toward Accelerating the Computation

Meta-controlled Boltzmann Machine toward Accelerating the Computation. Tran Duc Minh (*), Junzo Watada (**) (*) Institute Of Information Technology-Viet Nam Academy of Science & Technology (**) Graduate School of Information, Production and System, Waseda University, Japan. TOKYO 02/2004.

mliss
Download Presentation

Meta-controlled Boltzmann Machine toward Accelerating the Computation

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Meta-controlled Boltzmann Machine toward Accelerating the Computation Tran Duc Minh (*), Junzo Watada (**) (*) Institute Of Information Technology-Viet Nam Academy of Science & Technology (**) Graduate School of Information, Production and System, Waseda University, Japan TOKYO 02/2004

  2. CONTENT • Introduction • The portfolio selection problem • Inner behaviors of the Meta-controlled Boltzmann machine • Some hints on accelerating the Meta-controlled Boltzmann machine • Conclusion at Meiji University 25/02/2004

  3. Introduction • H. Markowitz proposed a method to allocate an amount of funds to plural stocks for investment • Model of Meta-controlled Boltzmann Machine • The ability of Meta-controlled Boltzmann Machine in solving the quadratic programming problem at Meiji University 25/02/2004

  4. The portfolio selection problem Maximize Minimize Subject to and with mi {0, 1}, i = 1, .., n where ijdenotes a covariance between stocks i and j, i is an expected return rate of stock i, xiis investing rate to stock i, n denotes the total number of stocks and S denotes the number of stocks selected, and finally, mi denotes a selection variable of investing stocks. at Meiji University 25/02/2004

  5. The portfolio selection problem Convert the objective function into the energy functions of the two components that are Meta-controlling layer (Hopfield Network) and the Lower-layer (Boltzmann Machine) as described below: Meta-Controlling layer Lower Layer where Ku, Kl are weights of the expected return rate for each layer and si is the output value of the ith unit of the Meta-Controlling layer. at Meiji University 25/02/2004

  6. Algorithm of the Meta-controlled Boltzmann machine Step 1. Set each parameter to its initial value. Step 2. Input the values of Ku and Kl. Step 3. Execute the Meta-controlling layer. Step 4. If the output value of a unit in the Meta-controlling layer is 1, add some amount of value to the corresponding unit in the lower layer. Execute the lower layer. Step 5. After executing the lower layer the constant number of times, decreases the temperature. Step 6. If the output value is sufficiently large, add a certain amount of value to the corresponding unit in the Meta-controlling layer. Step 7. Iterate from Step 3 to Step 6 until the temperature reaches the restructuring temperature. Step 8. Restructure the lower layer using the selected units of the Meta-controlling layer. Step 9. Execute the lower layer until reaching at the termination. at Meiji University 25/02/2004

  7. Inner behaviors of the Meta-controlled Boltzmann machine • Some times, the Hopfield layer may converge to a local minimum but the disturb values make it to get over • The changes of Meta layer’s energy function are very small, while the lower layer’s energy function’s is quite large • The number of cycles to execute the Meta layer is much smaller than the cycles for the lower layer • Similar to the simulated annealing that we will “try to go downhill most of the time instead of always going downhill” • The time to converge is much shorter than a conventional Boltzmann machine • All the neurons that are “encouraged” will be selected before the system goes to the final computation. at Meiji University 25/02/2004

  8. Chart of behaviors of Meta-controlled Boltzmann Machine Disturb back value = 80% at Meiji University 25/02/2004

  9. Chart of behaviors of Meta-controlled Boltzmann Machine Disturb back value = 1 % at Meiji University 25/02/2004

  10. Comparison of computing time between a Conventional Boltzmann machine and a Meta-controlled Boltzmann Machine (1286 units) at Meiji University 25/02/2004

  11. Some hints on accelerating the Meta-controlled Boltzmann machine • Trying to use only a layer of Boltzmann Machine, modify the algorithm of original Boltzmann Machine by removing the discouraged units before goes into final computation. • The modification on Meta layer by replacing deterministic neurons by stochastic neurons • Some ideas employed from the other kinds of neural networks: using multi-layers structure, heuristic modification of the learning procedure. at Meiji University 25/02/2004

  12. at Meiji University 25/02/2004

  13. at Meiji University 25/02/2004

  14. CONCLUSION • The training algorithms for neural network models are varied from many simple but efficient ideas. • The trend of accelerating algorithms is focused mainly on heuristic modification and numeric optimization technique, i.e. toward the faster convergence of algorithms whereas keeping the correctness for them. • The Meta-controlled Boltzmann Machine can be used to solve quadratic programming problems. • Future works: • Try the model with other quadratic programming problem. • Modify the original algorithm toward accelerating computation. at Meiji University 25/02/2004

  15. THANK YOU! at Meiji University 25/02/2004

More Related