- 51 Views
- Uploaded on
- Presentation posted in: General

Meta-controlled Boltzmann Machine toward Accelerating the Computation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Meta-controlled Boltzmann Machine toward Accelerating the Computation

Tran Duc Minh (*), Junzo Watada (**)

(*) Institute Of Information Technology-Viet Nam Academy of Science & Technology

(**) Graduate School of Information, Production and System, Waseda University, Japan

TOKYO 02/2004

- Introduction
- The portfolio selection problem
- Inner behaviors of the Meta-controlled Boltzmann machine
- Some hints on accelerating the Meta-controlled Boltzmann machine
- Conclusion

at Meiji University 25/02/2004

- H. Markowitz proposed a method to allocate an amount of funds to plural stocks for investment
- Model of Meta-controlled Boltzmann Machine
- The ability of Meta-controlled Boltzmann Machine in solving the quadratic programming problem

at Meiji University 25/02/2004

Maximize

Minimize

Subject to and

with mi {0, 1}, i = 1, .., n

where ijdenotes a covariance between stocks i and j, i is an expected return rate of stock i, xiis investing rate to stock i, n denotes the total number of stocks and S denotes the number of stocks selected, and finally, mi denotes a selection variable of investing stocks.

at Meiji University 25/02/2004

Convert the objective function into the energy functions of the two components that are Meta-controlling layer (Hopfield Network) and the Lower-layer (Boltzmann Machine) as described below:

Meta-Controlling layer

Lower Layer

where Ku, Kl are weights of the expected return rate for each layer and si is the output value of the ith unit of the Meta-Controlling layer.

at Meiji University 25/02/2004

Step 1. Set each parameter to its initial value.

Step 2. Input the values of Ku and Kl.

Step 3. Execute the Meta-controlling layer.

Step 4. If the output value of a unit in the Meta-controlling layer is 1, add some amount of value to the corresponding unit in the lower layer. Execute the lower layer.

Step 5. After executing the lower layer the constant number of times, decreases the temperature.

Step 6. If the output value is sufficiently large, add a certain amount of value to the corresponding unit in the Meta-controlling layer.

Step 7. Iterate from Step 3 to Step 6 until the temperature reaches the restructuring temperature.

Step 8. Restructure the lower layer using the selected units of the Meta-controlling layer.

Step 9. Execute the lower layer until reaching at the termination.

at Meiji University 25/02/2004

- Some times, the Hopfield layer may converge to a local minimum but the disturb values make it to get over
- The changes of Meta layer’s energy function are very small, while the lower layer’s energy function’s is quite large
- The number of cycles to execute the Meta layer is much smaller than the cycles for the lower layer
- Similar to the simulated annealing that we will “try to go downhill most of the time instead of always going downhill”
- The time to converge is much shorter than a conventional Boltzmann machine
- All the neurons that are “encouraged” will be selected before the system goes to the final computation.

at Meiji University 25/02/2004

Chart of behaviors of Meta-controlled Boltzmann Machine

Disturb back value = 80%

at Meiji University 25/02/2004

Chart of behaviors of Meta-controlled Boltzmann Machine

Disturb back value = 1 %

at Meiji University 25/02/2004

Comparison of computing time between a Conventional Boltzmann machine and a Meta-controlled Boltzmann Machine (1286 units)

at Meiji University 25/02/2004

- Trying to use only a layer of Boltzmann Machine, modify the algorithm of original Boltzmann Machine by removing the discouraged units before goes into final computation.
- The modification on Meta layer by replacing deterministic neurons by stochastic neurons
- Some ideas employed from the other kinds of neural networks: using multi-layers structure, heuristic modification of the learning procedure.

at Meiji University 25/02/2004

at Meiji University 25/02/2004

at Meiji University 25/02/2004

- The training algorithms for neural network models are varied from many simple but efficient ideas.
- The trend of accelerating algorithms is focused mainly on heuristic modification and numeric optimization technique, i.e. toward the faster convergence of algorithms whereas keeping the correctness for them.
- The Meta-controlled Boltzmann Machine can be used to solve quadratic programming problems.
- Future works:
- Try the model with other quadratic programming problem.
- Modify the original algorithm toward accelerating computation.

at Meiji University 25/02/2004

THANK YOU!

at Meiji University 25/02/2004