1 / 6

Optimal Brain Surgeon Algorithm for Neural Network Pruning

Learn about the Optimal Brain Surgeon Algorithm, how it reduces the complexity of neural networks, and its benefits such as quicker calculations and reduced storage space. Follow the steps to perform the algorithm, understand the tests performed, and explore the results obtained using the wine data. Conclude with insights on the algorithm's effectiveness and potential improvements for real-world applications.

lautrey
Download Presentation

Optimal Brain Surgeon Algorithm for Neural Network Pruning

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Optimal Brain Surgeon ECE 539 Final Project Mark Slosarek

  2. Background • Optimal Brain Surgeon Algorithm (OBS) is a pruning algorithm • Reduces weights to reduce overall complexity of network

  3. Benefits • A pruned network has several benefits • Quicker Calculations • Less Storage Space required • A pruned network should not have significantly more error than an non-pruned network

  4. Steps to Perform OBS • Train the given MLP to minimize mean-square error • Calculate the cost of the equation • Compute the inverse Hessian • Find the smallest Saliency • If Saliency is much smaller than mean-square, delete that weight and repeat for next weight, other go to next step • Update all weights

  5. Tests Performed and Results • To test the effectiveness of the OSB, I used the wine data from the homework • The pruned network contained approximately 60% fewer weights • Results from both networks, however results of pruning were very inconclusive • The results were not very consistant

  6. Conclusion • A pruned network can save much space and time • My algorithm is not perfect, and could be recoded to be more efficient • Results not noticeable in simple networks, but in real world problems, could be results could be great

More Related