An empirical study on improving the speed and generalization of neural networks using a parallel circuit approach
One of the common problems of neural networks, especially those with many layers, consists of their lengthy training time. We attempt to solve this problem at the algorithmic level, proposing a simple parallel design which is inspired by the parallel circuits found in the human retina. To avoid larg...
Lưu vào:
| Tác giả chính: | , , |
|---|---|
| Định dạng: | Bài trích |
| Ngôn ngữ: | English |
| Chủ đề: | |
| Truy cập trực tuyến: | https://thuvienso.hoasen.edu.vn/handle/123456789/10895 |
| Từ khóa: |
Thêm từ khóa bạn đọc
Không có từ khóa, Hãy là người đầu tiên gắn từ khóa cho biểu ghi này!
|
| id |
oai:https:--thuvienso.hoasen.edu.vn:123456789-10895 |
|---|---|
| record_format |
dspace |
| spelling |
oai:https:--thuvienso.hoasen.edu.vn:123456789-108952023-10-17T04:09:41Z An empirical study on improving the speed and generalization of neural networks using a parallel circuit approach Phan, Kien Tuong Maul, Tomas Henrique Vu, Tuong Thuy Neural networks Parallel circuits Problem decomposition Backpropagation One of the common problems of neural networks, especially those with many layers, consists of their lengthy training time. We attempt to solve this problem at the algorithmic level, proposing a simple parallel design which is inspired by the parallel circuits found in the human retina. To avoid large matrix calculations, we split the original network vertically into parallel circuits and let the backpropagation algorithm flow in each subnetwork independently. Experimental results have shown the speed advantage of the proposed approach but also point out that this advantage is affected by multiple dependencies. The results also suggest that parallel circuits improve the generalization ability of neural networks presumably due to automatic problem decomposition. By studying network sparsity, we partly justified this theory and proposed a potential method for improving the design. 2016 Article https://thuvienso.hoasen.edu.vn/handle/123456789/10895 en Pp. 780–796 application/pdf International Journal of Parallel Programming. Volume 45 |
| institution |
Trường Đại học Hoa Sen |
| collection |
DSpaceHS |
| language |
English |
| topic |
Neural networks Parallel circuits Problem decomposition Backpropagation |
| spellingShingle |
Neural networks Parallel circuits Problem decomposition Backpropagation Phan, Kien Tuong Maul, Tomas Henrique Vu, Tuong Thuy An empirical study on improving the speed and generalization of neural networks using a parallel circuit approach |
| description |
One of the common problems of neural networks, especially those with many layers, consists of their lengthy training time. We attempt to solve this problem at the algorithmic level, proposing a simple parallel design which is inspired by the parallel circuits found in the human retina. To avoid large matrix calculations, we split the original network vertically into parallel circuits and let the backpropagation algorithm flow in each subnetwork independently. Experimental results have shown the speed advantage of the proposed approach but also point out that this advantage is affected by multiple dependencies. The results also suggest that parallel circuits improve the generalization ability of neural networks presumably due to automatic problem decomposition. By studying network sparsity, we partly justified this theory and proposed a potential method for improving the design. |
| format |
Article |
| author |
Phan, Kien Tuong Maul, Tomas Henrique Vu, Tuong Thuy |
| author_facet |
Phan, Kien Tuong Maul, Tomas Henrique Vu, Tuong Thuy |
| author_sort |
Phan, Kien Tuong |
| title |
An empirical study on improving the speed and generalization of neural networks using a parallel circuit approach |
| title_short |
An empirical study on improving the speed and generalization of neural networks using a parallel circuit approach |
| title_full |
An empirical study on improving the speed and generalization of neural networks using a parallel circuit approach |
| title_fullStr |
An empirical study on improving the speed and generalization of neural networks using a parallel circuit approach |
| title_full_unstemmed |
An empirical study on improving the speed and generalization of neural networks using a parallel circuit approach |
| title_sort |
empirical study on improving the speed and generalization of neural networks using a parallel circuit approach |
| url |
https://thuvienso.hoasen.edu.vn/handle/123456789/10895 |
| work_keys_str_mv |
AT phankientuong anempiricalstudyonimprovingthespeedandgeneralizationofneuralnetworksusingaparallelcircuitapproach AT maultomashenrique anempiricalstudyonimprovingthespeedandgeneralizationofneuralnetworksusingaparallelcircuitapproach AT vutuongthuy anempiricalstudyonimprovingthespeedandgeneralizationofneuralnetworksusingaparallelcircuitapproach AT phankientuong empiricalstudyonimprovingthespeedandgeneralizationofneuralnetworksusingaparallelcircuitapproach AT maultomashenrique empiricalstudyonimprovingthespeedandgeneralizationofneuralnetworksusingaparallelcircuitapproach AT vutuongthuy empiricalstudyonimprovingthespeedandgeneralizationofneuralnetworksusingaparallelcircuitapproach |
| _version_ |
1794706369903853568 |