Iclr2020: Compression based bound for non-compressed network
![](https://cdn.slidesharecdn.com/ss_thumbnails/iclr2020public-200422141800-thumbnail.jpg?width=640&height=640&fit=bounds)
Iclr2020: Compression based bound for non-compressed network: unified generalization error analysis of large compressible deep neural network - Download as a PDF or view online for free
1) The document presents a new compression-based bound for analyzing the generalization error of large deep neural networks, even when the networks are not explicitly compressed.
2) It shows that if a trained network's weights and covariance matrices exhibit low-rank properties, then the network has a small intrinsic dimensionality and can be efficiently compressed.
3) This allows deriving a tighter generalization bound than existing approaches, providing insight into why overparameterized networks generalize well despite having more parameters than training examples.
![](https://ars.els-cdn.com/content/image/1-s2.0-S0031320323001450-gr4.jpg)
A gradient optimization and manifold preserving based binary neural network for point cloud - ScienceDirect
![](https://media.springernature.com/full/springer-static/image/art%3A10.1038%2Fs41928-023-01054-3/MediaObjects/41928_2023_1054_Fig1_HTML.png)
Design principles for lifelong learning AI accelerators
![](https://image.slidesharecdn.com/iclr2020public-200422141800/85/iclr2020-compression-based-bound-for-noncompressed-network-unified-generalization-error-analysis-of-large-compressible-deep-neural-network-5-320.jpg?cb=1667961725)
Iclr2020: Compression based bound for non-compressed network: unified generalization error analysis of large compressible deep neural network
![](https://cdn.slidesharecdn.com/ss_thumbnails/cambridge20150915-160407010651-thumbnail.jpg?width=640&height=640&fit=bounds)
Higher Order Fused Regularization for Supervised Learning with Grouped Parameters
![](https://image.slidesharecdn.com/metalearninglowrankcovariancefactorsforenergy-baseddeterministicuncertainty-220325050604/85/meta-learning-low-rank-covariance-factors-for-energybased-deterministic-uncertainty-7-320.jpg?cb=1667675121)
Meta Learning Low Rank Covariance Factors for Energy-Based Deterministic Uncertainty
![](https://i1.rgstatic.net/publication/374320932_Information_Bound_and_Its_Applications_in_Bayesian_Neural_Networks/links/65181d121e2386049debd2b4/largepreview.png)
PDF) Information Bound and Its Applications in Bayesian Neural Networks
NeurIPS 2021 Lossy Compression For Lossless Prediction Paper, PDF, Data Compression
![](https://cdn.slidesharecdn.com/ss_thumbnails/howdoesunlabeleddataimprovegeneralizationinself-training-220310081549-thumbnail.jpg?width=640&height=640&fit=bounds)
How does unlabeled data improve generalization in self training
![](https://cdn.slidesharecdn.com/ss_thumbnails/ngboost-230620184603-8047fc6c-thumbnail.jpg?width=336&fit=bounds)
ICLR2021 (spotlight)] Benefit of deep learning with non-convex noisy gradient descent
Neural material (de)compression – data-driven nonlinear dimensionality reduction
![](https://pbs.twimg.com/media/EVPn4KmUMAILCzJ.jpg)
YK (@yhkwkm) / X
![](https://image.slidesharecdn.com/iclr2021-210331133549/85/iclr2021-spotlight-benefit-of-deep-learning-with-nonconvex-noisy-gradient-descent-9-320.jpg?cb=1666696498)
ICLR2021 (spotlight)] Benefit of deep learning with non-convex noisy gradient descent