Neural Networks for Xilinx FPGA
Xilinx Zync, Xilinx KriaSOM
GrovetyUpscaleNet10
Upscaling (x4) and enhancing images with Xilinx Zync UltraScale+ 256х256 to 1024х1024
Architecture
Results
| Metrics | Bicubic interpolation | Nearest neighbor | Neural Network 9(64)-5(32)-5(1) | Neural Network 9(64)-3(64)-1(32)-5(1) | Neural Network 7(32)-5(16)-5(1) | Neural Network 7(32)-5(8)-5(1) | |||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| Bicubic | Nearest | Bicubic | Nearest | Bicubic | Nearest | Bicubic | Nearest | ||||
| PSNR (Peak Signal-to-Noise Ratio) | 25.1 | 21.69 | 25.66 | 24.85 | 24.7 | 25.02 | 26.1896 | 24.8989 | 26.01164 | - | |
| SSIM (Structure Similarity) | 0.735 | 0.61 | 0.815 | 0.780 | 0.737 | 0.714 | 0.8172 | 0.7815 | 0.81351 | - | |
| FPS @ Xilinx ZCU102 (256x256 → 1024x1024) |
Speed ( without pre-processing and post-processing), theoretical limit | - | 7.5 | 8.1 | 23.7 | Ecpected: 43 FPS
Real: 23.4 FPS |
|||||
| Real speed (pre-processing, post-processing) | > 4.2 | > 4.6 | ~11.0 | ~11.5 | ~11.0 | ~11.5 | |||||
Plans
- Optimization for Xilinx KriaSOM
- Optimization to improve image quality
- Optimize inference for real-time applications
Useful links
- Visualizing and Understanding Convolutional Networks
- Trainable Nonlinear Reaction Diffusion: A Flexible Framework for Fast and Effective Image Restoration
- Limits on Super-Resolution and How to Break Them
- ImageNet Classification with Deep Convolutional Neural Networks
- Image Super-Resolution with Fast Approximate Convolutional Sparse Coding
- Image Super-Resolution Using Very Deep Residual Channel Attention Networks