Cluster Computing https://doi.org/10.1007/s10586-018-1839-2

Super-resolution compressed sensing imaging algorithm based on sub-pixel shift Bing Xu1 · Xiaoping Zhang2 · Xianjun Wu3 Received: 20 December 2017 / Revised: 10 January 2018 / Accepted: 11 January 2018 © Springer Science+Business Media, LLC, part of Springer Nature 2018

Abstract At present, some digital signal processing methods have attracted more and more attention in improving the resolution of images. Sub-pixel shift has been widely applied in improving the resolution of compressed sensing imaging system. The resolution of the compressed sensing imaging system is limited by pixel size of the modulation system. To overcome the resolution limitation of compressed sensing imaging system, a sub-pixel shift method is proposed to enhance the resolution of modulation information and achieve super-resolution images by compressed sensing imaging system. The principle of the proposed method is introduced and the proposed method is verified using numerical simulations. Experimental results revealed that the proposed method can effectively improve the resolution of compressed sensing imaging system and obtain super-resolution image information. Additionally, the signal to noise ratio of restoration results is positively related to the sample size. Keywords Compressed sensing · Super-resolution · Sub-pixel shift

1 Introduction As conventional imaging systems approaching super high pixels per inch (PPI), compressed sensing imaging system [1–6], which is a single-pixel imaging system, has also been intensively studied. In the compressed sensing imaging system, light intensity information obtained by collecting lens is randomly modulated by the modulation system, the overall light intensity is detected by the single-pixel detector, and the 2D information of imaging scene is obtained by information retrieval using compressed sensing algorithms. As an essential component of compressed sensing imaging sys-

B

Xianjun Wu [email protected] Bing Xu [email protected] Xiaoping Zhang [email protected]

1

School of Computer and Electronic Information, Guangdong University of Petrochemical Technology, Maoming, China

2

School of Mathematics and Statistics, The University of Sheffield, Sheffield S3 7RH, UK

3

School of Computing Center, Guangdong University of Petrochemical Technology, Maoming, China

tem, the light modulation system can be transmission LCD screen or micro-reflector array [7,8]. The algorithm currently used for compressed sensing imaging system is referred as compressed sensing algorithm. This algorithm can retrieve signals even if the sampling size does not meet the lower limit, thus providing fundamental principles for single-pixel imaging systems. The first passive single-pixel camera based on the compressed sensing algorithm has been proposed and intensively studied. Despite its slow development owing to limitations by the computer system in past years, the compressed sensing imaging system has attracted wide attentions [9–12]. The resolution is a key indicator for any imaging system [13,14], including compressed sensing imaging system. It has been demonstrated that the resolution of single-pixel imaging system is affected by the point spread function and the size of minimum modulation unit. The deconvolution method has been proposed to relieve the effects of point spread function, thus achieving high resolution images, while the limitation by the size of minimum modulation unit has not been overcome. To further enhance the resolution of compressed sensing imaging system and obtain high resolution images, the limitation by the size of minimum modulation unit in compressed sensing imaging systems must be overcome. Herein, a shift-based approach is proposed to achieve

123

Cluster Computing

super-resolution imaging by the compressed sensing imaging system, while maintaining its intrinsic advantages.

2 Improvement of compressed sensing imaging system resolution by sub-pixel shift To enhance resolution of imaging systems, reduction in modulation pixel size can be achieved by advances in manufacturing techniques [15–17]. However, this triggers increased manufacturing cost and higher requirements on sensitivity and signal-to-noise-ratio of detectors. Moreover, advances in manufacturing techniques cannot meet rapidly increasing requirements on imaging systems [18,19]. Therefore, numerical signal processing approaches have been employed to improve resolution of imaging systems. For instance, the subpixel shift approach has been widely applied to improve the resolution of area array imaging system. In this article, a subpixel shift approach is proposed to enhance the resolution of compressed sensing imaging system. Figure 1 shows the schematic diagram of a high resolution imaging system by area array system. A specific scene is shot using several different detectors so that subpixel shift is involved in each detector. The information obtained by different detectors, which is complementary to each other, is matching calibrated using the matching

algorithm to obtain high resolution images. For single-pixel imaging system, shift set-up needs to be installed on digital micro-mirror devices (DMD) to achieve shifts of numerical micro-reflectors in lateral and longitudinal directions. The shifts shall be smaller than the single pixel size of the micromirror devices. Super-resolution imaging was achieved by retrieval based on known shifts. Figure 2 shows the structure of the single-pixel super-resolution imaging system. The scene reflection information is converged on the DMD via lens, modulated by the micro-mirror devices, and then focalized on the single-pixel photodetector. For each group of modulation information, sub-pixel shift (in X or Y direction) of DMD are realized by shift set-up. Retrieval based on shift, modulation information and overall light intensity information are obtained to achieve super-resolution imaging. Define the modulation information as T, shifts of DMD (i.e., shifts of T) in X, Y and XY directions are designed. The overall light intensity information in each situation is regarded as I0 , IX , IY , and IXY . The scene information is defined as S. The information capture processes in these four situations can be described by the follow equations:

I0 =

ST0 ,

x,y

IX =

Fig. 1 Principle of high resolution imaging by sub-pixel shift in area array system

123

  x,y

STX ,

Cluster Computing

Fig. 2 Structure of single-pixel super-resolution imaging system

IY =



⇓

STY ,

I =

x,y

IXY =



STX Y ,

(1)

x,y

where T0 , TX , TY , and TX Y refer to modulation information distributions in case of no DMD shift, sub-pixel shift of numerical micro-reflector in X direction, sub-pixel shift of numerical micro-reflector in Y direction, and sub-pixel shift of numerical micro-reflector in XY direction, respectively. According to principles of single-pixel imaging system, the scene information resolution of imaging system is related to the pixel size of modulation information. The sub-pixel shift approach can realize further reduced pixel size of modulation information, as shown in Figure 3. The first column describes the four situations, the second column describes the modulation information distribution (red, green, blue, and purple sections refer to the pixel size of micro-reflectors) in different situations, the third column describes combined modulation information in different situations, which indicates that the pixel of modulation information has been significantly enhanced. Figure 3 shows the situation where the resolution was doubled. Accumulation of modulation information in four different situations leads to: I0 + I X + IY + I X Y =



ST0 +

x,y

+

 x,y



STX

x,y

STY +

 x,y

STX Y ,



S(T0 + TX + TY + TX Y ),

x,y

⇓ I =



ST ,

(2)

x,y

where T is the combined modulation information with higher resolution to achieve super-resolution imaging. Assume that the pixel count to be m, then m unknowns are present in the specific scene. The scene information is obtained based on m equation set developed by m incoherent measurements. In case of large PPI of imaging scene, each measurement is significantly time-consuming, thus limiting imaging efficiency. By employing scene information retrieval via compressed sensing algorithm, image sampling size can be effectively reduced and the imaging efficiency can be improved. Under certain base vectors, most natural scenes are sparse, meaning that the non-zero elements in certain transform matrices are few [20–22]. The compressed sensing algorithm is applicable for sparse or compressible signals. In most cases, signals after transform under certain vectors are sparse (e.g., wavelet transform, discrete cosine transform) and can be described by the following equation: S = ψα (3) where Ψ is sparse transform matrix, such as wavelet transform matrix, Fourier transform matrix, and gradient transform matrix. By substituting Eq. (3) into Eq. (2), the information of signal (I ) can be obtained by solving α:

123

Cluster Computing Fig. 3 Principle of super-resolution imaging by sub-pixel shift approach

αˆ = arg min α subject to I = T ψα

(4)

Currently, matching pursuit method, interior point method, and gradient method are employed to solve Eq. (5) and the result is substituted into Eq. (4) to obtain the final image (S) of the object. The sparse transform matrix is widely used in the compressed sensing algorithm. In most cases, the signal gradient is sparse, which is demonstrated by previous studies. The model for solution of gradient of discrete signals (S) is [14]: (5)

Fig. 4 Initial scene: a lateral resolution stripe, b longitudinal resolution stripe

To verify super-resolution imaging by the proposed algorithm, simulations of lateral and longitudinal resolution stripes were conducted. Figure 4a, b refer to initial scene of lateral and longitudinal resolution stripes, respectively. In Fig. 4a, the three resolution stripes from left to right were resolution stripes with 8, 4, and 2 pixels, respectively. Figure 4b illustrates longitudinal resolution stripes which are obtained by converting lateral resolution stripes in Fig. 4a. Assume that the minimum resolution is 4 × 4 as a result of limitation by pixel size of DMD in the single-pixel imaging system. The first group involves retrieval simulations of lateral resolution stripes in cases of sub-pixel shifts of 0, [0 1], [0 1

2], and [0 1 2 3] in X direction, respectively. Figure 5 summarizes retrieval results obtained in this experiment. Herein, Fig. 5a and e show the results obtained by the conventional methods. As observed, stripe plate with pixel size below 4 cannot be recognized due to limitation by the pixel size of DMD. By involving sub-pixel shifts, the imaging resolution is significantly enhanced as the scene information resolution increases as a result of increasing combined modulation information resolution. The second group involves retrieval simulations of longitudinal resolution stripes in cases of sub-pixel shifts of 0, [0 1], [0 1 2], and [0 1 2 3] in Y direction, respectively. Figure 6 summarizes retrieval results obtained in this experiment. Herein, Fig. 6a and e show the results obtained by the conventional methods. As observed, stripe plate with pixel size below 4 cannot be recognized due to limitation by the pixel

oˆ = arg min T V (O) subject to I = T S where TV refers to the gradient of discrete signals.

3 Results and discussion

123

Cluster Computing

Fig. 5 Retrieval results of lateral resolution stripe: a 0 in X direction, sample size = 200; b [0 1] in X direction, sample size = 200; c [0 1 2] in X direction, sample size = 200; d [0 1 2 3] in X direc-

tion, sample size = 200; e 0 in X direction, sample size = 600; f [0 1] in X direction, sample size = 600; g [0 1 2] in X direction, sample size = 600; h [0 1 2 3] in X direction, sample size = 600

Fig. 6 Retrieval results of longitudinal resolution stripe: a 0 in Y direction, sample size = 200; b [0 1] in Y direction, sample size = 200; c [0 1 2] in Y direction, sample size = 200; d [0 1 2 3] in Y direc-

tion, sample size = 200; e 0 in Y direction, sample size = 600; f [0 1] in Y direction, sample size = 600; g [0 1 2] in Y direction, sample size = 600; h [0 1 2 3] in Y direction, sample size = 600

size of DMD. By involving sub-pixel shifts, the imaging resolution is significantly enhanced as the scene information resolution increases as a result of increasing combined modulation information resolution. According to the results of the two groups of simulation experiments, it can be seen that the proposed method can effectively improve the resolution of the imaging system and

achieve super-resolution imaging. The accuracy of retrieval simulation results is positively related to the sample size. The imaging resolution is limited by the combined modulation information resolution and the distribution of optimized resolution is consistent with that of modulation information resolution.

123

Cluster Computing

4 Conclusions The pixel size of the modulation system limits the resolution of the compressed sensing imaging system. To further enhance the resolution of compressed sensing imaging system and obtain high resolution images, the limitation by the size of minimum modulation unit in compressed sensing imaging systems must be overcome. Based on super-resolution imaging achieved by conventional area array imaging system, a sub-pixel shift method is proposed to enhance the resolution of modulation information and achieve super-resolution images by compressed sensing imaging system. The principle of the proposed method is introduced and the proposed method is verified using numerical simulations. Experimental results revealed that the proposed method can effectively improve the resolution of compressed sensing imaging system and obtain superresolution image information. Acknowledgements This work is supported by Special Funds of Applied Science & Technology Research and Development of Guangdong Province, China (Grant: 2015B010128015).

References 1. Tao, C., Zhengwei, L., Jianli, W., et al.: Imaging system of single pixel camera based on compressed sensing. Opt. Precis. Eng. 11, 2523–2530 (2012) 2. Zhiyang, Q., Yarong, Y.: Application of compressed sensing on image processing. J. Yunnan Univ. 39(S1), 63–69 (2017) 3. Sun, B., Edgar, M.P., BOWMAN, R., et al.: 3D computational imaging with single-pixel detectors. Science 340(6134), 844–7 (2013) 4. Shuo, Z., Jie, W., Jincheng, W., et al.: Simple calculation method for three-dimensional imaging based on compressed sensing. Acta Opt. Sin. 01, 84–90 (2013) 5. Yanpeng, M., Yanan, W., Yikun, W., et al.: Study of single-pixel detection computational imaging technology based on compressive sensing. Acta Opt. Sin. 33, 1–7 (2013) 6. Jing, C., Yongtian, W.: Research of the compressive imaging technology. Laser Optoelectron. Prog. 03, 15–22 (2012) 7. Shichao, Z., Simin, L., Guang, Y., et al.: Optimization of single molecules axial localization precision in 3D stochastic optical reconstruction microscopy. Acta Photonica Sin. 44(10), 1–6 (2015) 8. Jiangqi, C., Jinwen, M.: The improved particle swarm optimization algorithm based compressive sensing. J. Signal Process. 33(4), 488–495 (2017) 9. AlSaafin, W., Villena, S., Vega, M.: Compressive sensing super resolution from multiple observations with application to passive millimeter wave images. Dig. Signal Process. 50, 180–190 (2016) 10. Jie, Zhang, Chao, Luo, Xiaoping, Shi, et al.: High resolution astronomical image denoising based on compressed sensing. J. Harbin Inst. Technol. 49(4), 22–27 (2017) 11. Jiang, Y., Miao, S.W., Luo, H.Z., et al.: Improved search algorithm for compressive sensing image recovery based on Lp norm. J. Image Graph. 22(4), 0435–0442 (2017) 12. Lu, W., Liu, Y.Z., Wang, D.S.: Efficient feedback scheme based on compressed sensing in MIMO wireless networks. Comput. Electr. Eng. 39(6), 1587–1600 (2013)

123

13. Shi, D., Huang, J., Wang, F., et al.: Enhancing resolution of singlepixel imaging system. Opt. Rev. 22, 1352–1359 (2015) 14. Shi, D., Fan, C., Shen, H., et al.: Reconstruction of spatially misaligned and turbulence degraded images. Opt. Lasers Eng. 50(5), 72–81 (2012) 15. Du, Y., Zhang, H., Zhao, M.: Faster super-resolution imaging of high density molecules via a cascading algorithm based on compressed sensing. Opt. Express 23(14), 18563–18576 (2015) 16. Renk, X.: Super-resolution images fusion via compressed sensing and low-rank matrix decomposition. Infrared Phys. Technol. 68, 61–68 (2015) 17. Dong, C., Loy, C.C., He, K., et al.: Image super-resolution using deep convolutional networks. IEEE Trans. Pattern Anal. Mach. Intell. 38(2), 295–307 (2016) 18. Yanpeng, S., Shi, Z., Lele, Q., et al.: Subspace projection based compressive sensing SFGPR imaging algorithm. J. Northeast. Univ. (Natural Sci.) 38(6), 789–792 (2017) 19. Jiancheng, Z., Li, F.: A method of image denoising based on compressive sensing. J. North China Univ. Technol. 24(1), 1–7 (2012) 20. Xinlei, L., Biao, L.: Review on progress of real-time THz sensing and imaging technology. Laser Optoelectron. Prog. 09, 55–60 (2012) 21. Ren, Y.M., Zhang, Y.N., Li, Y.: Advances and perspective on compressed sensing and application on image processing. Acta Autom. Sin. 40(8), 1563–1571 (2014) 22. Wenze, S., Zhihui, W.: Advances and perspectives on compressed sensing theory. J. Image Graph. 01, 1–12 (2012)

Bing Xu received the B.S. degree in computer science from Zhanjiang Ocean University, Zhanjiang, P.R. China in 2003, and M.S. degree in computer software from Guangdong University of Technology in 2006, and Ph.D. degree in ocean remote sensing and information technology from Nanjing Normal University, Nanjing, P.R. China, in 2013. Now he is an associate professor of the Computer Science, Guangdong University of Petrochemical Technology, P.R. China. His current interests include wireless sensor network and intelligent algorithm, etc.

Xiaoping Zhang received the B.S. degree in computer science from Fundamental Mathematics, Anhui University, Hefei, P.R. China in 2001, and M.S. degree in Nanjing University of Aeronautics & Astronautics , Nanjing, P.R.China in 2008, and currently studying for the Ph.D. degree in Statistics science, Nanjing Normal University, Nanjing, P.R. China. Now she is an lecturer of the Department of Applied Mathematics, Nanjing Technology University, P.R.China . Her current interests include the statistics, mathematics in combination and the quantitative analysis, etc.

Cluster Computing Xianjun Wu received the B.S. degree in computer science from Zhanjiang Ocean University, Zhanjiang, P.R. China in 2003, and M.S. degree in software engineering from Huazhong University of Science and Technology, Wuhang, P.R. China in 2005. Now he is an lecturer of the Computer Science, Guangdong University of Petrochemical Technology, P.R. China. His current interests include wireless sensor network and intelligent algorithm, etc.

123