Spatial and angular resolution enhancement of light fields using convolutional neural networks

Creative Commons License

Gul M. S. K., GÜNTÜRK B. K.

IEEE Transactions on Image Processing, vol.27, no.5, pp.2146-2159, 2018 (SCI-Expanded) identifier identifier identifier

  • Publication Type: Article / Article
  • Volume: 27 Issue: 5
  • Publication Date: 2018
  • Doi Number: 10.1109/tip.2018.2794181
  • Journal Name: IEEE Transactions on Image Processing
  • Journal Indexes: Science Citation Index Expanded (SCI-EXPANDED), Scopus
  • Page Numbers: pp.2146-2159
  • Keywords: Light field, super-resolution, convolutional neural network
  • Istanbul Medipol University Affiliated: Yes

Abstract

Light field imaging extends the traditional photography by capturing both spatial and angular distribution of light, which enables new capabilities, including post-capture refocusing, post-capture aperture control, and depth estimation from a single shot. Micro-lens array (MLA) based light field cameras offer a cost-effective approach to capture light field. A major drawback of MLA based light field cameras is low spatial resolution, which is due to the fact that a single image sensor is shared to capture both spatial and angular information. In this paper, we present a learning based light field enhancement approach. Both spatial and angular resolution of captured light field is enhanced using convolutional neural networks. The proposed method is tested with real light field data captured with a Lytro light field camera, clearly demonstrating spatial and angular resolution improvement.