Real-time robust direct and indirect photon separation with polarization imaging

Separation of reflections, such as superimposed scenes behind and in front of a glass window and semi-diffuse surfaces, permits the imaging of objects that are not in direct line of sight and field of view. Existing separation techniques are often computational intensive, time consuming, and not easily applicable to real-time, outdoor situations. In this work, we apply Stokes algebra and Mueller calculus formulae with a novel edge-based correlation technique to the problem of separating reflections in the visible, near infrared, and long wave infrared wavelengths. Our method exploits spectral information and patch-wise operation for improved robustness and can be applied to optically smooth reflecting and partially transmitting surfaces such as glass and optically semi-diffuse surfaces such as floors, glossy paper, and white painted walls. We demonstrate robust image separation in a variety of indoor and outdoor scenes and real-time acquisition using a division-of-focal plane polarization camera.