TED Talks – Ramesh Raskar on “Femto-Photography”

TED Talks – Ramesh Raskar on “Femto-Photography”

How can Femto-photography see what is beyond the line of sight?

Femto-photography exploits the finite speed of light and works like an ultra-fast time of flight camera. In traditional photography, the speed of light is infinite and does not play a role. In our transient light transport framework, the finite amount of time light takes to travel from one surface to another provides useful information. The key contribution is a computational tool of transient reasoning for the inversion of light transport. The basic concept of a transient imaging camera can be understood using a simple example of a room with an open door. The goal here is to compute the geometry of the object inside the room by exploiting light reflected off the door. The user directs an ultra short laser beam onto the door and after the first bounce the beam scatters into the room. The light reflects from objects inside the room and again from the door back toward the transient imaging camera. An ultra-fast array of detectors measures the time profile of the returned signal from multiple positions on the door. We analyze this multi-path light transport and infer shapes of objects that are in direct sight as well as beyond the line of sight. The analysis of the onsets in the time profile indicates the shape; we call this the inverse geometry problem.

How is this related to computer vision and general techniques in scene understanding?

Our goal is to exploit the finite speed of light to improve image capture and scene understanding. New theoretical analysis coupled with emerging ultra-high-speed imaging techniques can lead to a new source of computational visual perception. We are developing the theoretical foundation for sensing and reasoning using transient light transport, and experimenting with scenarios in which transient reasoning exposes scene properties that are beyond the reach of traditional computer vision.

What is a transient imaging camera?

We measure how the room responds to a very short duration laser. So transient imaging uses a transient response rather than a steady state response. Two common examples are the impulse response and the step response. In a room sized environment, the rate of arrival of photons after such an impulse provides a transient response. A traditional camera, on the other hand, uses a steady state response.

What is new about the Femto-photography approach?

Modern imaging technology captures and analyzes real world scenes using 2D camera images. These images correspond to steady state light transport which means that traditional computer vision ignores the light multipath due to time delay in propagation of light through the scene. Each ray of light takes a distinct path through the scene which contains a plethora of information which is lost when all the light rays are summed up at the traditional camera pixel. Light travels very fast (~1 foot in 1 nano sec) and sampling light at these time scales is werasll beyond the reach of conventional sensors (the fastest video cameras have microsecond exposures). On the other hand, Femtosecond imaging techniques such as optical coherence tomography which do employ ultra-fast sensing and laser illumination cannot be used beyond millimeter sized biological samples. Moreover all of imaging systems to date are line of sight. We propose to combine the recent advances in ultra-fast light sensing and illumination with a novel theoretical framework to exploit the information contained in light multipath to solve impossible problems in real world scenes such as looking around corners and material sensing.

How can one take a photo of photons in motion at a trillion frames per second?

We use a pico-second accurate detector (single pixel). Another option is a special camera called a streak camera that behaves like an oscilloscope with corresponding trigger and deflection of beams. A light pulse enters the instrument through a narrow slit along one direction. It is then deflected in the perpendicular direction so that photons that arrive first hit the detector at a different position compared to photons that arrive later. The resulting image forms a “streak” of light. Streak tubes are often used in chemistry or biology to observe milimeter sized objects but rarely for free space imaging. See recent movies of photons in motion captured by our group at [Video]

What are the challenges?

The number of possible light multipath grows exponentially in the number of scene points. There exists no prior theory which models time delayed light propagation which makes the modeling aspect a very hard theoretical and computationally intense problem. Moreover, we intend to develop a practical imaging device using this theory and need to factor in real world limitations such as sensor bandwidth, SNR, new noise models etc. Building safe, portable, free-space functioning device using highly experimental optics such as Femtosecond lasers and sensitive picoseconds cameras is extremely challenging and would require pushing modern photonics and optics technology to its limits, creating new hardware challenges and opportunities. The current resolution of the reconstructed data is low, but it is sufficient to recognize shapes. But with higher time and space resolution, the quality will improve significantly.

How can endoscopes see beyond the line of sight?

Consider the constraints on diagnostic endoscopy. Great progress in imaging hardware has allowed a gradual shift from rigid to flexible to digital endoscopes. Digital scopes put image sensors directly at the tip of the scopes. However, there is a natural limit to their reach due to constraints in the dimensions of human body that leave very little room for guiding the imager assemblies. Making imagers smaller is challenging due to the diffraction limits posed on the optics as well as due to sensor-noise limits on the sensor pixel size. In many scenarios, we want to avoid the maze of cavities and serial traversal for examination. We want the precise location and size of a lesion when deciding for or against application of limited or extended surgical procedures. Ideally we should be to explore multitude of paths in a simultaneous and parallel fashion. We use transient imaging to mathematically invert the data available in light reflected in complex optical reflections. We can convert elegant optical and mathematical insights into unique medical tools.

How will these complicated instruments transition out of the lab?

The ultrafast imaging devices today are quite bulky. The laser sources and high speed cameras fit on a small optical bench and need to be carefully calibrated for triggering. However, there is a parallel research in femtosecond solid state lasers and they will greatly simplify the illumination source. Pico-second accurate single pixel detectors are now available for under $100. Building an array of such pixels is non-trivial but comparable to thermal-IR cameras. Nevertheless, in the short run, we are building applications where portability is not as critical. For endoscopes, the imaging and illumination can be achieved via coupled fibers.

At TEDGlobal back in June, Ramesh Raskar, an MIT professor, described an innovative kind of photography he calls “femto-photography”. Femto-photography uses special cameras that can capture images at trillions of frames per second- so fast that one can observe the movement of light through a medium. But, femto-photography is far more useful than just for creating stunning works of art. Because of the extremely sensitive and sophisticated circuitry inside this kind of cameras, Raskar and his team have been able to turn them into cameras that can look around corners.

This amazing technology could have limitless applications in car navigation, rescue planning, and robotics, and Raskar thinks femto-photography could also become the next new medical imaging modality. He envisions a new type of endoscope that won’t traverse your arteries or colon like a snake, but will snap pictures from a single point in the body, utilizing femto-photography to peer around the various folds and spaces.

Source : http://web.media.mit.edu/~raskar/cornar/

Related Posts Plugin for WordPress, Blogger...
Be Sociable, Share!

About the Author

has written 1822 posts on this blog.

Copyright © 2018 Medical Technology & Gadgets Blog MedicalBuy.net. All rights reserved.
Proudly powered by WordPress. Developed by Deluxe Themes