In 2007, Ramesh Raskar was one of the most promising young researchers at the Mitsubishi Electric Research Labs in Cambridge, Mass. Four years earlier, Technology Review had honored him as one of its top innovators under 35, and while he was steadily publishing academic papers, his name was also on dozens of patents, several of which had found their way into Mitsubishi products. The job was lucrative and left him plenty of time for a personal life.
That winter, however, he surprised technology observers by announcing that he’d accepted an appointment to the MIT Media Lab for the following spring. Despite a cut in pay, he’d have to work longer hours, piling administrative and teaching duties on top of his research. But to Raskar, the move made sense.
“I was having a great time at MERL, but what I realized was that, as great a place as it is, at any industrial research lab, people only appreciate inventing new things in a field everybody understands,” he says today. “What I’m really passionate about is creating whole new fields.”
Raskar was born in India, in a predominately agrarian part of the state of Maharashtra. Unable to afford college, his father had enlisted in the Indian army out of high school, rising through the ranks and eventually earning a degree in electronics engineering. “Everyone else in our family is farmers,” Raskar says. “Nobody on my father’s or mother’s side of the family has even a high school education. But my father, being an electronics engineer, made sure all his children focused on education rather than farming.”
Nonetheless, Raskar went to high school in his rural hometown, rather than commuting 10 miles to a larger city with better schools, as many of his neighbors with academic ambitions did. His last year in high school, Raskar, like everyone else his age, sat for a series of standardized tests that are the sole determinant of university placements in India. He finished first out of the roughly 500,000 students in Maharashtra — the state that happens to include Mumbai. “Everybody was shocked in my town when I was first, and my photo was splashed across newspapers statewide,” Raskar says.
As an undergraduate at the College of Engineering in Pune, Raskar majored in electronics and telecommunication but developed an interest in computer graphics. “I saw ‘Jurassic Park,’ which came out at the time, and I said, ‘Wow, I want to do special effects like “Jurassic Park,”’” Raskar says.
So for his PhD, Raskar went to the University of North Carolina, which had one of the premier computer-graphics departments in the United States. “It was always a dream back then to go to the U.S. for higher studies,” Raskar says. “There were very few opportunities in India.” For his PhD thesis, Raskar developed a prototype of the first true 3-D videoconferencing system — meaning that moving around the display changed the perspective on whatever it depicted — in a project called “Office of the Future."
A camera with four flashes instead of one can produce images with harsh shadows thrown from multiple directions, which makes it much easier for software to determine the boundaries between objects.
Image: Ramesh Raskar
At MERL, Raskar became intrigued by the idea that novel hardware and software systems designed to complement each other could yield much more efficient solutions to computer-vision problems than either could offer on its own. Take, for instance, the problem of identifying boundaries between objects in digital images, which is trivial for humans but astronomically hard for computers. Raskar and his MERL colleagues developed a camera that has four flash bulbs at its four corners rather than just one at one corner. When the camera’s shutter button is depressed, the four bulbs go off in rapid sequence, producing four images with harsh shadows thrown at four different angles. The shadows provide much more information about object boundaries than would a single well-exposed image, and Raskar developed software that can use that information to produce what look like very accurate line drawings of the scene framed by the camera.
Like Raskar’s early interest in graphics, the multiflash camera had its roots in a seminal pop-culture fusion of computer animation and film: the 1985 video of the song “Take on Me” by the Norwegian band A-Ha. To convert live-action film into animated line drawings, the video’s producers used a technique called rotoscoping, which had been around for decades, but which 1980s computer technology was making much more practical.
“During my PhD, I sent an email to my whole research group saying, ‘We should be creating cool videos that look like A-Ha,’” Raskar says. “And they all kind of ignored me, saying, ‘Hey, you are a new immigrant. This is a 15-year-old video.’”
Raskar got some measure of vindication, however, when he demonstrated his multiflash-camera system at Siggraph, the premier computer graphics conference, and his display booth received a visit from Michael Patterson, the A-Ha video’s animator. “We were showing the live output of people dancing and so on,” Raskar says. “He said, ‘That’s impossible. It used to take us an hour to just do one frame.’”
The multiflash camera, Raskar says, is now the subject of a large development project at Mitsubishi, as is another of his MERL prototypes, a camera whose shutter closes several times during a single exposure, which makes it much easier to computationally remove blur from the resulting photo. Yet a third project, a low-cost version of the type of dome-shaped interactive display system used for flight simulators, was, he says, one of the few MERL research projects to yield an original commercial product.
But while he was at MERL, Raskar had dreamed up two projects that were even more ambitious, and they’re the ones that brought him to the Media Lab. One is a camera that can see around corners: It bounces ultrashort bursts of laser light off a rigid surface — the wall opposite an open door, for instance — at several different angles and measures the time it takes the light to return to the camera. Raskar’s Media Lab research group has shown how to use that information to piece together 3-D images of objects beyond the camera’s line of sight. But in order to measure the time of flight of particles of light, Raskar’s group and that of Moungi Bawendi, the Lester Wolfe Professor in Chemistry at MIT, had to develop a camera that can take a trillion exposures a second. In another ongoing project, Raskar and his colleagues have used that camera to capture video of a single pulse of light traveling through a one-liter Coke bottle.
Raskar’s other central long-term project is the development of a garment studded with sensors and actuators (electromechanical muscles) that can not only gauge the wearer’s motions during some physical task — such as practicing a golf swing or doing physical therapy following a stroke — but actually nudges him or her toward the proper execution. To Raskar’s surprise, progress on that project has been slower than on the camera that sees around corners.
But in the meantime, his Media Lab research group has produced a dizzying array of innovative systems that combine novel optics with novel algorithms, among them an LED bar code that holds thousands of times as much information as a conventional bar code but can be read at a distance of 13 feet; a computer display that is itself a camera capable of capturing three-dimensional data; a more energy-efficient and realistic glasses-free 3-D display; and a simple test for cataracts that can clip on onto a cell phone.
“I’ve done more in the last three years since I came here than I did in the previous 10 years,” Raskar says. “Creative freedom is very intoxicating.”
3 Questions with Ramesh Raskar
Video: Melanie Gonick
That winter, however, he surprised technology observers by announcing that he’d accepted an appointment to the MIT Media Lab for the following spring. Despite a cut in pay, he’d have to work longer hours, piling administrative and teaching duties on top of his research. But to Raskar, the move made sense.
“I was having a great time at MERL, but what I realized was that, as great a place as it is, at any industrial research lab, people only appreciate inventing new things in a field everybody understands,” he says today. “What I’m really passionate about is creating whole new fields.”
Raskar was born in India, in a predominately agrarian part of the state of Maharashtra. Unable to afford college, his father had enlisted in the Indian army out of high school, rising through the ranks and eventually earning a degree in electronics engineering. “Everyone else in our family is farmers,” Raskar says. “Nobody on my father’s or mother’s side of the family has even a high school education. But my father, being an electronics engineer, made sure all his children focused on education rather than farming.”
Nonetheless, Raskar went to high school in his rural hometown, rather than commuting 10 miles to a larger city with better schools, as many of his neighbors with academic ambitions did. His last year in high school, Raskar, like everyone else his age, sat for a series of standardized tests that are the sole determinant of university placements in India. He finished first out of the roughly 500,000 students in Maharashtra — the state that happens to include Mumbai. “Everybody was shocked in my town when I was first, and my photo was splashed across newspapers statewide,” Raskar says.
As an undergraduate at the College of Engineering in Pune, Raskar majored in electronics and telecommunication but developed an interest in computer graphics. “I saw ‘Jurassic Park,’ which came out at the time, and I said, ‘Wow, I want to do special effects like “Jurassic Park,”’” Raskar says.
So for his PhD, Raskar went to the University of North Carolina, which had one of the premier computer-graphics departments in the United States. “It was always a dream back then to go to the U.S. for higher studies,” Raskar says. “There were very few opportunities in India.” For his PhD thesis, Raskar developed a prototype of the first true 3-D videoconferencing system — meaning that moving around the display changed the perspective on whatever it depicted — in a project called “Office of the Future."
A camera with four flashes instead of one can produce images with harsh shadows thrown from multiple directions, which makes it much easier for software to determine the boundaries between objects.
Image: Ramesh Raskar
Like Raskar’s early interest in graphics, the multiflash camera had its roots in a seminal pop-culture fusion of computer animation and film: the 1985 video of the song “Take on Me” by the Norwegian band A-Ha. To convert live-action film into animated line drawings, the video’s producers used a technique called rotoscoping, which had been around for decades, but which 1980s computer technology was making much more practical.
“During my PhD, I sent an email to my whole research group saying, ‘We should be creating cool videos that look like A-Ha,’” Raskar says. “And they all kind of ignored me, saying, ‘Hey, you are a new immigrant. This is a 15-year-old video.’”
Raskar got some measure of vindication, however, when he demonstrated his multiflash-camera system at Siggraph, the premier computer graphics conference, and his display booth received a visit from Michael Patterson, the A-Ha video’s animator. “We were showing the live output of people dancing and so on,” Raskar says. “He said, ‘That’s impossible. It used to take us an hour to just do one frame.’”
The multiflash camera, Raskar says, is now the subject of a large development project at Mitsubishi, as is another of his MERL prototypes, a camera whose shutter closes several times during a single exposure, which makes it much easier to computationally remove blur from the resulting photo. Yet a third project, a low-cost version of the type of dome-shaped interactive display system used for flight simulators, was, he says, one of the few MERL research projects to yield an original commercial product.
But while he was at MERL, Raskar had dreamed up two projects that were even more ambitious, and they’re the ones that brought him to the Media Lab. One is a camera that can see around corners: It bounces ultrashort bursts of laser light off a rigid surface — the wall opposite an open door, for instance — at several different angles and measures the time it takes the light to return to the camera. Raskar’s Media Lab research group has shown how to use that information to piece together 3-D images of objects beyond the camera’s line of sight. But in order to measure the time of flight of particles of light, Raskar’s group and that of Moungi Bawendi, the Lester Wolfe Professor in Chemistry at MIT, had to develop a camera that can take a trillion exposures a second. In another ongoing project, Raskar and his colleagues have used that camera to capture video of a single pulse of light traveling through a one-liter Coke bottle.
Raskar’s other central long-term project is the development of a garment studded with sensors and actuators (electromechanical muscles) that can not only gauge the wearer’s motions during some physical task — such as practicing a golf swing or doing physical therapy following a stroke — but actually nudges him or her toward the proper execution. To Raskar’s surprise, progress on that project has been slower than on the camera that sees around corners.
But in the meantime, his Media Lab research group has produced a dizzying array of innovative systems that combine novel optics with novel algorithms, among them an LED bar code that holds thousands of times as much information as a conventional bar code but can be read at a distance of 13 feet; a computer display that is itself a camera capable of capturing three-dimensional data; a more energy-efficient and realistic glasses-free 3-D display; and a simple test for cataracts that can clip on onto a cell phone.
“I’ve done more in the last three years since I came here than I did in the previous 10 years,” Raskar says. “Creative freedom is very intoxicating.”
3 Questions with Ramesh Raskar
Video: Melanie Gonick
