3-D scanning, with your smartphone

Traditionally, 3-D scanning has required expensive laser scanner equipment, complicated software, and technological expertise. 

But MIT spinout Viztu Technologies helped change that: Back in 2011, Viztu released software, free online, that essentially replaced expensive scanning hardware with personal cameras. This innovation led to a rapidly rising commercial enterprise that concluded with Viztu’s sale to a tech giant, which is now bringing the technology to the public worldwide.

Viztu’s flagship web service, Hypr3D, could rapidly generate digital 3-D models of an object (human or inanimate) or scene from a series of user-uploaded 2-D digital photos or videos, usually captured by digital cameras, smartphones, or webcams.

“We gave people the easiest scanner available: the cameras they already owned,” says Thomas Milnes PhD ’13, Viztu’s chief technology officer, who developed the software behind Hypr3D as part of his MIT dissertation. “Now it only takes a smartphone or digital camera a few minutes online to build a 3-D model and only a few moments more to send it off to be 3-D printed.”

Less than a year after launching Hypr3D, Viztu — co-founded by Milnes and CEO Michael “Ash” Martin MBA ’10 — boasted thousands of users and tens of thousands of model views and downloads.

But while Hypr3D was freely available, the company was making significant revenue: Big-name clients were contracting Viztu to tailor its technology to their own unique applications, from 3-D filmmaking to 3-D mapping.

Shortly after taking the grand prize in the “Innovative Web” track at the South by Southwest (SXSW) Accelerator competition in 2012, Viztu sold to 3-D Systems, one of the world’s leading 3-D printing companies. Since the acquisition, 3-D Systems has integrated Viztu’s software into its global Cubify consumer platform.

After helping 3-D Systems integrate the Viztu technology, Milnes returned to MIT to finish his PhD in mechanical engineering. He is now working with two other MIT alumni on another startup, OpenWater Power, developing a novel and nontoxic fuel cell for underwater vehicles that can outlast traditional lithium-ion batteries. Martin is still with 3-D Systems, running its consumer products division — including the Cubify software platform that includes much of Viztu’s original work.

A simple pipeline

To use Hypr3D, users would generally place an object on a visually textured material, such as newsprint — so the software could use the background for camera orientation — and snap five to 30 images, usually in a circle around the object, to capture it from all angles. About 15 minutes after uploading those photos to Hypr3D, users could view the model on a separate Web page — and all model files could be downloaded for free. Users could share and comment on models, request 3-D-printed versions from Viztu, or print their models at home.

Part of Viztu’s novelty comes from taking existing technologies and combining them into an easy-to-use, automated pipeline. All said, about 15 separate pieces of software — written in different programming languages and running on different operating systems — were part of the Hypr3D pipeline.

Milnes breaks the complicated process down into a few basic steps. First, the software determines the camera’s location. Then, it executes a kind of triangulation that computes the real-world position of every point in each image. This step creates a point cloud — scattered dots that represent the photographed object.

Then, through computational tessellation, the software creates a solid surface mesh over the points. Finally, the original image data is overlain on this surface mesh, making the model full-color and photo-realistic.

Much of the technology needed to create the point cloud and construct the mesh had already existed before Viztu; some of it came from the open-source community. But blending and layering the original image data on the mesh, for example, was uniquely Viztu. “Basically, the software takes the photos and pastes them, papier-mâché style, back on the model so that the model looks just like the original object,” Milnes says.

This required Viztu to develop a process for seamlessly integrating all images taken of an object to create a final “texture map” image. When you take side-by-side photos of the same object with a smartphone, Milnes explains, the color and brightness of each image will often be different, because cameras may automatically adjust exposure and white balance differently with each shot.

In work done primarily by intern Ami Patel ’12, MNG ’13, Viztu created a software component that “globally balanced” the colors of all the photos taken to create a single, seamless “texture map” to paste back on the mesh.

All this was automated for the user. “We had it to the point where you just put a bunch of images into a folder, and hit ‘Go,’ and on the other end you got this really nice model,” Milnes says.

Using ‘the hardware people carry’

Milnes and Martin launched Viztu shortly after collaborating on a project for MIT Sloan School of Management’s iTeams class, which brings together students from across MIT to develop commercial products.

The team was focused on a special kind of “light field” camera that is capable of taking 3-D images. This camera was the focus of Milnes’ PhD work: He was improving on earlier versions of this technology, which he’d worked on as an intern for mechanical engineering professor Doug Hart’s startup Brontes Technologies. Martin and three other students from MIT Sloan and Harvard Business School focused on identifying a market for the camera.

Ultimately, though, they decided that the market wasn’t ready for their camera. However, there was commercial potential in the software Milnes had developed to power the camera. “The attractive thing was that now you could do the 3-D scanning everyone wanted, but with hardware that people already had,” Milnes says.

After iTeams ended, Martin and Milnes entered the MIT $100K competition (finishing as semifinalists), and founded Viztu — making use of MIT’s Venture Mentoring Service for advising. Initially, the company was run out of Milnes’ apartment, with a high-powered computer and 3-D printer, but soon moved to office space in Cambridge.

There, user demand kept the company’s technology evolving. For instance, aerial-vehicle enthusiasts, who mounted cameras on their remote-controlled planes, “would give us hundreds of photos of a landscape and ask us to build a model from them,” Milnes says. “That’s about 10 times more data than we’d ever considered being able to handle.”

This led to tweaks of the software that could handle a higher volume of photos and multiple models processing simultaneously. These advancements came alongside a revamped web application to handle the increasing traffic.

Selling points

Soon, funding would come entirely, and quickly, from several large firms. A 3-D movie technology company, RealD Cinema, contracted Viztu to demonstrate how its software could be used to edit live-action 3-D video — just like computer-generated 3-D movies — by tweaking a movie’s frames in postproduction, instead of reshooting entire scenes. Viztu successfully created a working proof of concept. Another 3-D-mapping company, C3 Technologies (acquired by Apple in 2011), contracted Viztu to see how alternative methods of computation could help them recreate city-scale 3-D models.

Another client, Geomagic, wanted to incorporate Viztu’s software into its 3-D scanning software suite. Although Geomagic offered to buy Viztu — an offer Milnes and Martin gladly accepted — the deal eventually fell through.

However, this near-acquisition helped Martin and Milnes realize Viztu’s best path was acquisition. After winning at the SXSW festival in Austin, Tex., Viztu returned to Boston and “put their company up for sale,” Milnes says. Among others, they reached out to a contact at 3-D Systems Milnes had known from his time at Brontes.

Only a few days later, Milnes and Martin were in Los Angeles hashing out their final acquisition deal with 3-D Systems’ CEO, Abe Reichental — an acquisition that became official in July 2012.

Viztu’s path to acquisition was an unusual one, Milnes says, as the company was completely bootstrapped on client revenue and received its first acquisition offer in only 18 months. Additionally, the founders were full-time students for some or all of the company’s life.

But apart from providing a big payday, Milnes says, selling Viztu to a larger company with far more resources was ideal for the technology, as 3-D Systems could bring Viztu’s inexpensive alternatives to 3-D scanning to the masses.

“As a tech entrepreneur, you really want to see your technology make it out into the wider world,” Milnes says. “Viztu was all about democratizing 3-D scanning, and 3-D Systems’ purchase helped us do that in a really big way."