The manufacturing industry (largely) welcomed artificial intelligence with open arms. Less of the dull, dirty, and dangerous? Say no more. Planning for mechanical assemblies still requires more than scratching out some sketches, of course — it’s a complex conundrum that means dealing with arbitrary 3D shapes and highly constrained motion required for real-world assemblies.
Human engineers, understandably, need to jump in the ring and manually design assembly plans and instructions before sending the parts to assembly lines, and this manual nature translates to high labor costs and the potential for error.
In a quest to ease some of said burdens, researchers from MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL), Autodesk Research, and Texas A&M University came up with a method to automatically assemble products that’s accurate, efficient, and generalizable to a wide range of complex real-world assemblies. Their algorithm efficiently determines the order for multipart assembly, and then searches for a physically realistic motion path for each step.
The team cooked up a Spartan-level large-scale dataset with thousands of physically valid industrial assemblies and motions to test their method. The proposed method is capable of solving almost all of them, especially outperforming previous methods by a large margin on rotational assemblies, like screws and puzzles. Also, it’s a bit of a speed demon in that it solves 80-part assemblies within several minutes.
“Instead of one assembly line specifically designed for one specific product, if we can automatically figure out ways to sequence and move, we can use a fully adaptive setup,” says Yunsheng Tian, a PhD student at MIT CSAIL and lead author on the paper. “Maybe one assembly line can be used for tons of different products. We think of this as low-volume, high-mixed assembly, opposed to traditional high-volume, low-mixed assembly, which is very specific to a certain product.”
Given the objective of assembling a screw attached to a rod, for example, the algorithm would find the assembly strategy through two stages: disassembly and assembly. The disassembly planning algorithm searches for a collision-free path to disassemble the screw from the rod. Using physics-based simulation, the algorithm applies different forces to the screw and observes the movement. As a result, a torque rotating along the rod’s central axis moves the screw to the end of the rod, then a straight force pointing away from the rod separates the screw and the rod. In the assembly stage, the algorithm reverses the disassembly path to get an assembly solution from individual parts.
“Think about IKEA furniture — it has step-by-step instructions with the little white book. All of those have to be manually authored by people today, so now we can figure out how to make those assembly instructions,” says Karl D.D. Willis, a senior research manager at Autodesk Research. “You can imagine how, for people designing products, this could be helpful for building up those types of instructions. Either it's for people, as in laying out these assembly plans, or it could be for some kind of robotic system right down the line.”
The disassemble/assemble dance
With current manufacturing, in a factory or assembly line, everything is typically hard-coded. If you want to assemble a given product, you have to precisely control or program instructions to assemble or disassemble a product. Which part should be assembled first? Which part should be assembled next? And how are you going to assemble this?
Previous attempts have been mostly limited to simple assembly paths, like a very straight translation of parts — nothing too complicated. To move beyond this, the team used a physics-based simulator — a tool commonly used to train robots and self-driving cars — to guide the search for assembly paths, which makes things much easier and more generalizable.
“Let’s say you want to disassemble a washer from the shaft, which is very tightly geometrically assembled. The status quo would simply try to sample a bunch of different ways to separate them, and it’s very possible you can’t create a simple path that’s perfectly collision-free. Using physics, you don't have this limitation. You can try, for example, adding a simple downward force, and the simulator will find the correct motion to disassemble the washer from the shaft,” says Tian.
While the system handled rigid objects with ease, it remains in future work to plan for soft, deformable assemblies.
One avenue of work the team is looking to explore is making a physical robotic setup to assemble items. This would require more work in terms of robotic control and planning to be integrated with the team’s system, as a step toward their broader goal: to make an assembly line that can adaptively assemble everything without humans.
“The long-term vision here is, how do you take any object in the world and be able to either put that together from the parts, using automation and robotics?,” says Willis. “Inversely, how do we take any object in the world that's made up of many different types of materials and pull it apart so that we can recycle and get them into the correct waste streams? The step we're taking is looking at how we can use some advanced simulation to be able to begin to pull apart those parts, and eventually get to the point where we can test that in the real world.”
“Assembly is a longstanding challenge in the robotics, manufacturing, and graphics communities,” says Yashraj Narang, senior robotics research scientist at NVIDIA. “This work is an important step forward in simulating mechanical assemblies and solving assembly planning problems. It proposes a method that is a clever combination of solving the computationally-simpler disassembly problem, using force-based actions in a custom simulator for contact-rich physics, and using a progressively-deepening search algorithm. Impressively, the method can discover an assembly plan for a 50-part engine in a few minutes. In the future, it will be exciting to see other researchers and engineers build upon this excellent work, perhaps allowing robots to perform the assembly operations in simulation and then transferring those behaviors to real-world industrial settings.”
MIT professor and CSAIL principal investigator Wojciech Matusik is a senior author on the paper, with PhD students Yunsheng Tian, Jie Xu (now a research scientist at NVIDIA) and Yichen Li also noted as CSAIL authors. Research scientists from Autodesk Research Jieliang Luo, Hui Li, Karl D.D. Willis, and assistant professor of computer science at Texas A&M University Shinjiro Sueda also worked on the paper. The team will present their findings at SIGGRAPH Asia 2022, with the paper also being published in ACM Transactions on Graphics. Their research was supported in part by the National Science Foundation.
