A preview tool helps builders visualize 3D printed objects | MIT News

Designers, builders, and others often use 3D printing to quickly prototype a range of practical things, from movie theaters to medical devices. Accurate print previews are essential for users to know that the invention will perform as expected.

But the previews produced by most 3D printing software focus on functionality rather than aesthetics. The printed object may end up with a different color, texture, or blurring than the user expected, resulting in multiple reprints that waste time, effort, and resources.

To help users see what the product will look like, researchers from MIT and elsewhere have developed an easy-to-use preview tool that prioritizes appearance.

Users upload a screenshot of the object from their 3D printing software, along with one image of the printed object. From this input, the system automatically generates a rendering of what the generated object might look like.

The powerful artificial intelligence system, called VisiPrint, is designed to work with a range of 3D printing software and can handle any virtual prototype. It looks not only at the color of the material, but also at the gloss, flexibility, and how the nuances of the manufacturing process affect the appearance of the object.

Such aesthetic-oriented previews may be especially useful in areas such as dentistry, by helping doctors to ensure that temporary crowns and bridges match the appearance of the patient’s teeth, or architecture, to help designers in evaluating the visual impact of the models.

“3D printing can be a very wasteful process. Some studies estimate that a third of the material used goes directly to the landfill, usually from prototypes that the user ends up throwing away. To make 3D printing sustainable, we want to reduce the number of efforts needed to get the desired prototype. The user should not have to pay for the job before he has printed everything,” said Maxine. Perroni-Scharf, a graduate student in electrical engineering and computer science (EECS) and lead author of the paper on VisiPrint.

He is joined on paper by Faraz Faruqi, fellow EECS graduate student; Raul Hernandez, MIT undergraduate; SooYeon Ahn, graduate student at Gwangju Institute of Science and Technology; Szymon Rusinkiewicz, professor of computer science at Princeton University; William Freeman, Thomas and Gerd Perkins Professor of EECS at MIT and member of the Computer Science and Artificial Intelligence Laboratory (CSAIL); and senior author Stefanie Mueller, associate professor of EECS and Mechanical Engineering at MIT, and a member of CSAIL. The research will be presented at the ACM CHI Conference on Human Factors in Computing Systems.

Intuitive aesthetics

The researchers focused on modeling fused deposition modeling (FDM), the most common form of 3D printing. In FDM, the printing filament is melted and blown through a nozzle to fabricate the object one layer at a time.

Producing an accurate aesthetic preview is challenging because the melting and extrusion process can change the appearance of the object, as can the height of each deposited layer and the path followed by the nozzle during fabrication.

VisiPrint uses two AI models that work together to overcome those challenges.

VisiPrint’s preview is based on two inputs: a screenshot of the digital design from the user’s 3D printing software (called “cut” software), and an image of the printed object, which can be taken from an online source or taken from a printed sample.

From this input, a computer vision model extracts features from a sample of objects that are important to the object’s appearance.

It feeds those features into a generative AI model that combines the geometry and structure of the object, while incorporating a so-called “cutting pattern” that the nozzle will follow as it ejects each layer.

The key to the researchers’ method is a special way of doing the situation. This involves carefully adjusting the inner workings of the model to guide it, so it follows the cutting pattern and complies with the parameters of the 3D printing process.

Their configuration method uses a depth map that preserves the object’s shape and shading, and an edge map that shows the interior contours and boundaries of the structure.

“If you don’t have the right balance of these two things, you can use bad geometry or the wrong cutting pattern. We had to be careful to combine them in the right way,” says Perroni-Scharf.

A user-centered system

The team also produced a user-friendly interface where one can upload the required images and check the preview.

The VisiPrint interface enables advanced printers to adjust many settings, such as the influence of certain colors on the final appearance.

Ultimately, the aesthetic preview is intended to complement the functional preview generated by the cutting software, as VisiPrint does not estimate printability, performance feasibility, or the probability of failure.

To test VisiPrint, researchers conducted a user study that asked participants to compare the program to other methods. Almost all participants said it provided a better overall appearance and consistency of text and printed material.

In addition, VisiPrint’s preview process took about a minute on average, which was twice as fast as any competitive method.

“VisiPrint really shines compared to other AI interfaces. If you give a very common AI model the same screenshots, it might randomly change the shape or use the wrong cropping pattern because it didn’t have a precise setting,” he said.

In the future, the researchers want to deal with artifacts that may occur if the model preview is too detailed. They also want to add features that allow users to customize parts of the printing process in addition to the color of the item.

“It is important to think about the way we do things. We need to continue to strive to develop methods that reduce waste. To do so, this marriage of AI with the physical process is an exciting area for future work,” said Perroni-Scharf.

“‘What you see is what you get’ was the main thing that made desktop publishing ‘happen’ in the 1980s, as it allowed users to find what they wanted in the first place. It’s time to get WYSIWYG for 3D printing again. VisiPrint is a good step in this,” said Patrick Baudisch, professor, professor, associate professor of Computer Science.

This research was funded, in part, by an MIT Morningside Academy for Design Fellowship and an MIT MathWorks Fellowship.