How can artificial intelligence move off the screen and into something we can physically touch and interact with?
That question forms the basis of class 4.043/4.044 (Interaction Intelligence), an MIT course focused on designing a new class of AI-driven interactive objects. Known as large language objects (LLOs), these virtual interfaces extend large language models to the real world. Their behavior can be customized for specific people or applications, and their interactions can change from simple to very complex – providing meaningful support to both novice and expert users.
“I realized that, although powerful, these new forms of intelligence remain ignorant of the world outside of language,” said Marcelo Coelho, associate professor of this practice at the MIT Department of Architecture, who has been teaching the design studio for several years and directing the Design Intelligence Lab. “They don’t have real-time, contextual understanding of our surroundings, physical experiences, and social relationships to be truly intelligent. On the contrary, LLOs are physical and interact in real-time with their physical environment. The course is an attempt to address this gap and develop a new type of design discipline for the age of AI.”
Given an assignment to design an interactive device they would want in their lives, students Jacob Payne and Ayah Mahmoud focused on the kitchen. While each enjoyed cooking and baking, their design inspiration came from the first home computer: the Honeywell 316 Kitchen Computer, marketed by Neiman Marcus in 1969. Priced at $10,000, there is no record of it ever being sold.
“It was an ambitious but impossible first attempt at a home kitchen computer,” said Payne, an architecture student. “Make reference to the interesting history of this project.”
“As someone who likes to learn how to cook – especially now, in college as an undergrad – the thought of designing something that makes cooking easier for those who may not have a culinary background and just want delicious food to satisfy their cravings was a great start for me,” said Mahmoud, chief design officer.
“We thought about the leftovers you have in the fridge or the pantry, and how AI can help you find new things that you might throw away,” Payne said.
Generative cuisine
The students designed their device – called the Kitchen Cosmo – with instructions to act as a “recipe generator.” Another challenge was to encourage LLM to always acknowledge real-world cooking parameters, such as heating, time, or temperature. Another issue they addressed was for LLM to identify accurate flavor and spice profiles in regional and cultural dishes around the world to support a variety of cuisines. Problem solving included taste test recipes produced by Kitchen Cosmo. Not all original recipes produce a winning dish.
“There were a lot of little things that the AI wasn’t capable of understanding,” Mahmoud said. “The LLM requires a fundamental understanding of human taste to create delicious food.”
They have fine-tuned their device to accommodate the many ways people approach food preparation. Is this for breakfast, lunch, dinner, or dinner? How much have you developed as a chef? How much time do you have to prepare food? How many meals will you make? Food preferences are also planned, along with the type of atmosphere or vibe you want to achieve. Are you feeling nostalgic, or in the mood to celebrate? There is a call for that.
“This selection was the focus of this device because we were eager to see how LLM would interpret the attributes as ideas and use them to change the type of recipe results we would get,” said Payne.
Unlike many AI collaborations that are often invisible, Payne and Mahmoud wanted their device to be more of a “partner” in the kitchen. The tactile interface was purposefully designed to streamline interactions, giving users physical control over how the AI responded.
“While I’ve worked with electronics and hardware before, this project pushed me to assemble components with a level of precision and refinement that feels very close to a production-ready device,” Payne said of the coursework.
Retro and red
After their electronic work was completed, the students designed a series of models using cardboard until they arrived at a final look, which Payne described as “retro.” The body was designed with 3D modeling software and printed. In a nod to Honeywell’s first computer, they painted it red.
A slim, rectangular device about 18 inches tall, the Kitchen Cosmo has a dangling webcam to monitor ingredients set out on the counter. It translates this into a recipe that takes into account common spices and seasonings that are common in most households. An integrated thermal printer delivers a tear-off printed recipe. Recipes can be stored in a plastic container on its base.
While Kitchen Cosmo makes a modest splash in design magazines, both readers have ideas for where to take future iterations.
Payne would like to see it “take advantage of the vast amount of data we have in the kitchen and use AI as an intermediary, giving tips on how to improve what you’re cooking at that moment.”
Mahmoud looks at how to prepare Kitchen Cosmo in his thesis. His classmates gave feedback to improve his skills. Another suggestion is to provide multi-person instructions that assign a few people the tasks required to complete the recipe. Another idea is to create a “learning mode” where a kitchen tool – for example, a paring knife – is placed in front of Kitchen Cosmo, and it gives instructions on how to use the tool. Mahmoud was also researching the history of food science.
“I would like to get a better handle on how to train AI to fully understand food so that it can combine recipes in a way that the user likes,” he said.
Having begun his MIT education as a geologist, Mahmoud’s pivot to design has been a revelation, he says. Each design class was inspiring. Coelho’s course was his first class to include design with AI. It refers to the oft-cited analogy of “drinking in firehou
“For the first time in that class I felt like I was drinking too much and I didn’t feel stressed, I could see myself doing long-term design, which I don’t think I would have said before in terms of technology.”
