Apple researchers have developed a new method for training large language models (LLMs) that seamlessly integrates both text and visual information.


The company’s findings, detailed in a research paper titled “MM1: Methods, Analysis & Insights from Multimodal LLM Pre-training,” showcase a new approach to creating more intelligent and flexible AI systems. By utilizing a diverse dataset comprising image-caption pairs, interleaved image-text documents, and text-only data, Apple’s claims that the MM1 model sets a new standard in AI’s ability to perform tasks such as image captioning, visual question answering, and natural language inference with a high level of accuracy.

Apple’s research focuses on the combination of different types of training data and model architectures, which enables the AI to understand and generate language based on a mix of visual and linguistic cues. This capability is vital for tasks that require a nuanced comprehension of the world, such as interpreting complex images or answering questions that involve visual elements.

The paper also highlights the MM1 model’s exceptional in-context learning abilities, particularly in the largest 30 billion parameter configuration of the model. This version apparently exhibits remarkable capabilities for multi-step reasoning over multiple images using few-shot “chain-of-thought” prompting, a technique that allows the AI to perform complex, open-ended problem solving based on minimal examples.

This research emerges as part of Apple’s broader initiative to enhance its AI capabilities amid growing competition. Earlier today, Bloomberg‘s Mark Gurman reported that Apple is in discussions with Google to license Google’s Gemini generative large-language models to power new features coming to the iPhone as part of iOS 18.

See also  More iOS 17 beta 6 details revealed (Video)

Popular Stories

Five Key Upgrades Coming to iPhone 16

The iPhone is Apple’s top-selling product, and it gets an update every year. In 2024, we’re expecting the iPhone 16 and iPhone 16 Pro lineup, with an arguably more interesting feature set than we got with the iPhone 15 and iPhone 15 Pro. Subscribe to the MacRumors YouTube channel for more videos. Capture Button All four iPhone 16 models are set to get a whole new button, which will be…

iPhone SE 4 Expected to Depreciate Heavily

Resale value trends suggest the iPhone SE 4 may not hold its value as well as Apple’s flagship models, according to SellCell. According to the report, Apple’s iPhone SE models have historically depreciated much more rapidly than the company’s more premium offerings. The third-generation iPhone SE, which launched in March 2022, experienced a significant drop in resale value, losing 42.6%…

Apple Said iPhone Driver’s Licenses Would Expand to These 8 U.S. States… Two Years Ago

In just four U.S. states, residents can add their driver’s license or ID to the Apple Wallet app on the iPhone and Apple Watch, providing a convenient and contactless way to display proof of identity or age at select airports, businesses, and venues. Adoption of the feature has been slow since Apple first announced it in September 2021, with IDs in the Wallet app only available in Arizona,…

Two New AirPods 4 Models Expected to Launch in September or October

Apple suppliers will begin production of two new fourth-generation AirPods models in May, according to Bloomberg’s Mark Gurman. Based on this production timeframe, he expects the headphones to be released in September or October. Gurman expects both fourth-generation AirPods models to feature a new design with better fit, improved sound quality, and an updated charging case with a USB-C…

See also  Apple Seeds Release Candidate Version of watchOS 10.5 to Developers

iPad Air Next on Apple’s OLED Tablet Roadmap Following iPad Pro

The iPad Air will be the next Apple tablet to adopt OLED display panel technology following its upcoming OLED iPad Pro models, according to an updated forecast from research firm Omdia (via The Elec). “In 2028, we expect the iPad Air to go OLED as well, with the iPad Air using single-stack and the iPad Pro using two-stack tandem OLEDs,” said Omdia researcher Kang Min-soo, speaking on…

Walmart Begins Selling MacBook Air With M1 Chip for $699 in U.S.

Walmart today announced that it has started selling the MacBook Air with the M1 chip in the U.S., with pricing set at a very reasonable $699. The laptop can be ordered now on Walmart.com, and it will be available soon at select Walmart stores. This is the first time that customers can purchase a Mac that is sold directly by Walmart, the company said in a press release. Some other retailers…

AirPods Case With Touchscreen Knock-Off Copies Apple Patent

Apple registers many patents most of which never see the light of day as consumer products – officially, that is. One burgeoning knock-off merchant in China has apparently taken cues from an idea Apple had in 2021 for AirPods featuring a case with a built-in touchscreen display. Pictures and video shared on X (Twitter) by occasional leaker @lipilipsi show production units of counterfeit…