Apple researchers have released a new open source artificial intelligence model that can edit images based on natural language instructions from users. MGIE is the abbreviation of MLLM-GuidedImageEditing, which uses Multimodal Large Language Model (MLLM) to interpret user requests and perform pixel-level operations.
This model is capable of editing every aspect of an image. Global photo enhancements can include brightness, contrast, or sharpness, or apply artistic effects like sketching. Local editing can modify the shape, size, color, or texture of specific areas or objects in an image, while Photoshop-style modifications include cropping, resizing, rotating, and adding filters, or even changing the background and blending the image.
A user's input for a picture of a pizza might be "make it look healthier." Using common sense reasoning, the model can add vegetable ingredients like tomatoes and herbs. Global optimization input requests could be in the form of "increase contrast, simulate more light," while Photoshop-style modifications could be asking the model to remove people from the background of the photo, shifting the focus of the image to the subject's facial expressions.
Apple collaborated with researchers from the University of California to create MGIE and published a paper at the 2024 International Conference on Learning Representations (ICLR). The model is available on GitHub, including code, data, and pre-trained models.
This is Apple’s second breakthrough in artificial intelligence research in as many months. In late December, Apple revealed that it had made strides in deploying large language models (LLMs) on iPhones and other memory-constrained Apple devices by inventing an innovative flash memory utilization technology.
For the past few months, Apple has been testing an "AppleGPT" that could compete with ChatGPT. According to Bloomberg's Mark Gurman, AI work is a priority for Apple, and the company is designing an "Ajax" framework for large language models.
Both The Information and analyst Jeff Pu claim that Apple will launch some kind of generative artificial intelligence feature on iPhone and iPad around the end of 2024, which is when iOS 18 is launched. According to Gurman, iOS 18 is said to include an enhanced version of Siri with ChatGPT-like generative AI capabilities and has the potential to be the "biggest" software update in iPhone history.