OpenAI has released the third version of its generative artificial intelligence visual arts platform DALL-E, which now allows users to create prompts using ChatGPT and includes more security options. DALL-E converts text prompts into images. But even the DALL-E2 makes mistakes, often overlooking specific wording. OpenAI researchers say the latest version better understands context.
A new feature of DALL-E3 is integration with ChatGPT. By using ChatGPT, users don't have to write detailed prompts themselves to guide DALL-E3; they simply ask ChatGPT to come up with a prompt, and the chatbot writes a paragraph (DALL-E is better at using longer sentences) for DALL-E3 to follow. Other users can also use their own tips if they have special ideas for DALL-E.
DALL-E was first released in January 2021, ahead of StabilityAI and Midjourney’s other text-to-image generative AI art platforms. By the time DALL-E2 launches in 2022, OpenAI has opened a waiting list to control who can use the platform amid criticism that DALL-E can generate realistically explicit images and exhibit bias in the photos it generates. Last September, the company eliminated the waiting list and opened the DALL-E2 to the public.
The new version of DALL-E will be released first to ChatGPTPlus and ChatGPTEnterprise users in October, followed by research labs and their API services in the fall. OpenAI plans to stagger the release of DALL-E3, but has not committed to when a free public version will be released.
OpenAI claims that it has done extensive work on DALL-E3, focusing on creating strong security measures to prevent the creation of obscene or potentially hateful images. OpenAI says it works with external redteamers — a group that intentionally attempts to compromise a system to test its security — and relies on input classifiers, a method that teaches language models to ignore certain words to avoid explicit or brute-force prompts. The DALL-E3 also cannot reproduce the image of a public figure - provided that the name is explicitly mentioned in the prompt.
Sandhini Agarwal, a policy researcher at the company, said she was "very confident" in the company's security measures, but clarified that the model is constantly improving and is not perfect. A representative from OpenAI said in an email that DALL-E3 has been trained not to generate images in the style of living artists. Unlike DALL-E2, DALL-E2 can imitate the art style of certain artists when prompted.
Possibly to avoid potential lawsuits, OpenAI will also allow artists to choose to exclude their artwork from future versions of the text-to-image AI model. Creators can submit images they own copyright and request their removal by filling out a form on the site. This way, future versions of DALL-E can block results that resemble the artist's image and style. The artists sued DALL-E competitors StabilityAI and Midjourney, as well as art website DeviantArt, accusing them of using their copyrighted works to train text-image models.