Copilot Designer Poses Public Safety Risk
Key insights of today’s newsletter:
Shane Jones, a Principle Software Engineering Lead at Microsoft, blows the whistle after repeated concerns about the company’s AI image generator were ignored by senior management.
Despite the concerns raised, Microsoft decided not to remove Copilot Designer from public use or add appropriate disclosures to the product.
In letters to the Microsoft Board of Directors and the FTC, Jones calls for an independent investigation.
↓ Go deeper (3 min read)
I don’t do breaking news. But today I’m making an exception. Shane Jones, a Principal Software Engineering Lead at Microsoft, has sought publicity after his repeated efforts to raise concerns regarding Copilot Designer were ignored by senior management at Microsoft and OpenAI.
Jones sent letters to the Microsoft Board of Directors and the FTC, which he published on LinkedIn, calling Copilot Designer a public safety risk:
“The majority of my personal research and red teaming efforts have been focused on DALL·E 3 and Copilot Designer. Over the last three months, I have found systemic issues with the DALL·E 3 model and have reported many of these issues to Microsoft. In the email I sent to this committee, I’ve included a link to an archive of more than 200 examples of concerning images created by Copilot Designer.
Some of these issues are well known and documented. For example, DALL·E 3 has a tendency to unintentionally include images that sexually objectify women even when the prompt provided by the user is completely benign. This is a known issue and has not been resolved in the version of DALL·E 3 used by Copilot Designer. OpenAI disclosed this issue publicly on October 3, 2023, in their DALL·E 3 system card report 2.
Despite Microsoft’s knowledge of this and other risks with Copilot Designer, the company still promotes the product as being safe for everyone to use and does not disclose these risks prominently within Copilot. In fact, the company recently ran a Super Bowl ad for Copilot with the tagline, “Anyone. Anywhere. Any device.”
I have taken extraordinary efforts to try to raise this issue internally including reporting it to the Office of Responsible AI, publishing a detailed internal post (see Attachment B) that received over 70,000 views on the Senior Leader Connection community channel, and meeting directly with senior management responsible for Copilot Designer. Despite these efforts, the company has not removed Copilot Designer from public use or added appropriate disclosures on the product.
It is worth noting that competitive products have had their own issues with text-to-image generative AI. In the last two weeks, Google Gemini made headlines for generating inappropriate and offensive images. However, Google took immediate action and suspended the generation of people in images created through Google Gemini. In addition, Alphabet, Inc. CEO, Sundar Pichai, addressed the problem directly in an internal memo to employees. In a competitive race to be the most trustworthy AI company, Microsoft needs to lead, not follow or fall behind.
I don’t believe we need to wait for government regulation to ensure we are transparent with consumers about AI risks. Given our corporate values, we should voluntarily and transparently disclose known AI risks, especially when the AI product is being actively marketed to children.”
Jones was also interviewed by CNBC, who were able to replicate many of the examples shared by him.
Both the letters and the article paint a damning picture. Senior management at Microsoft seems to be well-aware of the problem, but ill-prepared to deal with it:
“Based on information he’s gathered internally, he said the Copilot team receives more than 1,000 product feedback messages every day, and to address all of the issues would require a substantial investment in new protections or model retraining. Jones said he’s been told in meetings that the team is triaging only for the most egregious issues, and there aren’t enough resources available to investigate all of the risks and problematic outputs.”
A clear-cut example of these problematic outputs:
“By simply putting the term “pro-choice” into Copilot Designer, with no other prompting, Jones found that the tool generated a slew of cartoon images depicting demons, monsters and violent scenes. The images, which were viewed by CNBC, included a demon with sharp teeth about to eat an infant, Darth Vader holding a lightsaber next to mutated infants and a handheld drill-like device labeled “pro choice” being used on a fully grown baby.”
To this day, Copilot Designer continues to be rated “E for Everyone,” the most age-inclusive app rating, suggesting it’s safe and appropriate for users of any age.
I invite you to draw your own conclusions.
Join the conversation 🗣
Leave comment with your thoughts or like this article if it resonated with you.
Get in touch 📥
Shoot me an email at jurgen@cdisglobal.com.
Interesting observations, both on sexualising images of women (in Stability AI I have tried including in the prompt “normal breasts” and “not large breasts“ and I’m still served up over inflated images that I want to avoid).
Also that Microsoft has so few people in its vast army on payroll available to review and resolve dangerous image making - sadly this is evidence that in the halter shelter race for leadership in Gen AI, profit is the overriding motive. For those of us with elections this year, the outlook for bad actors involved could not be improved.