On September 13, the Wharton School of the University of Pennsylvania, one of the world's largest business schools, announced a study on its official website. ChatGPT (GPT-4) surpassed elite MBA students in creative idea testing. The test asked ChatGPT and MBA students to design 200 product ideas for the college market that would retail for $50 or less. The criterion for measuring creativity is to see whose creativity can sell more products.
ChatGPT automatically generated 200 product ideas in 15 minutes; MBA students only came up with 5 ideas in 15 minutes. The results show that the average purchase rate of ChatGPT’s products is 46.8%, while students’ purchase rate is only 40.4%, lagging behind the AI robot.
Christian Terwiesc, Wharton professor and co-chair of the Innovation Research Management Institute, I have always thought that creativity is one of the areas where humans are best at, but the test results are surprising. It is obvious that everyone should try to generate better creative ideas through ChatGPT.
Key findings
Generative AI such as ChatGPT can help humans break through creative bottlenecks and absorb diverse inspirations to achieve broad creative thinking.
Compared with manual labor, ChatGPT can provide cheaper and more efficient work execution.
In this test, ChatGPT's quality and efficiency in generating creative ideas were comprehensively ahead of those highly intelligent and well-trained MBA students. In other words, generative AI can not only be applied to "rote learning" business, but can also be used for creative work.
You can try generative AI such as ChatGPT and apply it as a creative assistant in various business scenarios to improve work and creative efficiency.
A brief introduction to testing research
The Wharton School has more than 20 years of experience in teaching product design and innovation courses, and has held more than 10 similar product creativity challenges. This test consists of 200 questions selected from the 2021 class.
These questions include a title and a descriptive text, and the overall creation is aimed at the college student market, covering a variety of daily items such as shoes, notebooks, pens, clothes, etc., with a retail price of $50 or less (the price limit is set to increase the complexity of the test questions).
A tester input 200 test questions into ChatGPT, and 200 creative ideas were generated in 15 minutes (100 naturally generated, 100 with example prompts)). An MBA student only came up with 5 ideas in 15 minutes, and the execution efficiency of a team may be even worse. Because there will be differences of opinion, and there may be scenes of heated discussions that consume more time.
Although ChatGPT's creative efficiency is very high, it may also be mixed with many poor ideas. Therefore, economic value is the best choice to measure creativity.
The researchers found some students to form an evaluation team and conducted a comprehensive evaluation of 400 creative ideas generated by ChatGPT and MBA students.On average, each respondent evaluated 40 ideas, and each idea was evaluated an average of 20 times., to reflect their willingness to purchase the product.
Test results
Evaluators were asked to express purchase intentions using a standard "five-box" response: Definitely not buying, Probably not buying, Maybe or not buying, Probably buying, Definitely going to buy.
The five responses were weighted by 0, 0.25, 0.50, 0.75, and 1.00 to develop a measure of purchase probability. This weighting method was proposed by Professors Jameson and Bass in 1989 and is a mature evaluation system.
The test results show thatUsing purchase intention as a metric, the average quality of ideas generated by ChatGPT is higher than the average quality of ideas generated by humans.. The average purchase probability for human-generated ideas is 40.4%, the average purchase probability for original ChatGPT is 46.8%, and the average purchase probability for ChatGPT with example prompts is 49.3%.
also,ChatGPT generated the highest-rated creative idea in the test sample, with an 11% higher probability of purchase than the best human idea.
Overall, out of 400 ideas generated by ChatGPT and humans. Of the top 40 ideas (top 10%), 35 (87.5%) were generated by ChatGPT.In other words, in one-on-one competitions, most of the winners come from ChatGPT.
About the Wharton School
The Wharton School was founded in 1881 and is affiliated with the University of Pennsylvania. It is one of the oldest and largest business schools in the world and one of the most influential business schools in the United States.
The Wharton School is known for its excellence in education and research in areas such as finance, economics, industrial management, innovation and global business strategy. Wharton School alumni are spread across all walks of life around the world, including multiple Nobel Prize winners, successful business leaders, etc.