What Is a Concept Testing Survey with AI?
A concept testing survey is a type of research where new product concepts are evaluated by a target audience before launch. The goal is to uncover which ideas resonate most, reduce risk, and guide investment decisions.
Traditionally, these concepts are generated by innovation teams. But with the rise of generative AI tools like ChatGPT, researchers can now use AI to create new concepts at scale, saving time and giving teams more options to test with consumers.
Read more: 4 Ways to Use AI and Improve the Innovation Process
Why Use AI in Concept Testing?
AI doesn’t replace human creativity, but it makes the concept testing process faster, more efficient, and more diverse. Here are the biggest benefits:
1. Faster idea generation
Instead of spending hours brainstorming, AI can produce dozens of concepts in seconds. By generating ideas quicker, you can launch your testing sooner and get to insight faster.
2. More variety of concepts
AI can generate hundreds of unique ideas, some of which your team may not have considered. Even if not all are winners, they serve as valuable starting points.
3. Flexible and customizable
With the right prompts, researchers can guide AI to produce concepts that meet specific requirements, whether it’s focusing on sensory details, targeting a demographic, or mirroring brand guidelines.
How to Boost Your Concept Testing Survey with AI-Generated Ideas
If you want AI to strengthen your concept testing surveys, here are four practical steps:
- Start with AI prompts to spark creativity
Use AI to brainstorm a wide range of ideas. Example: “Generate 10 dipping sauce flavors that combine sweet and savory notes.” - Refine and tailor ideas for your audience
Don’t test raw AI output. Prompt AI to add sensory language, emphasize textures, or adapt ideas for specific audiences (e.g., Gen Z, health-conscious consumers).- Pro tip: Keep an eye on tone consistency. AI-generated ideas can vary in writing style—from neutral and factual to overly sales-y. Before testing, edit your concepts so they’re written in a consistent, objective voice. This helps ensure responses reflect the idea itself, not differences in tone.
- Increase the volume of concepts you test
Since AI removes the bottleneck of ideation, you can test more ideas at once. This leads to richer data and a higher likelihood of spotting breakthrough innovations. - Accelerate time to insights
By reducing time spent in brainstorming, your team can focus on analyzing results and shaping strategy. AI handles the volume; humans handle the vision.
Pro tip: Treat AI-generated ideas as both testable options and creative sparks. Even if they don’t win, they often inspire refinements that lead to stronger human-led concepts.
Case Studies: AI vs. Human Ideas in Concept Testing
To put AI to the test, we compared AI-generated ideas vs. human-generated ideas across three concept testing studies.
The questions we had were:
- Can AI generate compelling enough ideas for new products?
- Can humans come up with ideas which are as unique and interesting?
- And who would win this battle of best idea generation – AI or humans?
Study #1: Milkshake flavors
We asked ChatGPT to come up with 10 new flavors for milkshakes. Then, we asked our internal team to do the same. The goal was to create flavors that had never existed before so that respondents could judge not only on flavor but also creativity.
We then inputted all flavors into Dig Insights’ innovation testing platform, Upsiide, and surveyed 200 people in the US and Canada. Respondents swiped right if they liked a flavor and left if they didn’t like it; then, they were asked to choose a favorite.
The result? Humans won.
Chocolate + Baileys (a human-generated idea) came to the top with an Idea Score of 68.
But AI came just one point behind the winning idea: White Chocolate and Raspberry Truffle was a close runner-up, scoring 67 in Idea Score.
And what’s cool is that US consumers especially loved the idea of White Chocolate and Raspberry Truffle. So here’s the market for ChatGPT in case it wants to open a milkshake business (hey, you never know).
AI also did well capturing the attention of a small but dedicated group of people. Spicy Mango Habanero, an AI idea, appeared in the Niche quadrant – meaning that there is a group of consumers who would choose this flavor over all others.
Study #2: Dipping sauce flavors
Okay, after testing different milkshakes, we kind of got hooked on testing more food flavors. This time, we screened new flavors for dipping sauces.
Smoked Crispy Bacon Crunch (a human idea) topped the charts with an Idea Score of 66 – which we think makes sense. In our previous research on burger flavors, we discovered that North American consumers really love flavors that convey a crispy texture or contain bacon.
Maple Bourbon (a ChatGPT creation) came second, scoring 55. But we think if this idea was developed a little more, taking into context that consumers love flavors with different textures, it could have had a chance against Smoked Crispy Bacon Crunch.
As a matter of fact, we went back to ChatGPT and asked it to incorporate some texture-related wording in this flavor idea, and it gave us some interesting options:
- Maple Bourbon Crisp Delight
- Spicy Bacon Maple Bourbon
- Salted Pretzel Maple Bourbon
Who knows, maybe one of these flavors would have beaten the winner.
Study #3: Credit card rewards for Gen Z
For this last study, we thought AI needed a different kind of challenge. So, we entered the finance world and asked ChatGPT to develop some credit card reward ideas specifically targeted at Gen Z.
When we looked at the ideas submitted by our team and AI, we found many of the same ones. While our team has tons of experience working with finance clients, ChatGPT doesn’t.
So it was so cool (and sort of creepy) to see how well AI knows banking and what rewards consumers might like. And it was evident in the results too.
2 ideas in the top 3 belong to AI.
“Discounts on food delivery services, such as Uber Eats or DoorDash” and “Discounts on streaming services, such as Netflix or Spotify” scored 72 and 74 in the Idea Score, respectively. But the winner still became a human idea, “Cashback rewards on purchases from online retailers (e.g. Amazon)”.
AI again did well in finding a rewards idea that would be attractive to a small niche of Zoomers – it was “rewards points for making peer-to-peer transactions, such as splitting bills with friends or paying rent”.
Key Takeaways: Why You Should Use AI in Concept Testing
After comparing AI and human ideas, here’s what we learned:
- AI can match human creativity – With a bit of refinement, AI ideas can be just as compelling as human ones.
- AI is great for niche appeal – If presented by the right brand or with the right messaging, AI-generated concepts can perform even better than human ones.
- AI accelerates the process – Less time spent brainstorming means more time analyzing, strategizing, and acting on insights.
FAQs: AI in Concept Testing Surveys
Can AI replace humans in concept testing?
No. AI is best used as a partner to human creativity. It generates options quickly, but humans refine ideas and provide strategic context.
What’s the biggest benefit of AI in concept testing surveys?
Speed. AI reduces brainstorming time from days to minutes, letting teams test more ideas and get insights faster.
How do you ensure AI-generated ideas are useful?
By crafting strong prompts and refining output. Adding sensory details, textures, or demographic context makes ideas more realistic for testing.
Does AI always beat human ideas?
Not yet. Our studies show humans still win more often—but AI is a strong competitor, especially for niche audiences or simple categories.
Final Thought
Generative AI is reshaping the way researchers approach concept testing surveys. By using AI to generate, refine, and expand your pool of ideas, you can move faster, test more broadly, and unlock new opportunities.
Ready to run your next concept testing survey with AI-generated ideas? Start by experimenting with prompts, refining output, and feeding AI concepts into your testing platform.
About the Author
This article was written in collaboration with Leigh Greenberg, Customer Enablement Manager at Dig Insights. In her role, Leigh helps brands modernize their market research practice through Upsiide, bringing expertise in innovation research and concept testing to support smarter product and creative decisions.