A recent study finds that while generative AI (GenAI) can help define viable objectives for organizational and policy decision-making, the overall quality of those objectives falls short unless humans intervene. The paper is published in the journal Decision Analysis.
In the field of decision analysis, defining objectives is a foundational step. Before you can evaluate options, allocate resources or design policies, you must identify what you’re trying to achieve.
The research underscores that AI tools are valuable brainstorming partners, but sound decision analysis still requires a human in the loop.
The study, “ChatGPT vs. Experts: Can GenAI Develop High-Quality Organizational and Policy Objectives?” was authored by Jay Simon of American University and Johannes Ulrich Siebert of Management Center Innsbruck.
The researchers compared objectives generated by GenAI tools—including GPT-4o, Claude 3.7, Gemini 2.5 and Grok-2—to objectives created by professional decision analysts in six previously published Decision Analysis studies. Each GenAI-generated set was rated across nine key criteria from value-focused thinking (VFT), such as completeness, decomposability and redundancy.
They found that while GenAI frequently produced individually reasonable objectives, the sets as a whole were incomplete, redundant and often included “means objectives” despite explicit instructions to avoid them.
“In short, AI can list what might matter, but it cannot yet distinguish what truly matters,” the authors wrote.
“Both lists are better than most individuals could create. However, neither list should be used for a quality decision analysis, as you should only include the fundamental objectives in explicitly evaluating alternatives,” said Ralph Keeney, a pioneer of value-focused thinking, in response to two AI-produced lists of objectives in the study.
To improve GenAI output, the researchers tested several prompting strategies, including chain-of-thought reasoning and expert critique-and-revise methods. When both techniques were combined, the AI’s results significantly improved—producing smaller, more focused and more logically structured sets of objectives.
“Generative AI performs well on several criteria,” said Simon. “But it still struggles with producing coherent and nonredundant sets of objectives. Human decision analysts are essential to refine and validate what the AI produces.”
Siebert added, “Our findings make clear that GenAI should augment, not replace, expert judgment. When humans and AI work together, they can leverage each other’s strengths for better decision making.”
The study concludes with a four-step hybrid model for decision-makers that integrates GenAI brainstorming with expert refinement to ensure the objectives used in analysis are essential, decomposable and complete.
More information:
Jay Simon et al, ChatGPT vs. Experts: Can GenAI Develop High-Quality Organizational and Policy Objectives?, Decision Analysis (2025). DOI: 10.1287/deca.2025.0387
Institute for Operations Research and the Management Sciences
Citation:
Generative AI can brainstorm objectives, but needs human expertise for decision quality (2025, November 11)
retrieved 11 November 2025
from https://techxplore.com/news/2025-11-generative-ai-brainstorm-human-expertise.html
This document is subject to copyright. Apart from any fair dealing for the purpose of private study or research, no
part may be reproduced without the written permission. The content is provided for information purposes only.
