Maximize realism with granular simulation. The simulator models 26 customer segments and individual customers within each segment rather than only aggregate demand. Each customer has its own acquisition path, subscription state, price exposure, usage, satisfaction, and churn trajectory. Customers are also organized into diverse groups with different needs, budgets, price sensitivities, ad channel effectiveness, support expectations, and behavioral patterns.
Robust simulation with mechanistic rules. The world emulates real business behavior while maintaining stable cause-and-effect relationships. Almost all simulator outcomes are generated by explicit mechanisms rather than by using an LLM as an opaque judge.
Consistent simulation under stochasticity. While we inject stochasticity into world dynamics, we maintain consistency across runs with independent random number generators for different simulator components. Under the same random seed, after calling the market research tool multiple times, the agent always discovers the same sequence of new market segments, independent of actions in other areas.
Hidden information and indirect feedback. CEO-Bench tests whether agents can gather information in a partially observable world. The agent receives only information that a real operator could plausibly observe: dashboards, database records, social-media posts, research reports, and negotiation history. It does not observe true customer satisfaction, latent willingness to pay, churn propensity, competitor schedules, or demand parameters.
Interconnected world dynamics. We design the simulated world to make it difficult to isolate a single causal relationship and hill-climb on it. Every decision can influence many other parts of the market. Reputation propagates across related groups, so a quality failure in one enterprise segment can spill into nearby segments and eventually affect consumer demand.
Delayed and uncertain consequences. Many actions have delayed and uncertain effects, forcing long-horizon decision making under uncertainty. Costs may appear immediately, while corresponding revenue, retention, research, or reputation effects arrive weeks later.
Non-stationary environment. Agents must continually gather new information and adapt because the environment changes over the course of a simulation. Competitors place adaptive pressure on product quality, customer behavior drifts over time, and macroeconomic trends affect willingness to pay and enterprise seat counts.
