---
title: "CES 2024: Survey Fit Across Multiple Question Forms | Minds"
canonical_url: "https://getminds.ai/research/ces-2024-synthetic-survey-validation"
last_updated: "2026-10-03T11:35:57.543Z"
meta:
  description: "A CES comparison evaluates 300 matched respondents on 27 binary, ordinal and multiselect questions, separating aggregate fit from individual prediction."
  "og:description": "A CES comparison evaluates 300 matched respondents on 27 binary, ordinal and multiselect questions, separating aggregate fit from individual prediction."
  "og:title": "CES 2024: Survey Fit Across Multiple Question Forms | Minds"
  "twitter:description": "A CES comparison evaluates 300 matched respondents on 27 binary, ordinal and multiselect questions, separating aggregate fit from individual prediction."
  "twitter:title": "CES 2024: Survey Fit Across Multiple Question Forms | Minds"
---

Minds

September 5, 2026·Validation·Minds Team # **CES 2024: Survey Fit Across Multiple Question Forms** A CES comparison evaluates 300 matched respondents on 27 binary, ordinal and multiselect questions, separating aggregate fit from individual prediction. [Sign in to download PDF](https://getminds.ai/auth/login?redirect=%2Fresearch%2Fces-2024-synthetic-survey-validation) The CES 2024 test reached 95.28% aggregate approximation with profile-probability answers, equivalent to 4.72 percentage points of mean distribution error. Across the response-format comparison, probability elicitation reduced error by a registered bootstrap estimate of 19.72 points relative to forced answers. The study extends the survey evidence beyond a single question form. It also shows why a final approximation score must be accompanied by matched baselines. ## The evaluated questionnaire The comparison used 300 matched CES respondents and 27 held-out questions: 15 ordinal, 10 binary and two multiselect batteries. Earlier respondent information provided context; the evaluation answers remained held out from generation. The four conditions were generic forced choice, generic probabilities or marginals, demographic probability, and full-profile probability. Multiselect options require marginal probabilities because a respondent may choose more than one option. Their interpretation differs from a single-choice probability vector. ## The measured distributions_### **Mean item absolute error** Reported results for this study. The table and discussion retain the study scope, baselines and uncertainty._- Generic forced choice**26.927**- Generic probability or marginals**7.036**- Demographic probability**4.563**- Full-profile probability**4.720**0Percentage points · lower is better30 | Condition | Mean item absolute error |
| --- | ---: | | Generic forced choice | 26.927 pp | | Generic probability or marginals | 7.036 pp | | Demographic probability | 4.563 pp | | Full-profile probability | 4.720 pp | Approximation is 100 minus mean absolute percentage-point error. The 95.28% figure therefore describes average distribution fit on these evaluated questions. It is not a percentage of accurately simulated people. The registered bootstrap probability lift was 19.719 points, rounded to 19.72. The reported point contrast was 19.892 points, with a 95% interval from 16.257 to 23.401. The bootstrap summary and the displayed condition means use different summaries and rounding; they should not be silently substituted for one another. ## What richer profiles did and did not improve Full profiles improved on generic probability prompting but did not beat matched demographic probability on the primary aggregate metric. The profile-versus-demographic lift was -0.157 points and its interval crossed zero. Individual accuracy improved with profile context while aggregate error was slightly worse than with demographics. The result is therefore mixed, with a strong response-format finding but no confirmed aggregate advantage for the full-profile condition over the demographic condition. Only two multiselect batteries were available. This is evidence from a mixed questionnaire, not universal validation of multiselect calibration across domains and instruments. ## Applying the evidence For distribution estimates, response collection and aggregation should be treated as explicit parts of the research design. A forced categorical baseline cannot isolate the value of profiles when the richer audience is allowed to return probabilities. A comparison should also state whether its target is item-level population fit or individual prediction. More context is not automatically better for both. The tested sample, model configuration and questionnaire define the claim boundary. This isolated-harness test does not guarantee current product behavior or representative estimates for new customer audiences. Public-source pretraining exposure remains possible even with runtime target separation. The [ANES longitudinal study](https://getminds.ai/research/anes-2024-synthetic-survey-validation) examines the same aggregate-versus-individual distinction. The [PISA comparison](https://getminds.ai/research/pisa-2022-synthetic-survey-validation) adds a five-country setting. The [evidence overview](https://getminds.ai/research/synthetic-audiences-reality-benchmark-2026) connects these results without pooling them into one score. ## What the score is measured against A flat distribution, with no model and no data behind it, already scores 86.14% on these 54 items. The reported condition scores 95.28%, closing 66% of the distance between that floor and a perfect score. The floor belongs next to the percentage, because the percentage alone does not show it. ## Reference data [CES 2024](https://doi.org/10.7910/DVN/X11EP6) [Minds](https://getminds.ai/)© 2026 Minds. Your target audience. AI-driven and grounded in transparent evidence. Build within minutes. [Minds on X (Twitter)](https://x.com/mindsai_co) [Minds on LinkedIn](https://www.linkedin.com/company/mindsaicompany/) [Minds on Instagram](https://www.instagram.com/getminds.ai/)Minds is part of [![ESOMAR](https://getminds.ai/images/newsroom/logos/esomar-logo.svg)ESOMAR](https://esomar.org/) [![GreenBook](https://getminds.ai/images/newsroom/logos/greenbook.svg)GreenBook Directory](https://greenbook.org/company/Minds) [![Insight Platforms](https://getminds.ai/images/newsroom/logos/insight-platforms.png)Insight Platforms](https://www.insightplatforms.com/platforms/minds/) [![Capterra](https://getminds.ai/images/newsroom/logos/capterra.svg)Capterra](https://www.capterra.com/p/10046203/Minds/) [![G2](https://getminds.ai/images/newsroom/logos/g2.svg)G2](https://www.g2.com/products/minds/reviews) [![CSSDA Best UX Design Award](https://getminds.ai/images/newsroom/logos/cssda-best-ux-award.png)CSSDA Best UX Design Award](https://www.cssdesignawards.com/) [![CSSDA Best Innovation Award](https://getminds.ai/images/newsroom/logos/cssda-best-innovation-award.png)CSSDA Best Innovation Award](https://www.cssdesignawards.com/) [![CSSDA Best UI Design Award](https://getminds.ai/images/newsroom/logos/cssda-best-ui-award.png)CSSDA Best UI Design Award](https://www.cssdesignawards.com/)