AI
Prompt Engineering Adversarial Questions
Engineered adversarial questions targeting underrepresented topics using techniques that trick modern LLMs and question answering systems.
Jan 2023 — May 2023Final Report↗
Overview
Engineered 50 adversarial questions targeting topics underrepresented in western media and datasets, evaluating against ChatGPT, YouAI, and Bard. The goal: make QA datasets more inclusive of diverse global information.
Our findings show that modern LLMs have gotten extremely good at question answering — but still fail at highly specific cultural knowledge that falls outside their training distribution.
Documents