things i find severely lacking in current llm benchmarks but i feel are important for iterative, open-ended knowledge work: - problem-setting. a major part of creative/exploratory work is defining what the problem even looks like. benchmarks skip that and hand you the problem pre-formed. it's the same issue with using exams to predict how someone handles real world problems.