AI-SDM Seminar - Nihar Shah
September 3, 2026 12:00PM—1:30PM
Location:
In Person and Virtual - ET
-
Newell-Simon 3305 and Zoom
Speaker:
NIHAR SHAH,
Associate Professor, Computer Science Department and Machine Learning Department, Carnegie Mellon University
https://www.cs.cmu.edu/~nihars/
The talk will present three tales told by Scheherazade to the Sultan in ArabIan Nights about AI and Science.
- Sinbad and the Poisoned Datasets: There have been many cases where organizations with vested interests (e.g., tobacco companies) have looked to manipulate public opinion and policies towards their own interests. They have historically done so by bribing researchers into sham research aligned with those interests. This was challenging and expensive. Given we are in this new AI age, what more can they do now, and can we mitigate that?
- Ali Baba and the 40 Prompts: P-hacking involves researchers torturing data until they get desired – but often spurious – results. When using LLM as a judge or using LLMs for annotation, p-hacking is easily done by simply trying many prompts until a desired result is obtained. The conventional way of mitigating p-hacking is preregistration, where researchers must register their analysis plan before collecting any data. But that doesn’t work here. So what can we do about it?
- Aladdin and the Magic Latex: The Magic Latex (i.e., AI) has led to a rapid increase in the number of submissions. Some conferences and journals are putting fixed caps, e.g., nobody can submit more than a certain number of papers. But a large fixed cap leads to a ton of single-author submissions (e.g., in the TMLR journal we have seen many cases of 4-8 single-author submissions by the same person in a span of 1-2 weeks) and a small fixed cap means that advisors with multiple students cannot submit. So where do we draw the line? Or perhaps we draw a curve?
—
Nihar Shah is an associate professor at Carnegie Mellon University, with joint appointments in the Machine Learning and Computer Science departments. His group’s research is centered around the science of evaluation and the evaluation of science. They develop algorithms with strong theoretical guarantees, as well as conduct large-scale controlled experiments for evidence-based policy design and real-world deployments.
REGISTER →to attend in-person or on Zoom
For More Information:
pwerns@andrew.cmu.edu