Speaker: Dr. Ram D. Sriram, Senior Science Advisor, Information Technology Laboratory (ITL), National Institute of Standards and Technology
Generative AI is transforming the landscape of scientific discovery across three distinct operational pathways: combinatorial space search, data-driven hypothesis generation, and paradigmatic reframing. However, each pathway introduces unique evaluation challenges that require a formal discipline of AI metrology—a systematic framework for quantifying system reliability, operational scale, and human oversight requirements.
This talk presents a structured metrological roadmap mapping established cognitive theories (such as the Cattell-Horn-Carroll hierarchy) and stages of AI capability—Artificial Limited Intelligence (ALI), Artificial General Intelligence (AGI), and Artificial Superintelligence (ASI)—to these three modes of discovery:
– ALI & Combinatorial Search: Focuses on optimizing search bounds, parameter scaling, and uncertainty quantification, with humans acting as ground-truth oracles to measure correction rates and time-to-override.
– AGI & Hypothesis Generation: Addresses explore-exploit cycles via dual-layer neuro-symbolic governance, formal ontology mapping, and the Garland Test (evaluating perceived problem awareness), while humans serve as symbolic arbiters to calibrate appropriate reliance.
– ASI & Paradigmatic Reframing: Reconceptualizes fundamental laws through asynchronous multi-agent scaffolding ("A-Teams"), placing humans in an on-the-loop auditing role measured by spot-audit coverage and escalation latency.
Finally, the talk outlines actionable tracks for institutions to standardize evaluation protocols across this spectrum, emphasizing that AI metrology must evolve qualitatively—transitioning from output verification to trust calibration and system-level auditing—to ensure scientific claims remain trustworthy and reproducible.
REGISTER: https://fau-edu.zoom.us/meeting/register/kj39AvFAQViGa39JrgefUQ
Co-sponsored by: Dept of Electrical Engineering and Computer Science, Florida Atlantic University
Virtual: https://events.vtools.ieee.org/m/577624
This event has passed.
