New Technology– category –
-
Φ-Bench: New Benchmark with 85 Real-World Tasks Evaluates LLMs’ Ability to Engineer and Optimize Their Own Infrastructure
arXiv USA Overview Researchers have introduced Φ-Bench (Frontier AI Infrastructure Benchmark), a novel evaluation suite comprising 85 real-world tasks designed to assess Large Language Models' (LLMs) capacity to develop and optimize thei... -
NovGauge: A Human-Anchored Fine-Grained Benchmark Diagnoses LLMs’ Capability in Scientific Paper Novelty Assessment, Revealing Hallucinations and Lack of Justification
arXiv USA Overview Researchers have proposed NovGauge, a fine-grained, human-anchored benchmark designed to diagnose Large Language Models' (LLMs) capabilities in assessing scientific paper novelty. Comprising 619 paper pairs and 50 mult... -
Stanford University Develops ‘Virtual Biotech’ with 37,000 AI Agents for Drug Discovery, Published in Science
Science (via Chosunilbo DB) USA Overview A Stanford University research team has developed 'Virtual Biotech,' a system where up to 37,000 AI agents collaborate on drug discovery, with findings published in the journal Science. This syste... -
US Nonprofit METR Evaluates Frontier AI Systems from Anthropic, Google, Meta, OpenAI, Focusing on Catastrophic Risks
Business Standard USA Overview METR (Model Evaluation and Threat Research), a US-based nonprofit, conducted safety evaluations of frontier AI models from leading developers including Anthropic, Google, Meta, and OpenAI. Dedicated to deve...
