In this conversation, Mike discusses the latest developments in AI and machine learning, focusing on recent research papers that explore the reasoning capabilities of large language models (LLMs) and the implications of self-improving AI systems. The discussion includes a critical analysis of Apple's paper on LLM reasoning, comparisons between human and AI conceptual strategies, and insights into the Darwin-Girdle machine, a self-referential AI system that can modify its own code. Mike...
Jun 23, 2025
1 hr 5 min
Session Topics: The Llama 4 Controversy and Evaluation Mechanism Failure Llama 4’s initial high ELO score on LM Arena was driven by optimizations for human preferences—such as the use of emojis and overly positive tone. When these were removed, performance dropped significantly. This exposed weaknesses in existing evaluation mechanisms and raised concerns about benchmark reliability. Two Levels of AI Evaluation There are two main types of AI evaluation: model-level benchmarking for foundati...
Jun 3, 2025
1 hr
Some of the main topics discussed. Google Gemini 2.5 Release Gemini 2.5 is now leading AI benchmarks with exceptional reasoning capabilities baked into its base training. Features include a 1M token context window, multimodality (handling text, images, video together), and independence from Nvidia chips, giving Google a strategic advantage. Alibaba’s Omnimodal Model ("Gwen") Alibaba released an open-source model that can hear, talk, and write simultaneously with low latency. It uses a "thin...
Apr 29, 2025
1 hr 4 min
The sessions topics include: Reasoning Models: Mike highlights the rise of reasoning models dominating leaderboards, enabled by "inference time compute scaling." This allows models to allocate more computational power dynamically, leading to better accuracy and efficiency. These models use "chain of thought prompting," enhancing reasoning by generating intermediate steps, inspired by Daniel Kahneman's "System 2 thinking." He also discussed "Humanity's Last Exam," a challenging new benchmark d...
Feb 28, 2025
1 hr 3 min