Michael Galarnyk

Have you ever wondered how large language models (LLMs) could improve their understanding of long contexts and enhance document retrieval? The newly released paper, "𝗤𝘂𝗲𝗿𝘆-𝗳𝗼𝗰𝘂𝘀𝗲𝗱 𝗮𝗻𝗱 𝗠𝗲𝗺𝗼𝗿𝘆-𝗮𝘄𝗮𝗿𝗲 𝗥𝗲𝗿𝗮𝗻𝗸𝗲𝗿 𝗳𝗼𝗿 𝗟𝗼𝗻𝗴 𝗖𝗼𝗻𝘁𝗲𝘅𝘁 𝗣𝗿𝗼𝗰𝗲𝘀𝘀𝗶𝗻𝗴," by Yuqing Li, Jiangnan Li, Mo Yu, et al., addresses this very challenge. The authors propose a groundbreaking 𝗿𝗲𝗿𝗮𝗻𝗸𝗶𝗻𝗴 𝗳𝗿𝗮𝗺𝗲𝘄𝗼𝗿𝗸 that utilizes the attention scores of selected heads in LLMs to estimate passage-query relevance. This 𝗹𝗶𝘀𝘁𝘄𝗶𝘀𝗲 𝘀𝗼𝗹𝘂𝘁𝗶𝗼𝗻 leverages holistic information within the entire candidate shortlist, producing continuous relevance scores that facilitate training on arbitrary datasets without the need for Likert-scale supervision. The key contribution of this research is the demonstration that 𝘁𝗿𝗮𝗶𝗻𝗶𝗻𝗴 𝗮𝘁𝘁𝗲𝗻𝘁𝗶𝗼𝗻 𝗵𝗲𝗮𝗱𝘀 𝗱𝗶𝗿𝗲𝗰𝘁𝗹𝘆 𝗮𝘀 𝗿𝗲𝗿𝗮𝗻𝗸𝗲𝗿𝘀 can yield faster and more stable results compared to generation-based approaches. This method not only overcomes the geometric bottleneck of embedding models but also avoids the instability of listwise generation methods. However, it's important to note that 𝗽𝗼𝗶𝗻𝘁𝘄𝗶𝘀𝗲 𝗿𝗲𝗿𝗮𝗻𝗸𝗲𝗿𝘀 may still miss global context about other documents in the candidate set, potentially limiting ranking accuracy. For ML engineers, this research offers a 𝗹𝗶𝗴𝗵𝘁𝘄𝗲𝗶𝗴𝗵𝘁 𝗮𝗻𝗱 𝗲𝗳𝗳𝗲𝗰𝘁𝗶𝘃𝗲 solution that requires only small-scale models to achieve strong performance. Despite its advantages, the approach highlights the need for careful consideration of contextual information and the potential trade-offs in efficiency and performance. Follow us for more research insights, or dig deeper into this paper at PaperChime: https://lnkd.in/gVm_s5hD #LargeLanguageModels #AI #ContinuousRelevanceScores

2

This article from Shuai Guo is your guide to LLM applications in anomaly detection, empowering you to build a more secure, accurate, and explainable system.

7

Last week at International Symposium on Microarchitecture (MICRO) 2025, we presented our work PyTorchSim, a fast and cycle-accurate NPU simulation framework with comprehensive feature support. We would like to share it here as well. Highlights of PyTorchSim: - Integrated with PyTorch 2, it can simulate existing PyTorch models by simply designating a simulated NPU as the target device. - Provides an NPU-specific compiler backend based on MLIR and LLVM, enabling compiler optimizations and supporting the simulation of both training and inference. - Supports multi-core NPU and multi-model tenancy with detailed interconnect and DRAM models (Booksim and Ramulator 2). - Can model data-dependent timing behavior, such as that of mixture-of-experts models. - Implements a custom RISC-V–based ISA with a rich instruction set to express various operations in AI models. - Employs a Tile-Level Simulation technique, which enables fast simulation without loss of accuracy. - Validated against Google TPU v3, showing a mean absolute error (MAE) of 11.5%. PyTorchSim is available on GitHub. Please refer to our paper for more details. GitHub: https://lnkd.in/gmVDNCFD Paper: https://lnkd.in/gSKJVMwk MICRO’25 slide deck: https://lnkd.in/g6PjgFvm We will continue to enhance PyTorchSim’s capabilities to make it more useful for NPU architecture and compiler research. We would like to thank Prof. Jongse Park (KAIST) and Dr. Jeehoon Kang (Furiosa AI) for their valuable advice on this work. This project was led by our students Wonhyuk Yang, Yunseon Shin, and Okkyun Woo.

91

1 Comment

Reinforcement Learning from Human Feedback (RLHF) is powering the next wave of progress in AI alignment and model safety! If you're looking to deepen your practical and conceptual understanding of RLHF from preference data to reward modeling, instruction tuning, and advanced topics like Constitutional AI, check out the new open-access RLHF Book by Nathan Lambert. https://lnkd.in/dabupxS8 #machinelearning #ai #ml #rl #rlhf #llm #curiousnerd

2

🌍 Stanford Data Science scholar Luc Houriez is advancing our understanding of sea level rise through his research on Antarctica’s Thwaites Glacier—one of the fastest melting glaciers on Earth. By modeling how ice, ocean, and even the rebounding bedrock interact, his work helps refine projections that are critical for protecting coastal communities from billions of dollars in potential damage. 🔗 Read more about his research here: https://lnkd.in/gpcZ3NSm

13

What if small models with the right tools could beat large models without them? Lambda collaborated with researchers from Stanford University, Texas A&M University, and UC San Diego on AgentFlow, a trainable agentic system that rethinks how LLMs solve complex tasks. Instead of asking one model to do everything, AgentFlow assembles a team: Planner, Executor, Verifier, Generator. Each with a distinct role, learning to coordinate and call tools in the flow of a task through end-to-end reinforcement learning. 3B/7B models trained on just 8× NVIDIA A100 GPUs outperform GPT-4o across diverse reasoning benchmarks. The paper was accepted as an Oral presentation at ICLR 2026 (Top 1.1% of submissions) and received a Best Paper Nomination at the NeurIPS 2025 Efficient Reasoning Workshop. Project page: https://lnkd.in/gFWw_VTZ Code: https://lnkd.in/gn8ifjVQ

91

3 Comments