Cohere Labs

Don't forget to tune in tomorrow, October 2nd! 👀 Register now: https://luma.com/9bu7kr6i

Session 3 for World Model from Scratch book is on Thursday, October 1, at 9AM PST. We will look into Chapter 4 — post-training, fine-tuning, and inference-time improvements https://lnkd.in/d2u6UrYv Session Overview of the second session for the book via Cohere Labs 🎙️ Hosted by Nahid Alam, a computer vision researcher and 4-year Cohere community member, currently working on a world model project for drones 🏗️ The session builds on Chapters 1–2: Chapter 1 introduced world models; Chapter 2 covered running inference with the Cosmos model from Nvidia to generate rollouts (possible future video sequences) What Is a World Model? 🌍 A world model is a representation of an environment — visual, robotic, financial, or otherwise — that can reason about future actions within that environment VBench Evaluation 📊 VBench is a video quality benchmark covering many metrics; the session focuses on three: subject consistency, motion smoothness, and dynamic degree 🖼️ Subject consistency: measures DINO embedding similarity between frames; higher = more stable visual features, less distortion 🎞️ Motion smoothness: measures temporal smoothness between intermediate frames 🌊 Dynamic degree: based on optical flow after movement; VBench uses a threshold — videos with movement below it score zero 🎬 Four rollouts (from different seeds) from the Chapter 2 sand mining videos were evaluated 💡 Key takeaway from results: do not rely on a single metric — e.g., Rollout 2 had lower subject consistency (4.9071) despite similar motion smoothness scores to others 🐛 Known VBench issue: timestamped filenames use colons, which break on some systems; a fix is to replace colons with underscores PhyGenBench (PyBench G) Evaluation 📐 PyBench has three categories: U (understanding), C, and G (generation); the session uses PyBenchG 🧩 Key difference from VBench: PyBenchG requires a starting image, a prompt, and a specific set of questions tied to the video content — not just a raw video ⚠️ Because Chapter 2 rollouts lacked associated question sets, they cannot be directly used for PyBenchG evaluation; a separate set of videos with corresponding questions must be used 🧠 PyBenchG uses Qwen as its VQA model internally (not for generation) 🗂️ Evaluation dataset prepared: 2 seeds × 7 guidance values = 14 cases 📏 PyBenchG measures both standard video quality metrics and physical judgment scores (e.g., human activity accuracy, physics accuracy) GPU & Hardware Notes ⚡ A 40 GB GPU was used for the demo; ~32 GB should also work ⚙️ 16 GB is likely enough for evaluation, but probably not training 📂 Participants without a GPU can view uploaded results in the repo Explorer Benchmark Landscape 🔭 Other benchmarks: VideoPhy, climate/weather benchmarks, PDE benchmarks for physical simulation (e.g., aerodynamics, motor control) gitbook: https://lnkd.in/dXv7qDp6 github: https://lnkd.in/gytd7bwG

Only 2.6% of agentic tools match directly with an existing human work task. But does this mean agents are barely touching human work, or are we measuring with the wrong ruler? 💼 📏 Existing measures of AI automation generally begin from taxonomies of human work, and ask whether AI can perform the tasks they record. In this study, we do not start with the assumption that machine work will mirror human work. Rather, we start by understanding the actions made available to machines, and then ask how these functions relate to the existing structure of human work. Exploring tool descriptions from the bottom-up reveals three categories beyond what directly matches existing taxonomies of work: 1️⃣ The largest category is existing human work just described at a finer grain. For example, while occupational databases describe a task such as, "maintain configuration control” - we find tools that exist to read, write, validate, and rollback configs all in separate parts 2️⃣ We also see cases where several human tasks are bundled into one tool, and 3️⃣ The very beginning of work with no precedent in occupational data - mostly the work of managing agents Read about these findings and more here: https://lnkd.in/e-EYDau7

Cohere Labs and Cohere are proud to share some of our latest research next week at COLM 2026, in San Francisco, California! ☀️ We're honored to be featured as part of this conference focused on understanding and improving the development of language technologies.

Stellar opportunity: New co-lead for the Regional Africa group of the Cohere Labs Community wanted! Bronson Bakunga and Kato Steven Mubiru are the best examples of what growth in leadership looks like.🔥

Celebrating 4 years of Sree Harsha Nelaturu as a Cohere Labs Community Lead! 🌟 Harsha has been one of the longest-standing community leads, helping shape what our Open Science Community is today. He launched our ML Theory program - one of our first subfield groups in the community - before going on to lead ML Efficiency and now co-leads the ML Systems & Theory program, creating spaces for researchers to dive deep into foundational ML. 🚀 Beyond leading programs, Harsha has contributed across the community as an Aya Language Ambassador representing multiple languages, served as a captain during Expedition Aya, and shared his expertise as a speaker at Cohere Labs Connect Conference 2025. Thank you, Harsha, for 4 years of thoughtful leadership and for helping build an open, collaborative research community 🩵

Don't forget to tune in tomorrow, September 29th! 👀 Register today: https://luma.com/abhkk5cu

Ever thought about the cultural nuances in voice agents? 🗣️ Check out our community hosted session with Suneel Sunkara as he explores building voice agents for Asian languages. Watch the full discussion: https://lnkd.in/gRiTJdTY

Don't forget to tune in on Monday with Cohere Labs Research Scholar, Mehrnaz Mofakhami! 🤩 Register now: https://luma.com/6fk52mka