João Sousa

Robotics models are increasingly bulky and difficult to run directly on robots. With Remi Cadene and the team at LeRobot and Hugging Face, we’re changing that. Today, we're introducing SmolVLA, a sub-500M VLA designed for efficient training and inference. We present to the robotics and open-source community three main contributions: 1️⃣ SmolVLA is *small*. We use: - A small pretrained VLMs - Fewer visual tokens - A layer-skipping mechanism in the VLM - Interleaving self-/cross-attention layers in the action expert ➡️ This allows us to train on a single consumer-grade GPU, and run SmolVLA efficiently, even on CPUs 💥 2️⃣ We ditch proprietary megadatasets, using community-contributed datasets - SmolVLA is trained end-to-end on <30k episodes - All datasets are publicly available, community-contributed and available on Hugging Face Hub - Training this SmolVLA is a community effort, resulting on training on ~10x less data than SOTA VLAs ➡️ Across real-world and simulations, we match much larger models trained on 10x more data 💥 3️⃣ SmolVLA is deployed asynchronously, for greater adaptability - Robots shouldn’t lag (let that sink in) - With SmolVLA, we present an asynchronous inference stack decoupling action execution from planning. ➡️ This results in fast, smooth, and resource-efficient control in the real world (2x throughput!) With this, we hope to push open-source robotics research. We're releasing everything, from data, to training and inference recipes---the Hugging Face way 🤗 SmolVLA has been proudly brought to life by the SmolVLA team, Mustafa Shukor, Dana Aubakirova and yours truly 😊, standing on the shoulders of the entire LeRobot team, and with the guidance of Remi Cadene and Thomas Wolf. Interested? Check out our technical report (🔗 link in the first comment!) Thank you to the whole team behind this project & amazing co-authors Pepijn Kooijmans Michel Aractingi Adil Zouitine Martino Russi Caroline Pascal Andrés Marafioti Thomas Wolf

483

6 Comments

The wait is over! Following the strong interest in 𝐌𝐚𝐩𝐃𝐢𝐟𝐟𝐮𝐬𝐢𝐨𝐧 at IROS 2025, we are thrilled to announce the official open-source release of our code! We believe in contributing to the advancement of autonomous systems, and we are excited to share our work on using generative diffusion models for probabilistic map prediction with the research community. You can now access the full code repository on GitHub: https://lnkd.in/enrCRECn Read the paper for all the technical details here: https://lnkd.in/g7kz3m2d If you find our research valuable, please leave a star ⭐ on the repository and consider citing our paper! #MapDiffusion #OpenSource #AutonomousDriving #Robotics #DeepLearning #ComputerVision #DiffusionModels #AI

265

10 Comments

Interesting blog post on how tracing source code into machine code can be mind bending. Like how this common C function to clip negative values to zero gets transformed to a single Arm instruction. Makes debugging particularly confusing (like how can a breakpoint be put on one of the C branches when there aren't any branches). https://lnkd.in/eDriVwJU

159

68 Comments

🚗 Announcing AugMapNet: Advancing Road Perception for Autonomous Driving! I’m excited to share our latest work on online vectorized HD map construction. We propose Augmentation Map Network (AugMapNet), a novel method for latent BEV grid augmentation that significantly enhances the latent BEV representation. AugMapNet combines vector decoding and dense spatial supervision more effectively than existing architectures while remaining as straightforward to integrate and as generic as auxiliary supervision. 🖼️ We extensively evaluate AugMapNet on two major datasets demonstrating up to 13.3% performance gain over its baseline at a 60m perception range, with even larger improvements at greater distances. We also confirm robust transferability to another architecture. 💡 A key highlight of our work is a detailed introspection into the latent BEV grid, revealing a more structured latent space that translates into superior vector map decoding performance. See below for a qualitative example. 📃 Check out the paper here: https://lnkd.in/e_AQd_6t Joined work with Md Zafar Anwar, Stanislaw Antol, Steffen Staab, Sihao Ding. Thanks for the amazing collaboration! #AutonomousDriving #ComputerVision #MachineLearning #AI #Research

182

5 Comments

When Jake Conway, our Head of Systems Integration at Williams Grand Prix Technologies, sat down to talk about in-the-loop testing, it wasn’t just a technical explanation. It was a glimpse into how we actually engineer confidence into high-stakes systems. Jake’s focus is on making sure systems behave the way we expect them to before we put them into the real world. In-the-loop testing is a core part of that. Whether it's hardware or software in the loop, the idea is the same: simulate the system’s environment so your component thinks it's operating within the real asset. As Jake put it, “Your control unit thinks it’s connected to a multi-million-pound machine, but it’s really talking to a simulator. That’s where the value is. You can explore edge cases, inject faults, and test responses without putting the full system at risk.” What stood out in the conversation was how much this shifts the development timeline. You can run complex tests overnight, capture all the data you need, and make design decisions with confidence, without a person sitting at the bench all day. It’s especially valuable in sectors like defence and aerospace, where projects can run for years before a physical prototype is even built. But Jake is clear that this is not about replacing hands-on testing. It is part of a bigger validation picture. “A bad model gives you bad results, just more of them,” he said. “You still need real engineers who know the failure modes and can stress-test the system with intent.” It is this balance, high-fidelity simulation paired with deep domain knowledge, that defines the way we work at Williams Grand Prix Technologies. For Jake and his team, it is not about ticking boxes. It is about engineering systems that hold up under pressure, even when the stakes are high and the timelines are tight. #WilliamsGrandPrixTechnologies #InTheLoop #EngineeringExcellence

47

1 Comment

🚗✨ Dive into Vehicle Diagnostics – Part 1/2 ✨🚗 As the complexity of modern vehicles grows, so does the importance of robust diagnostics systems. I’m excited to share a two-part overview that delves into the fascinating world of automotive diagnostics. 🔍 Part 1 Highlights: 🔹 Introduction to Vehicle Diagnostics 🔹 On-board vs Off-board Diagnostics 🔹 Addressing Methods used in communication 🔹 Overview of the UDS Protocol (Unified Diagnostic Services) 🔹 Real-time role of the ECM (Electronic Control Module) 🔹 The significance of MIL (Malfunction Indicator Light) – also known as the Check Engine Light 🔹 Role of Data Scanning Tools in diagnostics 🎯 Agenda Covered in Part 1: ✅ Diagnostics Workflow ✅ UDS Protocol ✅ Diagnostic Services ✅ Request & Response Communication Stay tuned for Part 2, where I’ll go deeper into DTC (Diagnostic Trouble Codes) – their structure, classification, and how they help identify faults effectively. Let’s decode the vehicle's inner voice, one signal at a time! 🛠️💡 #AutomotiveDiagnostics #UDSProtocol #ECM #CheckEngineLight #VehicleTechnology #DiagnosticTools #DTC #Part1 #Autosar #Dignostics #AutomotiveDiagnostics #VehicleTechnology #UDSProtocol #DiagnosticTools #ECM #OnBoardDiagnostics #OffBoardDiagnostics #DTC #CheckEngineLight #SmartMobility #AutomotiveEngineering #ConnectedCar #MobilityTech #AutoTech #VehicleMaintenance #TechInAutomotive #AutomotiveInnovation #EngineeringExcellence #FutureOfMobility #DriveSmart

215

3 Comments

The titans of revolutionary SLAMs have returned with a new method this year using ACE this time! Dyson Robotics Lab, Imperial College London present "ACE-SLAM: Scene Coordinate Regression for Real-Time SLAM". They present a novel neural RGB-D SLAM system that learns an implicit map of the scene in real time. For the first time, they explore Scene Coordinate Regression (SCR) as the core implicit map representation in a neural SLAM pipeline, training a lightweight network to map 2D image features directly to 3D global coordinates. SCR networks provide efficient, low-memory 3D map representations, enable extremely fast relocalization, and inherently preserve privacy, making them particularly well-suited for neural implicit SLAM. See the comments for more info and all the links of the team behind this project! #SLAM #realtime #localization #sceneunderstanding #machinelearning #deeplearning #structurefrommotion #futurism #simultaneouslocalisationandmapping

174

2 Comments

🚗💻 The Evolution of Demand for Embedded & Automotive Software Engineers (2010–2028) As the automotive industry races into the future, one thing is clear: software is the new horsepower. Here's a look at how demand for embedded and automotive software engineers has evolved — and where it's headed. 👇 📅 2010–2013: The Embedded Systems Era Begins 🛠️ Traditional embedded systems powered internal combustion engines 🔌 CAN bus, basic ECU logic, and diagnostics dominated 🏭 Tier 1 suppliers led software work; Linux adoption was minimal ⏱️ RTOS and low-level systems were key 📅 2014–2017: Infotainment & Connectivity Surge 📱 Rise of IVI systems (Android Auto, Apple CarPlay) 🌐 Automotive Ethernet and early V2X experimentation 🐧 Linux stacks (GENIVI, AGL) gained traction 🎨 Demand exploded for HMI and multimedia developers 📅 2018–2021: The EV & ADAS Acceleration Phase ⚡ Tesla’s momentum sparked a global EV development boom 🧠 OEMs ramped up investment in ADAS (L2–L3 autonomy) 🤖 Increased need for embedded Linux, sensor fusion, AI/ML skills 🏎️ Software became a true differentiator for OEMs 🧱 AUTOSAR Adaptive emerged for high-performance ECUs 📅 2022–2024: Shift Toward Centralized Architectures 🧩 From dozens of ECUs to domain/zonal controllers 🔄 OTA updates and in-vehicle cybersecurity take center stage ☁️ Cloud integration & DevOps make their way into the automotive pipeline 📈 Software-defined vehicle (SDV) discussions gain real traction 👨‍💻 Demand surges for embedded + cloud + CI/CD skillsets 📅 2025–2028: Consolidation & Recovery Phase 🌍 EV production stabilizes and expands into new regions 🧠 ADAS becomes standard in mid-range cars 🧭 Zonal architecture becomes the new normal 🔐 High demand for engineers in: Real-time safety systems Vehicle cybersecurity Embedded Linux & QNX for advanced controllers 🔮 What's Next? 2029–2032 and Beyond... As we move forward, expect: 🚀 Mass rollout of Software-Defined Vehicles (SDVs) 🌐 Full-scale V2X infrastructure integration 🧠 Edge AI & real-time data learning in vehicles 🤝 Tight integration with smart city ecosystems 💡 Rise of cloud-native automotive software development 💬 What do YOU think? How will the role of software engineers evolve in the next 10 years? Will AI take over parts of the automotive development stack? What skills will define the next generation of automotive talent? 👇 Let's discuss in the comments! #AutomotiveSoftware #EmbeddedSystems #SoftwareEngineering #EVs #SDV #ADAS #V2X #AI #Cloud #QNX #Linux #AutomotiveTech #TechTrends #EngineeringCareers

787

30 Comments

Robotics is moving fast. Talent strategy needs to catch-up. In the past few months I’ve seen: 🔹 Startups scaling from 10 → 30 engineers almost overnight 🔹 Hardware-heavy teams struggling to find mechatronics, system integration and test engineers - especially outside the main hubs 🔹 Candidates weighing up exciting robotics opportunities against offers from big tech 🔹 Companies launching multi-robot fleets in production environments within months 🔹 New robotics hubs emerging outside traditional cities, creating both opportunities and recruitment challenges As someone recruiting only in robotics & autonomy, one thing’s clear: the demand for top talent is outpacing supply - especially in highly specialised areas. And here’s something I see often: companies with poor hiring strategies don’t just lose candidates in the short term - they damage their reputation in the long term. Word of mouth travels fast; I’ve spoken to candidates who categorically won’t consider certain employers because of what they’ve heard from friends or former colleagues. Reputation is a talent strategy. Curious to hear from the community: 👉 For companies – where’s your biggest hiring bottleneck right now? 👉 For candidates – what’s the #1 factor that makes you choose one robotics role over another? I’d be really interested to hear different perspectives here . . . #Robotics #RoboticsRecruitment #HiringStrategy #HiringDemand

47

8 Comments

🚗 Adopting Model-Based Development in Automotive Software 🚗 In the early 2000s, the growing complexity of software motivated us to adopt a Model-Based Development (MBD) approach instead of developing C-Code or Assembler Code. MBD is closer to the human brain, enabling us to shift-left in our testing, which makes the automotive development life cycle more cost-effective by identifying issues earlier.   In this article, Yatish highlights how MBD's advantages enable fast iterations in an agile development process using the Divide-and-Conquer approach.   🔗 Read the full article here: https://lnkd.in/gP95bbsZ #ModelBasedDevelopment

15

You don’t need more robot data. You need to look inside the data you already have. [📍 bookmark for later] Instead of just copying demos, STRAP pulls semantically meaningful pieces from large offline datasets to improve robustness and performance… no fine-tuning needed. Why STRAP is powerful ✅ Boosts few-shot imitation learning by up to 53 percent ✅ Uses sub-trajectory retrieval with S-DTW for better match quality ✅ Leverages foundation model features like DINOv2 out-of-the-box ✅ Scales naturally with data size and works across real and sim environments Thank you for sharing, Marius Memmel! Offline data useful again, for stronger, faster, more general robot learning. 📄 Paper: arxiv.org/abs/2412.15182 🌐 Website: weirdlabuw.github.io/strap

68

1 Comment

The biggest blind spot in modern system architecture? Real-world physics. Many may disagree, but experience proves otherwise. Modern systems are increasingly software intensive, yet many traditional companies hire engineers without a classical engineering background. But when these systems meet reality, what surprises many is how much they depend on real-world physics. This isn’t just about performance – safety can be at stake without it. These are often called cyber-physical systems. What makes these systems so special that architects must understand them? It’s their real – and perceived – system dynamics. Engineers from software backgrounds tend to think in instants. It’s easy: the system issues a command, gets a response, and voilà – mission accomplished. But things aren’t that simple. Every system has its own dynamics. Software systems can be seen as truly instant – electrical signals race through the CPU, RAM, and storage so fast that delay between command and action is negligible. Please don’t throw tomatoes, but only a small fraction of modern system architects have experience beyond using a Mac or Windows PC. Even those with real-world system experience often believe abstractions alone get the job done. This mindset leads to unrealistic expectations of system behaviour – and ultimately, production bugs. A practical example is the electronic parking brake operation. Modern vehicles have at least two braking systems: hydraulic for the service brakes and mechanical for the parking brake. Let’s pretend we have no FuSa, ASIL grades, or vendors delivering ISO 26262 certified components – and try to design things from scratch. As long as the system has two braking circuits, it seems sufficient to expose two functions – ServBrk and ParkBrk – and operate them independently. That gives us three possible states: ◦ ServBrk = Off, ParkBrk = Off – vehicle is movable ◦ ServBrk = On, ParkBrk = Off – vehicle is at a standstill ◦ ServBrk = Off, ParkBrk = On – vehicle is at a standstill Sounds simple enough. To switch from the parking brake to the service brakes, you just: ◦ Disengage the Park Brake ◦ Engage the Service Brakes Sounds logical, right? It should work – no doubt. And it does, until it doesn’t. At some point, what nobody expects will happen: between disengaging Park Brake and engaging Service Brakes, the vehicle briefly enters a free-moving state. It may last a fraction of a second – or sometimes much longer. Experienced engineers know that between these function calls, the real-world parts of the system change state – and this change is much slower than software state change. But it is easily overlooked by rookies or engineers lacking experience with real-world physics. This gap in understanding needs to be addressed promptly. Ignoring physics in system architecture can cause serious issues. How do you ensure your designs consider real-world dynamics? Have you had to consider physics in software-intensive architectures?

399

167 Comments

A new Open Source Suite to Manage Data from your ROS 2 Robots? I’ve been following Mosaico, an upcoming open-source project led by Gabriele Lini, and I genuinely think it could become a new standard about how we handle data in robotics. Most existing tools weren’t built for robotics so they are not always adapted to robotics constrains and it can be challenging to adapt for our use cases. Mosaico is building the robotics suite we’ve been missing to handle our data. The project will drop on GitHub soon, and I’ll definitely keep you posted. For anyone working with ROS 2 platforms, robots, perception systems… this could be a powerful add-on in your toolkit. 🔗 You can follow their Linkedin page to learn more about the project! How are you managing the data of your Robotics projects? Let's connect and share new Robotics projects 🔽 #ROS2 #Robotics #OpenSource #PhysicalAI

106

19 Comments

🏘️🤖 Can robotics really tackle one of the UK’s biggest challenges, the housing shortage? The numbers tell the story: The UK faces a housing shortage unlike any other. 📉 6.5 million homes short as of 2025. 🏗️ To close the gap by 2040, the UK needs to build over half a million homes every year. Last year? Just over 100,000 homes were built. This isn’t just a housing crisis. It’s a clear system bottleneck. And robotics might be the game changer we didn’t expect. In Oxfordshire, AI-driven robots are now framing timber homes. Faster. Greener. Far less waste. The crazy part? 🤯 Some experts believe this approach could deliver up to a third of the UK’s annual housing target. Timber construction is not only scalable but also cheaper and more sustainable than traditional masonry. But here’s what really stands out 💡 Robotics is no longer about replacing workers. It’s about stepping into gaps where human capacity falls short. Industries once thought “untouchable” are suddenly open for disruption. This bottleneck we see today? It’s where robotics and automation will find their biggest opportunities tomorrow. Robotics is not simply entering construction. It is transforming it. Shoutout to innovators like All3, Construction Automation Limited, Hyperion Robotics, BuildingPoint UK & Ireland, and Donaldson Timber Systems who are pushing boundaries in this space. What’s one sector you never expected robots to enter but now find totally plausible? Let’s start a conversation in the comments. PS:- Image is AI-generated for reference only. #Robotics #Automation #ConstructionTech #TimberHomes #AI #FutureOfConstruction #Innovation #SmartConstruction #TechInHousing #HousingCrisis

31

4 Comments

🚦 Announcing IPI — the Intersection Programming Interface (open-sourced) Why this matters • Intersections are evolving fast with CAV & V2X, but deployment is slowed by high barriers and inconsistent interfaces. A software-defined approach with standardized APIs is needed to make upgrades and safety apps deployable at scale.  What IPI is • A clear, open API and a reference implementation that forms a communications fabric linking intelligent intersections with CV/CAVs, pedestrians, peer intersections, and cloud services. It includes an API spec plus a C++ library with SAE J2735 helpers and an ASN.1 UPER codec for MAP/SPaT/BSM/SRM/SSM.  What this unlocks • Interoperability across vendors by aligning with V2X standards (e.g., J2735). • Faster validation & deployment of safety and efficiency applications via a software-defined, API-first design.  If you work in #ITS, #V2X, #CAV, or #SmartCities, I’d love your feedback—review the spec, try the reference library, and open issues/PRs. #ITS #V2X #CAV #AutonomousDriving #SmartInfrastructure #OpenSource The CAR Lab Autoware Foundation TIER IV Toyota North America — Code: • GitHub: https://lnkd.in/exkfQ2UM • Vision article: “A Vision for Transformative Intersections” (IEEE Computer, 2024)

18