Thong Le

Bad press around autonomous driving is inevitable. Especially when a headline involves a child. Recently, Waymo faced intense scrutiny after one of its vehicles struck a child near a school. The child was thankfully only lightly injured, but the story spread fast, and the reaction was strong. As it should be. What’s getting less attention is the uncomfortable nuance. According to Waymo’s data, the system detected the child immediately, braked hard, and reduced speed significantly before impact. Their internal analysis suggests a typical human driver, even an attentive one, would likely have hit the child at a much higher speed given the same conditions. That does not make the incident acceptable. But it does challenge the way we frame these conversations. We tend to ask: “Did the autonomous system fail?” We rarely ask: “Compared to what baseline?” Human driving sets a very low bar. We just don’t notice it because human errors are normalized. The real question isn’t whether autonomous systems are perfect. They’re not. It’s whether they can consistently make fewer and less severe mistakes than humans, especially in chaotic, high-risk environments like school zones. Public scrutiny is necessary. Transparency is non-negotiable. But progress in safety often looks worse before it looks better, because machine mistakes are visible, logged, and headline-worthy in a way human mistakes never are. If we want safer streets, the comparison has to be honest.

39

13 Comments

Some recent, exciting news on compressed tokenizers and how individual tokens modify images in encoder-decoder models. I hope this opens up a new avenue of research with smaller models across frameworks (perhaps JEPA?). https://lnkd.in/evFw8WBu

69

2 Comments

When it comes to autonomous driving, Tesla and Waymo couldn’t be more different in how they approach the problem, and what they believe it will take to solve it. Waymo is all-in on the “hardware-heavy” school of thought: LiDAR, radar, HD maps, redundant compute, and highly curated geo-fenced environments. It’s a masterpiece of engineering, no doubt. And it works, but only in specific cities, under tightly controlled conditions, and with a very high operational cost. Tesla, on the other hand, is betting on vision and scale. No LiDAR. No HD maps. Just neural networks trained on billions of real-world miles, constantly updated from a global fleet of cars. Less hardware. Lower cost. More scale. And most importantly, a direct path to consumer vehicles, not just robotaxi pilots. From a tech perspective, the contrast is clear: • Waymo builds autonomy like it’s building a #NASA mission. • Tesla builds it like it’s scaling a #smartphone feature. I’m leaning toward Tesla’s approach, not because it’s perfect, but because it’s data-rich, less dependent on expensive hardware, and more #economically viable in the long run. Of course, the race isn’t over. #Regulations, edge cases, and safety validation will shape the outcome. But if autonomy is ultimately a software problem at scale, I think Tesla may be closer than we think. HITEC Angeles Investors The Tech Series Porsche AG Pirelli NASA - National Aeronautics and Space Administration General Motors Rivian BYD U.S. Department of Transportation OpenAI

349

325 Comments

Stock vesting schedules have been morphing, as we all know. And Waymo just slipped a new experiment into the mix. Two recent Waymo offers shared on Levels.fyi tell the story: - An L3 software-engineer package that vests RSUs over two years, 50 percent each year. - An L5 package that keeps the classic four-year, evenly split design. That contrast hints at a deliberate split strategy: shorter horizons for early-career hires, longer runways for senior talent. The logic tracks. New grads often treat their first role as a springboard, so a two-year vest ensures most of their grant actually pays out before they move on. For seasoned engineers, who hold bigger grants and deeper domain knowledge, Waymo still wants the retention pull of a four-year schedule. Waymo isn’t alone. Across tech, equity timelines are turning into a tuning knob: - Lyft issues single-year new-hire grants, then refreshes annually. - Uber front-loads at 35 / 30 / 20 / 15. - NVIDIA moved to 40 / 30 / 20 / 10. - Pinterest compresses to roughly 50 / 33 / 17 over three years. - A handful of high-growth startups grant two-year blocks by default. What’s driving the shift? 1. Pay-for-performance. Shorter cycles let companies re-price grants and reward top performers without being locked into four-year promises. 2. Dilution math. Fewer unvested shares lower equity overhang, especially valuable when stock prices spike. 3. Flexibility. Refreshers become the main retention lever, not year-three and year-four vest cliffs. 4. Talent liquidity. For early-career hires, faster access to equity can feel more tangible than a promise that’s years away. If you’re negotiating, remember that earlier liquidity doesn’t always mean bigger upside. Always compare total grant size, ask how refreshers are determined, and run the tax math on concentrated vesting events. Have you seen a two-year or front-loaded schedule in your offer recently? Comment below with your thoughts and I’ll send over a vesting report we put together here at Levels.fyi. #vestingschedule

137

8 Comments

Get faster and smarter MOE inference straight out of the box. 👇Deep dive on scaling expert parallelism with TensorRT-LLM. LLMs with MOE promise higher model capacity without linearly increasing compute costs - but they introduce new challenges -- more conditional computation, dynamic routing, and non-uniform GPU utilization -- solved by TensorRT-LLM. ✨ New in TensorRT-LLM has native support for expert parallelism—designed for fast, efficient inference with MoE models like Mixtral (Mistral AI) and DeepSeek (DeepSeek AI). This gives you: ✅ Dynamic expert routing: Automatically route tokens to the top-k experts with minimal overhead. ✅ Efficient expert scheduling: Balance expert loads across GPUs using smart sharding and token bucketization. ✅ Memory-aware execution: Maximize hardware utilization while respecting memory budgets. ✅ Drop-in support: Use @HuggingFace models with minimal code changes via TensorRT-LLM's #Python API. 🧠 How it works: MoE models activate only a subset of "experts" for each token. This dynamic nature is powerful—but hard to optimize. It’s all done under the hood using custom #CUDA kernels and NCCL-based communication primitives—giving you low latency, high throughput, and better GPU scaling. ✨ TensorRT-LLM handles: ✅Token-expert mapping using the gating network. ✅Token sorting to batch same-expert tokens together. ✅Expert parallel execution across GPUs. ✅Merging outputs for final predictions. 🛠️ Developer workflow - here is the code to get started. # Clone the repo git clone https://lnkd.in/g-GiDX23 # Use included examples to load and run a Mixtral model cd TensorRT-LLM/examples/mixtral From there, the Python API lets you load the model, convert it with TensorRT, and run expert parallel inference—all with a few lines of code. Results? 📈 Performance at scale. Tests show up to 2.3x faster inference throughput compared to standard tensor parallelism when using 8 GPUs and top-2 experts per token. Even better—TensorRT-LLM keeps efficiency high across increasing batch sizes. Want to see it in action or contribute? 👉 Read the full tech blog: https://lnkd.in/g_7YV3vV 👉 Explore the code on GitHub: https://lnkd.in/gNjQ5W2U 👉 Follow updates in the TensorRT-LLM repo: https://lnkd.in/gqSHYQ4u Share your experiences with us.

78

3 Comments

We have a new blog post on using FlexAttention in FlashAttention CuTe DSL! FlexAttention is a popular PyTorch API for implementing many attention variants through custom score modifications ("score mod"), masking and block sparsity ("mask mod"), or both. However, while the original Triton implementation performed well on NVIDIA Ampere GPUs, it significantly lagged behind on Hopper GPUs compared to optimized kernels like FlashAttention-3. On the other hand, although it was possible in principle to add FlexAttention support to the C++ FlashAttention kernels via an EVT-like abstraction, the complexity of CUTLASS C++ and lack of native JIT compilation made this task unnecessarily cumbersome. Porting the FlashAttention kernels over to CuTe DSL therefore presented an ideal opportunity to revisit FlexAttention in FlashAttention. In collaboration with Driss Guessous and Tri Dao, we've now added a simple FlexAttention-style API to FlashAttention CuTe DSL that has minimal performance loss, achieving up to 95% of FA-3 on Hopper, for example. This work is currently available in the official Dao-AILab repo and is also integrated into PyTorch nightly. See the blog for a thorough user's guide on the API: https://lnkd.in/gETPGUzV

394

5 Comments

🆕 The Ultimate Guide to Open-Source RL Libraries for LLMs: 9 Frameworks Compared 🎯 Three primary RL applications for LLMs: 1️⃣ RLHF for human preference alignment 2️⃣ reasoning models for problem-solving 3️⃣ multi-turn agentic interactions [via Philipp Moritz of Anyscale ] https://lnkd.in/gcF2dNGq

57

1 Comment

NVIDIA's research flipping conventional wisdom on SFT and RL training. Math-only RL substantially improving code reasoning. No code training during RL, yet the model solves previously impossible coding problems. Cross-domain transfer at work. 🙃 The results: 72.6% AIME 2024, 64.8% AIME 2025, 52.1% LiveCodeBench v6. State-of-the-art for 7B models. Sometimes taking a step backwards is the fastest way forward. Check out the paper here 🤓 - https://lnkd.in/ebSzkUe3 #reinforcementlearning #llm #reasoning

12

2 Comments

It’s interesting to see the shift around reinforcement learning — it’s no longer about whether RL is the right approach, but how we can train it more efficiently and at scale. I’m expecting to see a lot more domain-specific applications pop up, both in public and private (enterprise) spaces. https://lnkd.in/gjc3RdcP

18

1 Comment