Cheng Wang

📣 Trillion Labs cut #data prep from days to hours to train highly accurate #LLMs in Korea. To address the lack of high-quality Korean language data, Trillion Labs used NVIDIA #NeMo Curator to accelerate the creation of high-quality datasets to improve Korean language accuracy. Explore the key results: ⏱️ Reduced training time from 24 hours to 3.4 hours for 1.7 trillion tokens of data 📈 Achieved a 5% accuracy increase using 100 billion tokens of data 💡 Realized up to 10x cost and energy savings compared to CPU-based methods 🔗 Read the full customer story: https://bit.ly/45YdhQF #LLM #CPU #AI #DeepLearning

5

🚨#Alert #ArXiv #NVIDIA #KAIST #NationalTaiwanUniversity 🔥 Thrilled to share our recent work (Work done during internship in NVIDIA) The title is "GenRecal: Generation after Recalibration from Large to Small Vision-Language Models". Thanks for our amazing collaborators Ryo Hachiuma, YM RO, Yu-Chiang Frank Wang, Yueh-Hua Wu 🏝️ TL;DR Have you thought how the two vision-language models (VLMs) can be distilled if they have different tokenizer (vocabulary size, token indices, and token order), even with different image embedding strategies (single shot embedding, 4 split image embedding, 12 split image embedding, and dynamically split image embedding). Our primary goal is to do general-purpose distillation framework. 🔨 Solution We present Recalibrator module to make sure that different VLMs's features can be matched in semantic space first, regardless of different tokenizer and image embedding strategies. To do it, we replace the language head of the student VLM with teacher VLM's one and subsequently use only one teacher language head to conduct distillation. 📊 Result Because teacher language head has more richer knowledge due to large dimension itself, we observe that it is really helpful for not only VLMs with different tokenizer but also same one as well. 🔗 Link: https://lnkd.in/gYq8T9bx 👏 Appreciate it if you do upvote it in #HuggingfaceDailyPaper! 🖥️ Please read it at a glance and look at the project page for brief understanding.

242

Maker Faire is a kind of mini-Burning Man for the hobbyists of all kinds, however you can always meet some industrial people having fun over there. I met over there a director of NVidia, an EDA guy from Cadence, a researcher from Berkeley, EE students from UCs and teachers of technology for the school kids. So we at Verilog Meetup are going to put three tables with six computers over there and teach everybody the basics of the SystemVerilog hardware description language using FPGA boards with Gowin chips and graphical LCD screens. You can write something like “always_ff @ (posedge clk) if (x > counter) red <= ‘1;” and see the red moving over the screen. A very affordable way to find out what the hardware design is about - just ask Cliff Cummings, Doulos or Willamette HDL how much they charge for their SystemVerilog tutorials. If you already know SystemVerilog, we have another offering for you - a set of microarchitectural exercises to train yourself for the job interviews. These exercises are popular among both graduate students and EDA AI companies. Here is our booth https://lnkd.in/g4ZNmN_9 See you there!

30

2 Comments

𝗜 𝘄𝗮𝗹𝗸𝗲𝗱 𝗶𝗻𝘁𝗼 𝘁𝗵𝗲 𝗠𝗶𝗰𝗿𝗼𝘀𝗼𝗳𝘁 𝗶𝗻𝘁𝗲𝗿𝘃𝗶𝗲𝘄 𝘁𝗵𝗶𝗻𝗸𝗶𝗻𝗴 𝗜 𝘄𝗮𝘀 𝗿𝗲𝗮𝗱𝘆. 𝗜 𝘄𝗮𝗹𝗸𝗲𝗱 𝗼𝘂𝘁 𝗿𝗲𝗮𝗹𝗶𝘇𝗶𝗻𝗴 𝗽𝗿𝗲𝗽𝗮𝗿𝗮𝘁𝗶𝗼𝗻 𝗶𝘀 𝗺𝗼𝗿𝗲 𝘁𝗵𝗮𝗻 𝘀𝗼𝗹𝘃𝗶𝗻𝗴 𝗟𝗲𝗲𝘁𝗖𝗼𝗱𝗲. It’s about clarity. Structure. And knowing what your work really means. I interviewed for SDE1 at Microsoft, Bellevue. Six rounds. Mix of coding, system thinking, and conversations about real-world work. Sharing my experience here because a lot of engineers prepare hard but sometimes prepare for the wrong things. 𝗪𝗵𝗮𝘁 𝗮𝗰𝘁𝘂𝗮𝗹𝗹𝘆 𝗵𝗲𝗹𝗽𝗲𝗱 𝗺𝗲: 🔹 𝗞𝗻𝗼𝘄 𝘆𝗼𝘂𝗿 𝗼𝘄𝗻 𝗽𝗿𝗼𝗷𝗲𝗰𝘁𝘀 𝗶𝗻𝘀𝗶𝗱𝗲 𝗼𝘂𝘁 They don’t just ask what you built, they ask why you designed it that way. In my case, discussing a recommendation system project turned into a deep conversation on optimization and PyTorch. That discussion mattered more than writing syntax on a whiteboard. 🔹 𝗖𝗹𝗮𝗿𝗶𝘁�� 𝗯𝗲𝗮𝘁𝘀 𝘀𝗽𝗲𝗲𝗱 One question was a classic matrix rotation. Pretty standard. The follow-up changed the shape from square to rectangle. There wasn’t one perfect solution. But talking clearly through edge cases impressed them more than solving fast. 🔹 𝗦𝘆𝘀𝘁𝗲𝗺 𝘁𝗵𝗶𝗻𝗸𝗶𝗻𝗴 𝗶𝘀 𝘂𝗻𝗱𝗲𝗿𝗿𝗮𝘁𝗲𝗱 They asked how I would detect deadlocks across threads. The trick wasn’t writing fancy code. It was mapping the situation to a cycle detection problem in graphs. If you can convert real problems into known concepts, you are already ahead. 🔹 𝗕𝗲𝗵𝗮𝘃𝗶𝗼𝗿𝗮𝗹 𝗿𝗼𝘂𝗻𝗱𝘀 𝗺𝗮𝘁𝘁𝗲𝗿 𝗺𝗼𝗿𝗲 𝘁𝗵𝗮𝗻 𝗽𝗲𝗼𝗽𝗹𝗲 𝘁𝗵𝗶𝗻𝗸 A lot of us ignore this. Microsoft doesn’t. They care about whether you can work in a team, handle disagreements, or explain decisions calmly. Practicing how to talk about your past experiences helps more than cramming 30 more DP problems. 🔹 𝗘𝗱𝗴𝗲 𝗰𝗮𝘀𝗲𝘀 𝗮𝗿𝗲 𝘆𝗼𝘂𝗿 𝗿𝗲𝗮𝗹 𝗶𝗻𝘁𝗲𝗿𝘃𝗶𝗲𝘄𝗲𝗿𝘀 Questions that look simple usually hide tricky edges. The smallest rectangle covering multiple windows. Even odd sorting with internal descending order. These questions are not about syntax. They test how you think through unclear constraints. 𝗕𝗶𝗴𝗴𝗲𝘀𝘁 𝘁𝗮𝗸𝗲𝗮𝘄𝗮𝘆: Good engineering is communication plus logic. If you can explain your decisions, not just code them, you stand out. If you’re preparing right now, focus on explaining things out loud. Even to yourself. Pretend the interviewer is sitting right there. That practice changes how you think. Happy to chat or share more details if anyone wants to prep together or walk through mock interviews. Thanks for reading and wishing strength to everyone preparing right now.

46

15 Comments

Glad to share our another paper about generative retrieval accepted at SlGIR 2025. This work is in collaboration with: Zhen Zhang, Xinyu Ma, Weiwei Sun, Pengjie Ren, Zhumin Chen, Shuaiqiang Wang, Dawei Yin, and Maarten de Rijke. 🔍We noticed most generative retrieval (GR) models are developed for static document collections, leaving their performance in continuously evolving dynamic corpora largely unstudied. Our work provides a systematic evaluation of representative GR approaches over dynamic corpora and found: 1. GR models employing text-based identifiers demonstrate superior generalization capability to unseen documents without requiring retraining. 2. Conversely, GR models using numeric-based docids, despite their efficiency, exhibit a significant performance decline over dynamic corpora. Through in-depth analysis, we identified critical advantages contributing to the robustness of text-based docids in dynamic corpora: (i) semantic alignment with LM pretraining, (ii) fine-grained docid representation, and (iii) high lexical diversity. Our work offers empirical evidence for advancing GR methods over dynamic corpora and paves the way for developing more generalized yet efficient GR models in real-world search engines. 🔗Paper: https://lnkd.in/dvGwjqKP 🔗Code: https://lnkd.in/dYeFWisi

44

1 Comment

Introducing the M5 The M5 chip brings significant advancements in performance and efficiency across various applications. 1. Massive GPU / AI performance boost The M5 features a next-generation 10-core GPU architecture, with each core equipped with an integrated Neural Accelerator. This results in over 4× the peak GPU compute performance for AI tasks compared to the M4. 2. Stronger CPU and enhanced Neural Engine With up to a 10-core CPU (6 efficiency cores + 4 performance cores), the M5 offers approximately 15% faster multithreaded performance over the M4. Additionally, the Neural Engine has been upgraded to 16 cores, enhancing performance for on-device AI and intelligent features. 3. Higher memory bandwidth / unified memory improvements The M5 provides around 153 GB/s of unified memory bandwidth, which is approximately 30% more than the M4. This unified memory architecture allows all components (CPU, GPU, Neural Engine) to access a larger memory pool, enabling larger AI models to run entirely on-device. 4. Graphics & rendering enhancements The improvements in the GPU also boost graphics performance, including ray tracing capabilities. Apple claims up to a ~45% uplift compared to the M4 in ray-traced workloads, enhancing visual fidelity, rendering speed, and smoothness in demanding 3D and graphics applications. 5. Deployment in new hardware + energy efficiency / environmental goals The M5 will power new devices, including the 14-inch MacBook Pro, iPad Pro, and upgraded Vision Pro, all of which are available for pre-order.

7

Excited to share a new top validated submission to #DABstep: Aeya from Tsinghua University Instead of a pure coding agent in a ReAct loop, Aeya’s agent builds relational query plans from a set of relational & _semantic_ operators — then executes them with an engine. The use of semantic operators bridges the gap between the natural language domain to the relational domain and once you get to the relational domain, you don't need arbitrary code generation -- this seems to reduce failure cases significantly!

36

5 Comments

We’re excited to welcome Ray to the PyTorch Foundation 👋 Ray is an open source distributed computing framework for AI workloads, including data processing, model training and inference at scale. By contributing Ray to the PyTorch Foundation, Anyscale reinforces its commitment to open governance and long-term sustainability for Ray and open source AI.    “The PyTorch Foundation is committed to fostering an open, interoperable, and production-ready AI ecosystem. By bringing Ray under the PyTorch Foundation umbrella, alongside projects like vLLM and DeepSpeed, we are uniting the critical components needed to build next-generation AI systems. Ray’s inclusion strengthens our collective mission to support developers with the tools to efficiently train, serve, and deploy AI models at scale.” - Matt White GM of AI at The Linux Foundation Foundation and Executive Director of the PyTorch Foundation “With Ray, our goal is to make distributed computing as straightforward as writing Python code. Joining the PyTorch Foundation helps us stay true to that mission, ensuring Ray continues to be an open, community-driven backbone for developers and their organizations.” - Robert Nishihara, co-founder of Anyscale ➡️ Read the announcement: https://lnkd.in/e3e4mjds #Ray #PyTorchCon #PyTorch #AI #OpenSource #AICompute #DistributedComputing #AIWorkloads #DataProcessing #ModelTraining #Inference

668

5 Comments