What does Radio Explorer do?

It’s not always easy to find radio content across the BBC, especially based on topics you like or interests you have. For instance: I may not be aware of the show name, or presenters on air but at the same time be interested in what I’ve just heard (producers would call me a “light listener”). Perhaps an interview with Lewis Hamilton on Radio 1 had sparked some interest and a listener would like to hear more interviews with him.

So we created an online application which searches not just programmes titles, but subtitles, short synopsis, long synopsis, descriptions, basically anything we can get our hands on! We then made sure any results coming back were available to play, this predominantly included clips (because they don’t expire) and programmes that can currently be played.

Radio Explorer offers you a stream of available content, giving you your own mini radio station of episodes, clips and podcasts which are related to your search term. It’s continuous so you can just leave it to play.

Below is a demo of what Radio Explorer looks like.

(Almost) One year on

Fast forward 8 months and Radio Explorer is on BBC Taster.

When checking feedback most requests were for adding stations such as BBC World Service and Local Radio. So we did exactly this. Radio Explorer now searches through all 57 Radio Stations, giving listeners local content as well as the World Service and music-based interviews from Radio 1 and 6 Music.

Since landing on Taster I was keen to find out what people have been listening to and how long they’ve been listening for.

We have recorded listening times from the past 3 months, i.e measuring how long people listen for and on what brands; these are the results.

Here almost 30% of listens on Radio Explorer have been to an “in Short” clip. Why is this so popular?

In Short, which is Radio 5 Live’s short-form content offering, has a substantial amount of clips in the database (over 4000+), each having a well-filled-out description, meaning they get picked up by Radio Explorer effortlessly. A good description means that Radio Explorer can offer more specific results which users are searching for. The user can also read the description and make a more informed decision to listen or not.

Although listening back to full episodes is available, our listeners seem to be saying short-form content is serving their needs more, at least in this case.

Searches Radio Explorer is always likely to give back content, due to its aggressive keyword searching across all fields in our database

I wanted to see how this fared when compared with the programme search box on the radio home page, here are 10 popular searches performed by users over the past month.

Searches

Radio Explorer is always likely to give back content, due to its aggressive keyword searching across all fields in our database

I wanted to see how this fared when compared with the programme search box on the radio home page, here are 10 popular searches performed by users over the past month.

On average Radio Explorer will almost always give back more results, whereas the radio home page search is currently trying to solve a different problem, namely to get you to the right piece of content. However, entering a category such as ‘comedy‘ will yield far more results (62) than radio explorer’s max which is 30. Seeing that BBC iPlayer Radio’s current search only checks programme titles, specific searches like “Katy Perry”, or “David Bowie” don’t perform too well.

As we move forward and continue to improve BBC iPlayer Radio, I hope we can improve search results across BBC iPlayer Radio and offer audiences a continuous stream to listen to which will include clips & podcasts from across the BBC.

Radio Explorer was active from 29th May 2014 to 15th December 2016

Originally posted on the BBC Internet Blog

It’s not all been plain sailing, it’s been a long journey to get here….

In May 2016 when my interest in Rust began to grow I was looking for a Sublime Package, I noticed one on GitHub, although it was quite inactive. To make matters more complicated the Sublime team decided to fork the package and add it natively to the editor so that users could get Rust support out the box. Although this was convenient it meant that upstream features from the Rust team would not make it in, or could take a long time (2 years ago Sublime’s update cycle was slow). So I decided to sort this out.

The first step was a conversation with the Sublime team, actually just wbond at that time. We came to the conclusion that we should keep this package alive and put all new features in there, wbond also made some fantastic additions, so we brought them in. The basic Rust package (which comes with Sublime now as of 3.0) has syntax highlighting but any new features such as syntax checking, building etc we would bring into our project.

Fast forward a year and we had some contributors come on board, syntax highlighting was improved, syntax checking was added, and we moved to a release based cycle. We changed our name from ‘Rust’ to ‘Rust Enhanced’ (as ‘Rust’ was now taken up by Sublime itself). You can check the releases tab on our Github page to see what’s been added, but we’ve generally been keeping up with features the Rust compiler offers.

2018

After getting feedback from users across rust-lang forums, the most asked about feature by far was RLS. Experimental RLS support has now been added (with use via the LSP package), so feel free to try this out, instructions are on the readme. The 2 packages should work well together although this is still very much work in progress. We hope to improve the experience with RLS this year by working closely with the RLS team and LSP. We also hope to continue improving syntax highlighting (its not perfect yet), and to provide better documentation.

Help is needed, feel free to join us, or even raise any issues you have with it, hopefully we will have another awesome year

But first.. An example. Below is the 6 Music homepage, we keep the paddles on below the scroll bar and visible at every breakpoint.

“Why do we need the paddles?”

The paddles are needed to enable the user to scroll left and right, if these were removed (and the scrollbar removed) the desktop user would not be able to scroll across the carousel to get more content, because the mouse cannot drag the div.

Mobile/Touch devices don’t need paddles, as touch devices can move the scrollable carousel without needing a scrollbar. This is ideal, because paddles can take up quite a bit of screen-real estate, and mobile devices don’t have much.

“Ok, so if we have a touch device it doesn’t need the paddles because the user can swipe?”

Yes, but the problem arises when trying to detect if a device is touch or not.

Detecting touch has become increasingly complex and no longer recommended. See http://www.stucox.com/blog/you-cant-detect-a-touchscreen/

Basically some devices advertise themselves as touch when they’re not and vice versa. Back in 2014 Chrome switched on touchstart permanently even if the user is on a desktop. This broke many websites at the time. https://bugs.chromium.org/p/chromium/issues/detail?id=392584

There are real world examples within the BBC where sites only showed paddles for non-touch-enabled devices and received complaints from users who’s browser was reporting touch but had a keyboard and mouse. There are also an increasing number of hybrid devices (such as Microsoft Surface) that are designed to be operated with touch OR mouse. A user can switch between both modes at any time so a decision cannot be made in advance..

“So we detect mobile and Desktop via breakpoints instead? If a user is below a certain breakpoint they must be on a touch-enabled mobile device?”

This also has issues, people use keyboard and mice on small screen tablets and these can fall into those lower breakpoints (which would then cause the paddles to disappear).

The Microsoft Surface Pro 2 for example, only has a CSS width of 720, but is used with a keyboard/mouse.

iPads (especially the Pro) can end up with a bigger resolution than some desktop so it doesn’t work the other way either.

Although using breakpoints is an improvement, there would still be a margin of error which users would fall into.

Proposal

The best solution is to only show paddles when a user hovers over the page, this means the user must be using a mouse, and we can safely assume they will need them. If a user is on a mobile device, they won’t need the paddles as their browser will allow them to swipe the carousel. This is also the less technically demanding (and therefore robust) approach. It can be done entirely in CSS with body:hover {display: block} and no other detection logic is required.

My first step is to open up Dev Tools and check the networks panel, then look at the initiator column. If you haven’t seen this column before it’s a great way to see what the dependencies are for scripts, or what originally initiated the download of an asset such as a CSS or JS file. You can find more out about the initiator column here.

Unfortunately it’s not always our own scripts that are doing the downloading, it can be jQuery, axios, requireJS or some other utility etc.

It’s most likely you’re using a library to handle your dependency management on the front end, and that can mask the real initiator in the network panel.

Here is an example looking at BBC’s Radio 6 Music’s page. The problem here is obvious, most scripts were pulled in by requireJS.

The BBC makes good use of require to handle asynchronous dependencies, but it can be unhelpful to know the real module doing the initiating. What makes debugging this harder is that requireJS doesn’t expose its dependency tree (because it doesn’t have one), so it’s not possible for me to see what required what else without going through each module and viewing the source.

Thanks to work from the Chromium team blackboxed scripts will also be removed from the initiator tab. This means the library will be removed from the call stack and the next script above (or below depending on how you visualise it) will be shown instead. Our previous example now looks like this.

Here is a gif showing how to enable this feature (this was taken on Chrome Canary 66.0.3328.0)

Blackboxing can also be enabled directly from the Networks Tab, thanks @Aleksey!

When?

This feature has been added to Chrome Canary, and should be available in Chrome v66

It has an estimated release date of April 17th

Links

• https://bugs.chromium.org/p/chromium/issues/detail?id=550453
• https://umaar.com/dev-tips/95-resource-initiator/
• https://developers.google.com/web/tools/chrome-devtools/network-performance/resource-loading?utm_source=dcc&utm_medium=redirect&utm_campaign=2016q3

• Learn Rust
• Learn more about how JS implementations work
• Learn more about the JS specification
• There isn’t a fully fledged compiler in Rust yet for JS
• It’s fun!

The interpreter itself is called Boa and you can join me on every step of the way on this too. First make sure you have Rust installed, check out the repository and build.

It works on very basic javascript, for instance you can create variables, objects, arrays and print their properties, concatenate strings etc. Go take a look and have a play with it.

Below is a working example.

Is Rust the best language for this?

Rust itself is a systems language and certainly at the right level to building an interpreter or compiler. Its next to C/C++ in terms of performance whilst also offering safety, concurrency, a great standard library and modules to help build any program. There will be plenty of scope for parallelism in future and Rust should make this quite safe without me shooting myself in the foot.

Rust has also proved its weight as a language with the performance improvements it’s given to firefox with Project Quantum.

How it works

It starts with a lexer, the job of a lexer is to convert a stream of characters from the source code into tokens. The lexer does this by scanning through characters and analysing when a token starts and finishes, once it has a token it adds it to an array (or vector in Rusts case), this is then sent to the parser.

The parser does a similar job but instead deals with a stream of tokens, it uses these to create expressions. So a couple of tokens representing a function call would generate a CallExpr. Expressions are then evaluated into an Abstract Syntax Tree, this is a high-level expression which represents the whole programme. We then create an executor, which then works it way through the tree and prints out the final value.

I will attempt to go through this post by post, including what i’ve done so far and the pain points i’ve come across, you don’t need to understand Rust to follow these posts.

Hopefully future posts will cover:

• Building a Rust Project
• Lexing/Parsing
• Garbage Collection
• Implementing Objects and prototypes
• Performance
• JITing (@_mttyng has been working on a JIT for this)
• Edge Cases

These posts have since moved to Boa's own blog.

Before we start looking at Rust code, lets go through our dev environment. I use Visual Studio Code with the Rust (RLS) plugin. This plugin is fantastic, and covers pretty much everything you need Rust wise, the Rust Language Server sets itself up mostly, but its worth checking out the Github page and following the steps.

If Sublime is more your thing, check out the Rust Enhanced plugin for Sublime Text 3. It has virtually the same functionality, and with a bit of extra setup you can get the Rust Language Server running too.

I also use Rustup, Rustup is great for managing your Rust version, switching to nightly/stable or adding some additional toolchains.

Finally, to follow along just clone https://github.com/jasonwilliams/boa

Lexical Scanning

When writing an interpreter or a compiler for any language, you usually need to start with a lexer and a parser. Boa here is no different, our first task will be to do the same but what do these do?

A lexer takes a stream of characters and breaks them up into tokens. Tokens are usually split by delimiters in the source code, such as whitespace, new lines or commas. There are plenty of libraries out there that can do this for you, even in Rust, and its recommended to use one, but as a learning exercise I’ve decided to write one from scratch.

Could we use regular expressions for doing this?

Regular expression engines are big, complicated, and usually overkill for this sort of operation. They can also be expensive if our only question is “What is the next character?”. Writing a loop is something that can optimise quite well and will be more lightweight than calling into the regex engine on every unit of text.

Rob Pike explains it best here

Reading the source code

let buffer = read_to_string("tests/js/test.js").unwrap();

We use Rust’s fs::read_to_string method to get the contents of the JS file into a String buffer, it returns a Result type, so we need to unwrap that back into the String. unwrap will attempt to return the wrapped String value, but if it fails it will emit an error and panic. For production code this isn’t great practise, as we want to deal with our errors or at least give a better message. We can improve this by changing unwrap to expect.

let buffer = read_to_string("tests/js/test.js").expect("Unable to read file");

Token Implementation

A token can be a string taken from the source code with some added metadata which gives it context. For example, instead of “function” a token would be {value: “function”, type: “keyword”, line: 2, column: 17}. As you can see a token is just like an object which has some added context around some text, they usually have a type, which would be “keyword”, “Boolean”, “Punctuation” etc, these types help the parser going forward.

We can use a struct to represent a token, if you’re coming from a JavaScript background you can think of a struct like an Object, or better yet a constructor, we can make instances of these. The pub is so that it can be publicly exported from the module its written in, similar to export in javascript. Properties are private by default, so we should make these public too. Here we’re declaring the data types this struct will hold, we can then populate them later.

pub struct Token {
  pub data: TokenData,
  pub pos: Position
}

Below shows TokenData represented as an enum. These are really useful when you have a group or collection of things you want to store as types within a namespace. The different values within the enum are called variants and they are always namespaced under their identifier, so if you wanted to use EOF it would be TokenData::EOF, we can also store the data directly inside the enum variant. If you search through the codebase you can see these being used.

/// Represents the type of Token
pub enum TokenData {
    /// A boolean literal, which is either `true` or `false`
    BooleanLiteral(bool),
    /// The end of the file
    EOF,
    /// An identifier
    Identifier(String),
    /// A keyword
    Keyword(Keyword),
    /// A `null` literal
    NullLiteral,
    /// A numeric literal
    NumericLiteral(f64),
    /// A piece of punctuation
    Punctuator(Punctuator),
    /// A string literal
    StringLiteral(String),
    /// A regular expression
    RegularExpression(String),
    /// A comment
    Comment(String),
}

Position is pretty easy, we just create a struct holding column number and line number, both are unsigned 64 bit integers.

pub struct Position {
    // Column number
    pub column_number: u64,
    // Line number
    pub line_number: u64,
}

So we are working our way through source code, taking each unit of text at a time, generating a token, then adding it to a vector (a growable array on the heap). The bulk of this happens here within the lex function. lex starts a loop which contains a match expression (similar to a switch/case in JavaScript), we then compare every character we come across and do some action against it, each action returns a new token. We keep going until we have no more characters in our buffer left!

In the example below let is an interesting unit to tokenise because it could either be a identifier or a keyword. We know its not a string because there are no quotes around it, its not a number or some punctuation, nor is it a boolean. What we do here is check if it exists in our Keyword’s enum (by calling FromStr which keywords implements). If we get a Keyword value we know we’re sitting on top of a keyword, else we assume its an Identifier. For each token we process, we update the column and row values so they are always correct.

We have now generated a vector of tokens, we can pass these to the Parser to deal with. More on that next time.

Esprit

Quick shoutout to Esprit.

Going forward I may farm out the work of lexing/parsing to a separate library, Esprit was created by Dave Herman, who’s also on the TC39 committee. It is a lexing and parsing library for JS written in Rust, it works in a very similar fashion to what I explained above.

https://esprit.surge.sh

The minimal functionality made it easier to get through specification and into the real world, but this can leave developers with limitations.

Promise combinator landscape

One common limitation I’ve come across is knowing when an array of promises have been settled whether they were fulfilled or rejected. If you were to use Promise.all() it would stop and return after the first rejection giving you a rejected promise; this is the job Promise.all was set out to do however getting the status and value from each promise is equally useful.

Promise.race() can have its uses but wouldn’t help us too much here as it would short circuit after the first promise is settled.

Here is an overview of the current and potential combinators available to the Promise constructor.

Motivation

Needing allSettled functionality is vital when dealing with multiple API fetches in a progressively enhanced application. For instance, on an article page the main content is useful, but you’re happy to disregard additional page furniture if it fails.

Being able to decide which promises are important required us to wrap current promises with extra logic. Workarounds involve looping through each promise, invoking the .then() and returning the result back into a new array. Here is an example:

function reflect(promise) {
  return promise.then(
    (v) => {
      return { status: "fulfilled", value: v };
    },
    (error) => {
      return { status: "rejected", reason: error };
    }
  );
}

const promises = [fetch("index.html"), fetch("https://does-not-exist/")];
const results = await Promise.all(promises.map(reflect));
const successfulPromises = results.filter((p) => p.status === "fulfilled");

And here is an example of the proposed solution in action:

const promises = [fetch("index.html"), fetch("https://does-not-exist/")];
const results = await Promise.allSettled(promises);
const successfulPromises = results.filter((p) => p.status === "fulfilled");

Above we pass 2 fetch promises into allSettled. This returns to us an array of objects giving us the “status” and “value”, or “status” and “reason” if its failed. It’s now easy for us to filter out any rejected promises.

In 2018 I started putting together an ECMAScript proposal to have this in the specification, I’ve had great help from @Mathias Bynens who then went on to champion it at TC39. After passing stage 1 he’s been helping myself and Rob Pamely through the standardisation and spec writing process.

I won’t lie, writing spec text was difficult, between the 3 of us we managed to work our way through.

We say that a promise is settled if it is not pending, i.e. if it is either fulfilled or rejected

States and Fates – Domenic Denicola

If you wish to understand more of the terminology @Domenic put together a great doc called “States and Fates”. whenever I thought I was getting lost with Promises, I would give this a read.

Whats next?

Promise.allSettled is now sitting at Stage 2 which means we have a first version of what will be in the specification.

We will need to spend more time getting feedback on the draft specification, naming, compatibility and any other concerns which appear. Experimental implementations will also be needed, core.js has Promise.allSettled() implemented and you can use this today. As always be weary that implementations can change as specifications do during the stages.

If you’re interested in following the process or contributing, you can view the allSettled repository here: https://github.com/tc39/proposal-promise-allSettled

Questions or comments about this API? Let me know on twitter @jason_williams

Its been a while since I’ve posted about debugging rust, we’ve come a long way since 2017 and I wanted to do an update on getting set up for Rust Debugging.

Installing Rust and VSCode

The best way to install Rust is via Rustup You can grab Visual Studio Code from here You may also need to run rustup component add rust-src if you wish to step into standard library components (mentioned below).

VSCode Extensions

If you’re on MacOS, Linux or Windows then CodeLLDB. Windows users can also use the C/C++ Extension instead if they prefer but for the benefit of this, we will choose CodeLLDB. Regardless of which OS you use I recommend getting Rust Analyzer (RA). It has excellent IDE support for Rust and is being actively developed. If you’ve heard of RLS then Rust Analyzer is a replacement for that (often dubbed RLS 2.0). It is now available on the VSCode Marketplace

Configure VSCode

If you don’t already have a launch.json file you can create one by opening up your menu (Ctrl +shift + p / cmd + shift + p) selecting “Debug: Open launch.json” and select C++ or lldb.

Below is the current configuration from Boa, you can copy and paste this and re-use for your own project.

{
  "version": "0.2.0",
  "configurations": [
    {
      "type": "lldb",
      "request": "launch",
      "name": "Launch",
      "args": [],
      "program": "${workspaceFolder}/target/debug/boa",
      "windows": {
        "program": "${workspaceFolder}/target/debug/boa.exe"
      },
      "cwd": "${workspaceFolder}",
      "stopOnEntry": false,
      "sourceLanguages": ["rust"],
      "sourceMap": {
        "/rustc/*": "${env:HOME}/.rustup/toolchains/stable-x86_64-apple-darwin/lib/rustlib/src/rust"
      }
    },
    {
      "name": "(Windows) Launch",
      "type": "cppvsdbg",
      "request": "launch",
      "program": "${workspaceFolder}/target/debug/boa.exe",
      "stopAtEntry": false,
      "cwd": "${workspaceFolder}",
      "sourceFileMap": {
        "/rustc/5e1a799842ba6ed4a57e91f7ab9435947482f7d8": "${env:USERPROFILE}/.rustup/toolchains/stable-x86_64-pc-windows-msvc/lib/rustlib/src/rust"
      },
      "symbolSearchPath": "https://msdl.microsoft.com/download/symbols",
      "environment": []
    }
  ]
}

Using the Dev Container

Some users prefer to do their development inside the VSCode Dev Container builds. One reason for using a dev container is because the C++ debugger is a bit more aggressive at optimizing away variables whilst the lldb debugger is much better.

Stepping into the Rust Std library

Stepping into is_digit from the standard library

If you wish to step into the Rust standard library code (and some do for curiosity or to help debug a deeper problem) you can do this with sourceFileMap. With CodeLLDB you could use a wildcard (/rustc/*) but that feature has since broke, see tracking issues at the bottom.

On the C/C++ extension this isn’t possible without updating the string each time you update your toolchain. This is because the long string you see after /rustc/[here] depends on the exact toolchain you have installed, so every time you update the toolchain this string will change.

You can find out the string by stepping into a native function and copying the “source location”. It will look something like this:

; id = {0x00000180}, range = [0x0000000100002760-0x00000001000027b0), name="_$LT$alloc
; Source location: /rustc/9fda7c2237db910e41d6a712e9a2139b352e558b/src/liballoc/vec.rs
100002760: 55                         pushq  %rbp
100002761: 48 89 E5                   movq   %rsp, %rbp
100002764: 48 83 EC 20                subq   $0x20, %rsp

Variable is optimized away and not available.

I’m sure you’ve seen this, it can be quite frustrating. Remember, you do get a bit more info using LLDB so consider using that.

There is an initiative to move debug builds to run on Cranelift.

One of the outcomes from this is better handling of Rust variables during debug builds and not optimizing away everything. I think this is the way forward, especially as LLDB will only get us so far

Personally, what would be amazing is if the Rust Analyzer VSCode extension could eventually hook into Cranelift for debugging in the future to make life as easy as possible. Wishful thinking? Maybe.

Tracking Issues (things to keep an eye on):

• https://github.com/microsoft/vscode-cpptools/issues/3022 – Glob Support in the C/C++ plugin.
• https://github.com/bjorn3/rustc_codegen_cranelift/issues/166 – This is Cranelift’s debug tracking issue. If you’re interested in better debugging user experience this is an issue to watch.
• https://github.com/rust-lang/rust/issues/48168 – Ship a custom LLDB with Rust support 
• https://github.com/rust-analyzer/rust-analyzer/issues/2013 – Rust Analyzer being available on the marketplace
• https://github.com/vadimcn/vscode-lldb/issues/264 – Glob support has stopped working

View discussion on Reddit

You can scan through your source code, parse it into an Abstract Syntax Tree (AST) then generate some abstract language (let’s call this Intermediate Representation) which LLVM can understand, LLVM says “Thanks! We’ll take it from here”.

So as long as you can generate something LLVM understands, it can make fast binaries for you, and that’s the advantage; you focus on your syntactical analysis, they focus on generating fast executables.

Rust, C, C++, Swift, Kotlin and many other languages do this and have been doing so for years. Often it’s achieved by having some component that generates LLVM IR.

Is that a compiler backend or frontend?

In Rust, rustc is the main compiler which your source code is fed into first, it does things a typical compiler would do like generate an AST (in this context we call a High-Level Representation, or HIR for short). Once in tree-form type checking and other tasks can be performed then it’s compiled down to another representation ready for whichever backend takes it. The default backend of rustc is called rustc_codegen_llvm (or cg_llvm) which itself also acts as a front end to LLVM.

To make the above more clear, I’ve taken the chart from the 2016 MIR blog post and annotated the responsibilities over it.

From a high level that’s the architecture of Rust in 2020. I say “high level” because the space between LLVM IR and Machine Code alone has multiple steps of compilation and that’s where a chunk of time can go.

Compared to Go, Rust hasn’t been the fastest to compile. Incremental compilation helped a lot but cold-cache builds suffer.

Because LLVM is so efficient at making fast/optimised binaries it’s inefficient at making slow/cheap ones, even with optimisations turned off you still end up with a slow compile and a some-what fast executable. Apart from being a good problem to have it can be a real problem for development because while you’re fixing that bug or testing a function you want quick feedback, and this has irked seasoned rustaceans for some time plus put off new ones.

Compiling development builds at least as fast as Go would be table stakes for us to consider Rust_

Rust Survey 2019

The dilemma starts to compound when you realise LLVM was also designed around compiling C/C++ more than anything else, even an IR needs to come from something and why not use an existing language to model it ?

There seems to be no accurate & explicit specification of the semantics of infinite loops in LLVM IR. It aligns with the semantics of C++, most likely for historical reasons and for convenience

Cranelift

Meanwhile, there exists Cranelift, a [machine] code generator written in Rust developed by the Bytecode Alliance8.

It generates code for WebAssembly and replaces the optimizing compiler in Firefox. It was designed to generate code fast (using parallelism) but is generic enough to be a compile target, meaning just like LLVM you can compile any language to its IR and have it do the rest.

The idea of using it for Rust has floated around for years, and why not? It introduces some healthy competition on the backend, is defined for speedy compilation and the Rust team (plus Mozilla) would be able to help steer the design goals. There’s also the added bonus of giving rustaceans an all-rust compiler for the first time compared to the Rust/C++ hybrid that exists today.

Of course, Cranelift may not be able to catch-up with LLVM’s optimizations and support for all those architectures, but using it for debug builds could prove useful.

One of the things is that LLVM has several layers of IR while Cranelift has only one. Another is that Cranelift does use a graph coloring register allocator, but simply a dumber one, thus being faster.

That’s Bjorn3, he decided to experiment in this area whilst on a summer vacation, and a year & half later single-handedly (bar a couple of PRs) achieved a working Cranelift frontend. The effort here cannot be understated, this is usually the work of an entire team not a curious student in his spare time. There’s worry about the high bus-factor but I can imagine this made the initial development process faster.

I have the freedom to change what I want whenever I want. Sometimes there are problems I can’t solve myself though as I am not familiar enough with the respective area. For example object files, linkers and DWARF debuginfo. Luckily I know people who do know a lot about those things.

So rustc_codegen_cranelift (cg_clif for short) exists and has existed quietly in the background for some time, not only it proved worthwhile as a proof-of-concept, it exceeded expectations. In 2018 measurements showed it being 33% faster to compile. In 2020 we’re seeing anything from 20-80% depending on the crate.11 That’s an incredible feat considering there are more improvements in sight.

There are bits and pieces missing, such as partial SIMD support, ABI Compatibility, unsized values and many more. There’s also lack of feature parity with Cranelift itself, such as cg_clif being blocked because Cranelift doesn’t support a feature LLVM does. However, these problems are shrinking and most crates do build today.

Bringing this together

In April 2020, the rust compiler team decided to catch up with bjorn3 and add cg_clif as a git subtree and “gate on builds”. This means the rust compiler will build against both the LLVM and Cranelift backends then fail the build should either of them not work properly.

cg_clif can be worked on independently whilst having the wider team build against changes whenever they decide to pull in updates.

Although this does not mean the Rust compiler team are officially supporting a Cranelift build, it does offer a step forward for the ecosystem to start thinking about an LLVM/Cranelift future. For instance, the compiler team tested some LLVM features directly, these obviously fail in the Cranelift build, now some thought is put into separating these tests out or at least marking them as “LLVM specific” so other backends can be tested properly.

Below shows the ambition some rustaceans would like to get towards.

Benchmarks

The corpus used for this benchmark is a checkout of Boa specifically commit 8002a95, a built checkout of rustc_codegen_cranelift (8002a95) and rustc 1.44.0-nightly.

This machine is an AMD Ryzen 7 2700X 3.70GHz, 16 CPUs, 32GB memory and an SSD, however, these are running in a container which only has access to 12 CPUs & 12GB memory.

Hyperfine was used with 10 runs of both backends, cargo cleaning between each run.

This benchmark compares the time it takes to build Boa.

The Cranelift backend is a clear winner, knocking off almost a whole minute of build times. I was expecting around 20-30s before running this, so a delta of 56s was quite significant.

The next set of benchmarks were run on a laptop with an Intel® Core™ i3-7130U CPU @ 2.70GHz and an SSD. This gives a more broad view of some popular rust packages being compiled from an empty cache. We’re comparing the avg time to build compared to cg_llvm so 0% would mean they’re the same.

Builds times (cg_llvm baseline)

SIMD support is only partial so that could explain packed-simd and deep-vector but we don’t know that for sure. However on the whole most packages will build faster than they do today. The average improvement today is still around 30% but I’m sure results will only improve as time goes on.

Conclusion

Overall, it’s quite exciting to have a new backend to help with debug builds by delivering much faster build times. The benchmark results look promising and its clear more communication between cg_clif, rustc and cranelift is now happening.

Help is certainly needed. https://github.com/bjorn3/rustc_codegen_cranelift is where the bulk of development is happening, the readme has improved since my first glance. You can run your own benchmarks using a tool like Hyperfine.

The next step on from that would be filing an issue if you come across any problems, or diving into the issues that are already available.

Cranelift parity is also important to unblocking cg_clif, so improvements there are still needed.

With all that being said, a new backend could be one of the most interesting developments this year.

View discussion on Reddit
View discussion on Hacker News

VSCode has a broad range of extensions, from knowledge management to image editors, but what does the growing ecosystem mean for raw performance?

It can often be quick and easy to point at the underlying stack (in this case, Electron) and say that’s where problems lay, but I’ve found it isn’t always the case. This post explores a deep dive into the internals, shows areas that can be improved, plus talks through some changes we may see this year. It should be of interest to anybody who is planning to work on an extension or has a general interest in the performance of VSCode.

A note on architecture

The crux of the design is for extensions to run in a separate process to the UI. This way, they’re more free to do their own thing without competing with the core runtime.

They are written in JS and share the same event loop which is advantageous because:

1. Extensions don’t need to run all the time, so having a dedicated thread per-extension would be overkill and memory intensive.
2. They can yield back control when performing I/O (like reading a file or fetching from the network).
3. Sharing memory/configurations between them works a lot easier.

However, It’s still possible for 1 extension to block another. Below we can see that extensions load in the “renderer” process. If Ext 1 holds the event loop (stuck in a loop or long running action), then the subsequent extensions will also be slow to start. Below is a rough layout.

So, the main lifecycle of each extension is to parse the code (starting from the package.json file), instantiate, and call activate. The running time for this can sometimes be around 300ms.

R.A.I.L , the user-centric performance model for web apps, describes 100ms+ “representing a task” (I.E you know something is happening), and 1000ms+ as “users lose focus on the task they are performing”. Now R.A.I.L is a guide for web, but the same applies in apps such as VSCode too. Having 10 extensions all taking around 300ms is not only 3s of startup time but falls into the realm of noticeable delay. Most extensions don’t need to spend that long starting up and these issues can be avoided.

Let’s look at a practical example.

Case Study: Postfix TS

PR: https://github.com/ipatalas/vscode-postfix-ts/pull/52

Postfix TS is an extension that allows you to add completions to the end of already-existing expressions. So data.log becomes console.log(data). It’s fairly straight forward as far as extensions go, so I was intrigued as to why it had some beefy startup times.

First, I started with the “Developer: Startup Performance” command.

This shows you where time is being spent across the application. There is a section near the top just for extensions.

Let’s focus on 3 columns:

• Load Code (Column 3): How long is spent parsing and executing the source code supplied by the extension (in ms). CPU-intensive script parsing and execution can delay not only other extensions, but also user interaction (not to mention cause battery drain if using a laptop or mobile device). In the above image, load code is the third column in, showing the value 153ms.
• Call Activate (Column 4): How long the extension takes to “activate”. This is the fourth column set to 15ms.
• Event (Column 6): What triggered the extension to activate? This is the sixth column with *.

Event

Let’s start with Event. _ is not ideal. _ means the extension starts immediately, competing with other extensions and VSCode itself during startup. This often isn’t needed, as most extensions don’t do anything until called-upon or just run in the background. An exception to the rule is anything that changes the UI. Having a flash of unstyled-to-styled content can be jarring on the web, this UI is no different, things like sudden syntax colour change can be frustrating.

Extensions that offer codelens (like Gitlens) are fine to be delayed, as they are more of an enhancement to the current view. Plus, they’re not really useful until there’s some interaction with the editor (such as selecting a line).

If we imagine the loading of extensions like a queue from the top image, then it makes sense to have visual changes towards the front and then features that require interaction to be near the back. Developing for the web works the same way; fonts and CSS are loaded as early as possible, whereas some JS uses the defer attribute.

VSCode offers a comprehensive range of different activation events for extensions to use, but if you really need a startup hook, then consider using onStartupFinished. This will kick off your extension after VSCode has loaded and will also give other extensions time to start up. Coming back to Postfix TS, it’s only effective on TypeScript/JavaScript files, so there’s no point loading it any time other than when you’re using these languages. So, let’s change the activation event to:

["onLanguage:javascript", "onLanguage:typescript", "onLanguage:javascriptreact", "onLanguage:typescriptreact"]

This allows VSCode to ignore the extension if I’m not using those languages and will save me a whole chunk of startup time.

Load Code

In the image above, we see 153ms spent on load code. Now, this is relative, but anything much higher than the others tends to mean no bundling is happening. This is a problem because there’s a cost to opening and closing files. If it’s > 100ms, that starts to become noticeable. Loading 600 small files is much slower than loading one large file. When I unpacked the extension I saw entire projects in there, such as TypeScript (that thing is 50MB!). The node_modules folder was over 60MB and had 1,373 files.

If you’re putting together an extension, there’s nothing wrong with using the tsc CLI. It comes with TypeScript and is fully available without needing other packages. But, once you’re ready to distribute your extension (even for testing), you should switch to a bundler. I’ve found ESbuild is the easiest one to get up and running.

I set up an ESBuild workflow; you can see it here. Now only a single file (extension.js) is generated and published.

Reduce code-gen

I see far too many extensions setting ES6 or ES2015 as a compile target; there is no need for this. ES6 is almost seven years old. Almost no one is using a version of code that old.

Updating the compile target means having less code generated. Since newer syntax doesn’t need to be downlevelled, it also means faster build times, as there’s less work for the code transformer to do.

If you’re unsure which to choose, ES2020 is a good target, as that will cover the last few versions back to April 2021. Be sure to set a minimum version higher than v1.56. Anyone using a version lower will continue to use the previous version of your extension.

Results

Before:

58.4 MB (61,314,657 bytes), 201ms startup time

After:

3.43 MB (3,607,464 bytes), 32ms startup time

We’ve shaved off quite a bit of space and time, but remember, this is just one extension, there were many in this shape. I did the same thing with the Open In Github extension.

Between the 2 extensions that’s just under half a second saved. There’s more improvement opportunities to dig into, some of which aren’t available today but are worth highlighting.

ESModules

We saw above that using a bundler really helps bring down both size and load times. Part of this is due to tree-shaking out unused code. However, sometimes the tree-shaking process wrongly includes code because it can’t confidently know whether to leave something out or not.

Today all extensions are exported as CommonJS, which, due to its dynamic nature, is difficult to optimize for bundling. ESModules are more statically analyzable in comparison due to their import/export syntax being standardized and paths needing to be strings. This, coupled with better loading performance (due to its asynchronous nature), should improve overall load/run times. If you’re using a bundler, it should be a simple case of changing your output from CJS to ESM (don’t do this today though, as it won’t work yet).

When will we see a transition? It seems VSCode may be waiting on TypeScript for full ESModules support. You can follow the issue here (feel free to vote on it). The TypeScript team looks to be aiming for a release once they resolve their remaining concerns. I hope to see both of these happening in 2022.

Tree-sitter

Slow loading of large files can be due to syntactical analysis.

Today tokenization (for syntax highlighting) runs on the main thread, if too much time is spent there things will quickly freeze up, so in order to avoid that the syntax highlighting process will periodically yield back until it’s finished. But why is it slow in the first place?

Syntax highlighting uses inefficient textmate grammars which are regex based, these regular expressions can get pretty immense, the example below is from the TypeScript ruleset.

Despite the effort the team have put in to speeding things up, the aging system using these regex grammars is hitting a limit.

The fact that we now have these complex grammars that end up producing beautiful tokens is more of a testament to the amazing computing power available to us than to the design of the [TextMate] grammar semantics.

Alex Dima - Microsoft

Tree-sitter is a new concurrent, incremental parsing system created to solve this problem. The “incremental” bit is note-worthy because it’s designed to handle updates as syntax changes; in fact, it’s fast enough to run on each keystroke. Max Brunfield goes into more detail in his talk about Tree-Sitter here. GitHub migrated to Tree-Sitter for the parsing of syntax and code navigation, NeoVim added experimental support in 2021, and former Atom team members will be moving forward with Zed, a Rust-based text editor that will use Tree-Sitter from the outset.

Web Assembly (WASM)

If you really need to do some CPU intensive work, it’s now possible to offload some of the workload to a language server. This allows you to implement the bulk of your extension in another language (for instance, writing Rust code and compiling it down to WASM).

Thanks to wasm-pack it’s easy to write a Rust extension, or module, and export it. In the following example I am making a change to the Rust code for the server part of the extension which triggers an update, then I can see it being ran on the right hand Code window.

I’ve created a useful template to get started with here: https://github.com/jasonwilliams/hello-wasm

This is made possible not only by wasm-pack, but also by the great esbuild-wasm-pack-plugin that will watch both Rust and TypeScript/JavaScript code, then rebuild on a change.

As Gabe Jackson from OSO explained, bundling to WASM has its advantages. One is that you don’t need to provide a binary for every architecture.

Wrapping Up

So there are changes that can be made today and there are features to look forward to in the future. It will be an interesting year if some of these projects reach prime time. I also believe we’ll see more competition in this space, especially from Zed.

That being said, there are plenty of improvements that can be made in the extensions space today without needing to perform wholesale changes to the architecture.

View discussion on Hacker News
View discussion on Reddit