fix Pyrefly extension not supported for VSCode "browser" mode (vscode.dev) #2240

[asukaminato0721]

Summary

Fixes #2240

Implemented a web worker–based LSP for vscode.dev using the Pyrefly WASM playground API. This enables hover, completion, definitions, inlay hints, semantic tokens, and diagnostics in VS Code Web.

Added web activation that spins up the worker and indexes the workspace (with size limits) to seed the WASM state.

Added a WASM stub + build wiring so real WASM artifacts can be dropped in, and the build copies the .wasm into dist.

Test Plan

[asukaminato0721]

1. Build the WASM

cd pyrefly_wasm
./build.sh

2. Copy the JS binding into the extension

cp pyrefly_wasm/target/pyrefly_wasm.js lsp/wasm/pyrefly_wasm.js

3. Build the extension

cd lsp
npm install
npm run compile

then code .

pop a window. Test like normal case.

[Copilot]

Pull request overview

This pull request implements web browser support for the Pyrefly VS Code extension to enable usage on vscode.dev. The implementation uses a web worker-based LSP architecture with WebAssembly to provide Python language features in browser environments where native binaries cannot run.

Changes:

• Added web worker LSP server using WASM-compiled Pyrefly with browser-compatible APIs
• Implemented workspace file indexing with size limits (2000 files, 5MB) for web environments
• Modified desktop extension to gracefully handle missing Python extension dependency

Reviewed changes

Copilot reviewed 9 out of 11 changed files in this pull request and generated 6 comments.

Show a summary per file

File	Description
lsp/wasm/pyrefly_wasm.js	WASM bindings generated code providing JavaScript interface to Pyrefly WASM module
lsp/wasm/pyrefly_wasm.d.ts	TypeScript definitions for WASM API including State class and LSP methods
lsp/src/pyrefly-web-worker.ts	Web worker implementing LSP protocol handlers using WASM state management
lsp/src/extension-web.ts	Web extension entry point with worker creation and workspace indexing
lsp/src/extension.ts	Updated to handle optional Python extension for web compatibility
lsp/src/worker-globals.d.ts	Stub type definitions for DedicatedWorkerGlobalScope
lsp/tsconfig.json	Added DOM lib for web API type support
lsp/package.json	Added browser entry point and changed to extensionPack for optional Python extension
lsp/esbuild.js	Build configuration for web bundles with WASM artifact copying
lsp/.vscode/tasks.json	NPM watch task configuration for development
lsp/.vscode/launch.json	Launch configuration for web extension debugging

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[kinto0]

1. Build the WASM

cd pyrefly_wasm ./build.sh

2. Copy the JS binding into the extension

cp pyrefly_wasm/target/pyrefly_wasm.js lsp/wasm/pyrefly_wasm.js

3. Build the extension

cd lsp npm install npm run compile

then code .

pop a window. Test like normal case.

I wonder if you could add this to a dev instructions markdown file somewhere?

additionally, I don't really understand how this gets distributed in it's current form. do we need to update the publish-vsix action to publish a wasm?

[meta-codesync]

@kinto0 has imported this pull request. If you are a Meta employee, you can view this in D93630167.

[kinto0]

looks really great, but I have a few concerns. thanks for picking this up!

[Copilot]

Pull request overview

Copilot reviewed 12 out of 13 changed files in this pull request and generated 1 comment.

Comments suppressed due to low confidence (3)

lsp/tsconfig.json:19

• tsconfig.json excludes src/worker-globals.d.ts, but that file doesn’t appear to exist in lsp/src/. If this was meant to be added as part of the web worker typing setup, it should be included in the PR; otherwise, drop the stale exclude entry to avoid confusion.

        ".vscode-test",
        "src/extension-web.ts",
        "src/pyrefly-web-worker.ts",
        "src/worker-globals.d.ts"

lsp/esbuild.js:81

• In --watch mode the build never runs copyWasmArtifact(). That means dist/pyrefly_wasm_bg.wasm (and any updated wasm glue) may be missing/stale during web extension development, even though the provided .vscode/launch.json uses the watch task. Consider calling copyWasmArtifact() once before starting the watchers (and possibly after successful rebuilds) so the web extension can run reliably in watch mode.

  if (watch) {
    await Promise.all([nodeCtx.watch(), webCtx.watch(), webWorkerCtx.watch()]);
  } else {
    await Promise.all([nodeCtx.rebuild(), webCtx.rebuild(), webWorkerCtx.rebuild()]);
    copyWasmArtifact();
    await Promise.all([nodeCtx.dispose(), webCtx.dispose(), webWorkerCtx.dispose()]);

pyrefly_wasm/lib.rs:99

• This binding returns JsValue::NULL when there are no definition results. The web worker code expects an array and will currently throw if it receives null. Consider returning an empty JS array instead (which is also compatible with LSP’s “no results” semantics once converted), or ensure all JS callers/types explicitly handle null for this API.

    #[wasm_bindgen(js_name=gotoDefinitionLocations)]
    pub fn goto_definition_locations(&mut self, line: i32, column: i32) -> JsValue {
        let results = self
            .0
            .goto_definition_locations(Position::new(line, column));
        if results.is_empty() {
            JsValue::NULL
        } else {
            serde_wasm_bindgen::to_value(&results).unwrap_or(JsValue::NULL)
        }

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

gotoDefinitionLocations can return null (the WASM binding returns JsValue::NULL when there are no results). The handler treats the return value as an array and calls .filter(...), which will throw at runtime when null is returned. Handle the null case (treat as empty array) and/or adjust the WASM API/types so this method always returns an array.

[asukaminato0721]

fixed semantic highlight.

[Copilot]

Pull request overview

Copilot reviewed 12 out of 13 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (8)

lsp/src/pyrefly-web-worker.ts:363

• The publishDiagnostics function only publishes diagnostics for files currently in the files map, but diagnostics from WASM (allDiagnostics) may include errors in files that haven't been opened or indexed yet. This means cross-file import errors or issues in external files won't be shown to the user. Consider also publishing diagnostics for any file mentioned in allDiagnostics, even if it's not in the files map, to ensure users see all errors from type checking.

  for (const filename of files.keys()) {
    const uri = filenameToUri(filename);
    if (!uri) {
      continue;
    }
    connection.sendNotification('textDocument/publishDiagnostics', {
      uri,
      diagnostics: byFile.get(filename) ?? [],
    });
  }

lsp/src/extension-web.ts:64

• When the INDEX_MAX_BYTES limit is reached, indexing stops immediately with a warning but files already read are still sent to the worker. However, this can lead to incomplete type checking results since imported modules may be missing. Consider tracking which files were successfully indexed and adding that information to the warning message (e.g., "Indexed X out of Y files"), or implementing a smarter strategy that prioritizes important files like those in the current workspace folder over deeply nested dependencies.

    if (totalBytes > INDEX_MAX_BYTES) {
      vscode.window.showWarningMessage(
        'Pyrefly web indexing stopped early due to workspace size. Open files will still work.',
      );
      break;

lsp/tsconfig.json:19

• The file 'src/worker-globals.d.ts' is excluded in tsconfig.json but doesn't exist in the codebase. This could cause confusion or issues if the file is created in the future. Consider removing this exclusion since the file doesn't exist, or add a comment explaining why it's excluded if it's intentionally excluded for future use.

        "src/worker-globals.d.ts"

lsp/src/pyrefly-web-worker.ts:188

• The Python version is hardcoded to '3.12' in the WASM State constructor. Unlike the desktop extension which can detect the active Python interpreter via the Python extension, the web version always uses Python 3.12 semantics. This means users working with Python 3.8-3.11 or 3.13+ projects may see incorrect type checking results or missing features. Consider adding a configuration option to let users specify the Python version, or detecting it from pyproject.toml/setup.cfg if available in the indexed files.

    wasmState = new wasmModule.State('3.12');

lsp/esbuild.js:77

• The WASM artifacts are not copied in watch mode, which means developers running the extension in watch mode won't have the necessary WASM files unless they manually run a production build first. This could lead to confusing runtime errors during development. Consider calling copyWasmArtifact() before starting watch mode, or add a plugin that copies WASM files on each rebuild during watch mode.

  if (watch) {
    await Promise.all([nodeCtx.watch(), webCtx.watch(), webWorkerCtx.watch()]);

lsp/src/extension-web.ts:92

• If the WASM file doesn't exist (hasn't been built yet), readFile will throw an unhandled error that will crash the extension activation. Consider wrapping the WASM file reading in a try-catch block and showing a user-friendly error message directing users to build the WASM artifacts with instructions (e.g., "Run pyrefly_wasm/build.sh to enable web support").

  const wasmFileUri = vscode.Uri.joinPath(
    context.extensionUri,
    'dist',
    'pyrefly_wasm_bg.wasm',
  );
  const wasmUri = await vscode.env.asExternalUri(wasmFileUri);
  const wasmBytes = await vscode.workspace.fs.readFile(wasmFileUri);

lsp/package.json:268

• The change from "extensionDependencies" to "extensionPack" changes the semantic meaning. "extensionDependencies" declares required dependencies that must be installed for the extension to work, while "extensionPack" bundles extensions together for convenience but doesn't enforce installation. The desktop Node.js extension still requires the Python extension for interpreter detection (as seen in the try-catch added to extension.ts), but this won't be installed automatically. Consider using both fields, or document this limitation that users must manually install ms-python.python for desktop mode.

    "extensionPack": [
        "ms-python.python"
    ]

lsp/src/pyrefly-web-worker.ts:254

• The filenameToUri function only uses the first workspace root (workspaceRoots[0]), which means files from other workspace folders won't be resolved correctly in multi-root workspaces. When converting filenames back to URIs for cross-file definitions or diagnostics, files outside the first workspace folder will either get incorrect URIs or be silently dropped. Consider iterating through all workspace roots to find the matching one, or storing a filename-to-workspace mapping during indexing.

function filenameToUri(filename: string): string | null {
  const root = workspaceRoots[0];
  if (!root) {
    return null;
  }
  const trimmed = filename.startsWith('/') ? filename.slice(1) : filename;
  const url = new URL(root.uri);
  const base = root.path.endsWith('/') ? root.path : `${root.path}/`;
  url.pathname = `${base}${trimmed}`;
  return url.toString();
}

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[asukaminato0721]

awesome! thanks for the work on this and the prompt replies. it'll take me a bit to test this since I fly out to pycon very soon and won't get a chance to before then.

still curious about the changes to the sandbox. I imagine this also makes the sandbox work multi-file? are there tests that changed for that?

https://github.com/facebook/pyrefly/pull/2460/changes#diff-23aaad95a91940165a87e1b9646ed6879755641d332e2d510583b18afe3fe924R787 I think this count?

test_cross_file_goto_definition_locations for cross-file definition returning filename

[NathanTempest]

Hello @asukaminato0721 thank you for your persistence on working on this issue. We are grateful for your efforts in improving Pyrefly! I have reviewed this PR and here are my findings:

Setup + Blockers
Pulled, built locally (pyrefly_wasm/build.sh + npm run compile in lsp/), ran Run Web Extension against a two-file Python workspace. Reviewing b298620.

Big picture: the slim-down in 2fe4927 was the right call and the new playground.rs::goto_definition_locations + cross-file contract test is a solid foundation. Two surgical fixes needed:

1. gotoDefinitionLocations throws on any token without a definition
pyrefly_wasm/lib.rs:91-99 returns JsValue::NULL when empty; pyrefly-web-worker.ts:520 calls .filter() on it with no guard; pyrefly_wasm.d.ts:73 declares the return as always-array.

Live repro: Cmd-click a variable declaration like bad: str = ... or a literal 42. Dev Tools console:

ERR Request textDocument/definition failed:
    Cannot read properties of null (reading 'filter')
    at handleResponse (extension-web.js:2049:40)

User-visible toast: Request textDocument/definition failed. Triggered by every "what's this?" / pre-rename click on a declaration.

Fix at the source: return an empty array from WASM:

pub fn goto_definition_locations(&mut self, line: i32, column: i32) -> JsValue {
    let results = self.0.goto_definition_locations(Position::new(line, column));
    serde_wasm_bindgen::to_value(&results).unwrap()
}

2. Missing WASM artifact silently disables the extension
extension-web.ts:47 vscode.workspace.fs.readFile(wasmFileUri) is unguarded. A clone with npm install but without pyrefly_wasm/build.sh produces this state: no toast, no Output channel error, no path forward and Pyrefly just silently does nothing. Wrap in try/catch with a showErrorMessage pointing at build.sh.

Also fires from your own F5 watch flow in esbuild.js:77 only calls copyWasmArtifact() in non-watch builds, so the watch-driven launch config hits this every time the wasm hasn't been rebuilt. Adding copyWasmArtifact() before Promise.all([... .watch()]) fixes both.

[NathanTempest]

Cross-file imports break until every dependency is opened
The sharpest edge of the "open files only" approach. With lib.py saved next to main.py and only main.py open:

Cannot find module `lib`
  Did you mean `zlib`?
  Looked in these locations:
  Build system source database
Pyrefly(missing-import)

Opening lib.py in another tab clears it. The worker only knows about files arriving via didOpen. A real project with 20 files looks broken until the user opens every dependency. Two ways to soften without re-introducing speculative full-workspace indexing:

• On-didOpen sibling pull: when a file opens, also readFile siblings in the same folder up to ~100 files / 1 MB and updateSingleFile them.
• Lazy worker-initiated reads: custom request from worker → extension to readFile when state.getErrors() reports a missing import.

Python version hardcoded to '3.12'
pyrefly-web-worker.ts:188 new wasmModule.State('3.12'). Projects on 3.8 or with a pinned python_version get silently wrong errors. Pick one: README note (min) / pyrefly.python.version setting (better) / detect from pyrefly.toml/pyproject.toml among open files (best).

Nits + validation

• Three Pylance namespace collisions at activation
  [meta.pyrefly]: Cannot register 'python.analysis.completeFunctionParens'. ...
  [meta.pyrefly]: Cannot register 'python.analysis.diagnosticMode'. ...
  [meta.pyrefly]: Cannot register 'python.analysis.importFormat'. ...

Pre-existing in the extension (not this PR), but worth a follow-up. Move to python.pyrefly.* (the convention the rest of the extension already uses) so it composes with Pylance.

• Deprecated wasm-bindgen init call
  pyrefly_wasm.js:623 using deprecated parameters for the initialization function
  ensureWasmState @ pyrefly-web-worker.ts:181
  wasmModule.default(wasmUrl) → wasmModule.default({ module_or_path: wasmUrl }). Warning today, future wasm-bindgen will reject.

What I tried:

local build on macOS (clean) → dev host launch → hover, Cmd-click defs, inlay hints, diagnostics all functional alongside Pylance → captured the live Cannot read properties of null trace for blocker #1 → characterized silent failure for #2 by removing the wasm → confirmed cross-file resolution by opening/closing lib.py.

[kinto0]

Two ways to soften without re-introducing speculative full-workspace indexing

This sounds like the biggest dealbreaker. I wonder if we can copy how other web extensions work?

[github-actions]

According to mypy_primer, this change doesn't affect type check results on a corpus of open source code. ✅