Skip to content

loader: support "all:" prefix in //go:embed patterns#5213

Open
abgoyal wants to merge 1 commit intotinygo-org:devfrom
abgoyal:fix-embed-all-prefix
Open

loader: support "all:" prefix in //go:embed patterns#5213
abgoyal wants to merge 1 commit intotinygo-org:devfrom
abgoyal:fix-embed-all-prefix

Conversation

@abgoyal
Copy link

@abgoyal abgoyal commented Feb 21, 2026

Summary

  • Fixes handling of the all: prefix in //go:embed directives. This prefix tells Go to include hidden files (starting with . or _) when embedding a directory.

  • Previously, TinyGo passed patterns like all:dirname literally to path.Match, which never matched any files since no file is actually named "all:dirname". This caused //go:embed all:... to silently embed nothing.

Changes

  • Strip all: prefix before pattern matching
  • Pass includeHidden flag to matchPattern function
  • Only exclude hidden files when includeHidden is false
  • Strip all: prefix before pattern validation in parseGoEmbed
  • Add test case for all: prefix

Example

//go:embed all:static
var content embed.FS
  • Before: content.ReadDir("static") returns "file does not exist"
  • After: Returns all files including .gitkeep, .hidden, etc.

Test

Added test in testdata/embed/ that verifies:

  • //go:embed a excludes .hidden (existing behavior)
  • //go:embed all:a includes .hidden (new behavior)
@abgoyal
Copy link
Author

abgoyal commented Feb 21, 2026

The circleci failed test looks like a flaky test/race unrelated to my patch. Could a maintainer please look into my PR?
eliasnaur
eliasnaur reviewed Feb 21, 2026
View reviewed changes
eliasnaur
eliasnaur reviewed Feb 21, 2026
View reviewed changes
loader/loader.go Outdated
// The "all:" prefix (if present) is stripped and reflected in includeHidden.
type embedPattern struct {
pattern string // the glob pattern (without "all:" prefix)
includeHidden bool // true if "all:" prefix was present
Copy link
Contributor

@eliasnaur eliasnaur Feb 21, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suspect this data structure is just as efficient:

type embedPattern string

func (e embedPattern) Match(file string) bool {
    includeHidden := strings.HasPrefix(s, "all:")
    ...
}
@abgoyal
loader: support "all:" prefix in //go:embed patterns
4a74cf4 
  The "all:" prefix in //go:embed directives instructs the compiler to
  include hidden files (starting with "." or "_") when embedding a
  directory. Previously, the prefix was passed literally to path.Match,
  which would never match.

  Introduce an embedPattern struct that parses and validates the pattern
  at construction time via newEmbedPattern(). The "all:" prefix is
  stripped once and stored along with the glob pattern, avoiding repeated
  string prefix checks. Pattern validation is absorbed into the
  constructor, guaranteeing that any embedPattern value is valid and
  eliminating the need for separate validation loops.

  Fixes embedding with patterns like "//go:embed all:static".
@abgoyal abgoyal force-pushed the fix-embed-all-prefix branch from dfc474a to 4a74cf4 Compare February 21, 2026 15:55
@abgoyal abgoyal requested a review from eliasnaur February 21, 2026 16:51
@abgoyal
Copy link
Author

abgoyal commented Feb 24, 2026

Do you think this one works better? i have incorporated the separate pattern validation into the ep creation itself, and then just use that all over. I think it works well. and instead of striping out the prefix later, I just strip it out once at ep creation time - not really from a performance pov, which i think is same either way, just clarity and non-repetitiveness pov.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

2 participants

@abgoyal @eliasnaur