Skip to content

Avoid recompiling the shared framework when publishing to the layout root#67469

Merged
wtgodbe merged 3 commits into
dotnet:mainfrom
wtgodbe:publishtodisk-no-rebuild
Jun 30, 2026
Merged

Avoid recompiling the shared framework when publishing to the layout root#67469
wtgodbe merged 3 commits into
dotnet:mainfrom
wtgodbe:publishtodisk-no-rebuild

Conversation

@wtgodbe

@wtgodbe wtgodbe commented Jun 29, 2026

Copy link
Copy Markdown
Member

Problem

The runtime and targeting-pack sfxprojs published the shared framework / targeting pack to the layout root by invoking arcade's PublishToDisk through a nested <MSBuild> call that overrode OutputPath:

<MSBuild Projects="$(MSBuildProjectFullPath)" Targets="PublishToDisk"
         Properties="OutputPath=$(SharedFrameworkLayoutRoot);_PublishingToLayout=true" />

OutputPath is a well-known global property, so overriding it forks a second, non-deduplicated project instance. That duplicate instance runs the framework project's Build (and its copy/output-stage targets) concurrently with the primary instance. The two instances race on the same obj/bin PDBs, which is the intermittent MSB3030 "could not copy ... not found" failure parallel consumers hit (dotnet/msbuild#12927). Note the race is in the copy stage and happens whether or not the duplicate instance actually recompiles — in incremental builds its CoreCompile usually skips as up-to-date but the copy targets still run.

Fix

Run PublishToSharedLayoutRoot AfterTargets="Build" and copy the already-built files to the layout root in the same configuration, so there's no forked instance and no second actor to race.

To get the file list we call _GetAllSharedFrameworkFiles via a nested <MSBuild> and capture its TargetOutputs into a private item rather than depending on GetFilesToPublish directly. This matters: GetFilesToPublish populates the project-global FilesToPackage/_FilesToPackage items that arcade's Pack path (_AddFilesToNuGetPackage) also consumes, so resolving them in this configuration would add every assembly to the package twice (plain + R2R) and trip NU5118. Capturing through TargetOutputs into a private item avoids mutating the packaging items, and passing no global-property overrides means the nested call reuses the already-computed file list (no extra compile). The LayoutTargetPath transform mirrors arcade's GetFilesToPublish for a RuntimePack / TargetingPack.

_PublishingToLayout is removed — it only existed to keep the forked configuration distinct, which is no longer needed.

Files

  • src/Framework/App.Runtime/src/Microsoft.AspNetCore.App.Runtime.sfxproj
  • src/Framework/App.Ref/src/Microsoft.AspNetCore.App.Ref.sfxproj

SharedFrameworkLayoutRoot / TargetingPackLayoutRoot are consumed only by other projects (e.g. test infra) that already build after these, so moving the publish from BeforeTargets to AfterTargets="Build" does not affect consumers.

Validation

Before/after binlog diff (linux-x64):

  • The forked OutputPath=…/SharedFx.Layout/… (and TargetingPack.Layout) project instance is gone.
  • Redundant incremental re-entries drop sharply (skipped CoreCompile invocations 119 → 14); genuine compile count is unchanged (it was already deduped by incremental build).
  • Layout roots are byte-identical before vs after: 136 files under shared/Microsoft.AspNetCore.App/ and 405 under packs/Microsoft.AspNetCore.App.Ref/.

Fixes the recompile/PDB-race half of dotnet/msbuild#12927 and the NU5118 packaging error.

Copilot AI review requested due to automatic review settings June 29, 2026 16:39

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates the build/publish targets used to populate the shared framework “layout root” so that publishing doesn’t trigger a second build of referenced projects under a forked configuration (caused by changing OutputPath in nested MSBuild calls). The intent is to eliminate the intermittent MSB3030 race caused by a brief delete/recreate window for PDBs when project references compile twice.

Changes:

  • Pass BuildProjectReferences=false to nested PublishToDisk MSBuild invocations to prevent rebuilding referenced projects during publish.
  • Add DependsOnTargets="ResolveReferences" to the two *.sfxproj self-invocations so project references are built once in the outer configuration before the nested publish/copy.
  • Add explanatory comments (including the linked msbuild issue) at the affected call sites.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated no comments.

File Description
src/Framework/App.Runtime/src/Microsoft.AspNetCore.App.Runtime.sfxproj Ensures references are resolved/built once before nested publish; disables project-reference rebuilds in nested PublishToDisk.
src/Framework/App.Ref/src/Microsoft.AspNetCore.App.Ref.sfxproj Same pattern for targeting pack: resolve/build references first, then nested publish with BuildProjectReferences=false.
src/Framework/App.Runtime/src/aspnetcore-runtime.proj Updates traversal publish to invoke referenced publish without rebuilding project references.
src/Framework/App.Runtime/src/aspnetcore-runtime-composite.proj Same as above for composite runtime publish traversal.
@wtgodbe wtgodbe marked this pull request as draft June 29, 2026 17:23
PublishToSharedLayoutRoot published the shared framework / targeting pack
by invoking arcade's PublishToDisk through a nested <MSBuild> call that
overrode OutputPath. OutputPath is a well-known global property, so the
override forks a new project configuration; resolving the framework file
list under that forked configuration rebuilds the member assemblies a
second time. The second compile deletes and recreates PDBs in a small
window, causing the intermittent MSB3030 'could not copy ... not found'
race that parallel consumers hit (dotnet/msbuild#12927).

Instead, run PublishToSharedLayoutRoot AfterTargets=Build (so the files
are already built and there is no circular dependency) and copy the
already-built FilesToPublish list to the layout root in the same
configuration, mirroring the Copy that arcade's PublishToDisk performs.
This removes the OutputPath fork and the duplicate compile. The
_PublishingToLayout differentiator is no longer needed because we no
longer reuse the OutputPath-differentiated configuration.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

Comment thread src/Framework/App.Runtime/src/Microsoft.AspNetCore.App.Runtime.sfxproj Outdated
Comment thread src/Framework/App.Ref/src/Microsoft.AspNetCore.App.Ref.sfxproj Outdated
wtgodbe and others added 2 commits June 29, 2026 10:54
Update comments to clarify the publishing process for the shared framework.
Option B ran GetFilesToPublish via DependsOnTargets in the primary
configuration. GetFilesToPublish captures _GetAllSharedFrameworkFiles
into the project-global _FilesToPackage item, which arcade's pack path
(_AddFilesToNuGetPackage) later appends to as well -- doubling every
entry (plain + R2R) and tripping NU5118.

Instead, call _GetAllSharedFrameworkFiles directly and capture its
TargetOutputs into a private item, then replicate arcade's
GetFilesToPublish RuntimePack/TargetingPack transform to compute the
layout destination. This avoids mutating the packaging items while still
reusing the already-built file list (no extra compile).

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@wtgodbe wtgodbe marked this pull request as ready for review June 29, 2026 21:21
@wtgodbe wtgodbe requested a review from ilonatommy June 29, 2026 21:21
@wtgodbe

wtgodbe commented Jun 29, 2026

Copy link
Copy Markdown
Member Author

I think this is a real fix, it gets rid of 105 duplicate project entries. CC @ilonatommy

@ilonatommy ilonatommy left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we apply the same treatment to

? Its PublishToSharedLayoutRoot uses the same OutputPath=$(RedistSharedFrameworkLayoutRoot) override, and its PublishToDisk re-invokes the sfxproj's PublishToDisk with an overridden OutputPath. This seems to trigger the same duplicate build of the framework, only on the redistribution path (not covered by the "byte-identical before vs after" check).

Nit: maybe we should shrink the comments a bit? The reasoning for this change will be in blame history anyway.

Otherwise, I would merge it.

@wtgodbe

wtgodbe commented Jun 30, 2026

Copy link
Copy Markdown
Member Author

Should we apply the same treatment to

aspnetcore-runtime.proj uses the traversal SDK, not the SharedFx SDK, so it rebuilding itself (via PublishToSharedLayoutRoot, not PublishToDisk) never triggers a CoreCompile. And its PublishToDisk target calls MSBuild on its ProjectReferences, not on itself.

@wtgodbe wtgodbe merged commit 1da09bd into dotnet:main Jun 30, 2026
25 checks passed
javiercn pushed a commit that referenced this pull request Jun 30, 2026
…root (#67469)

* Avoid recompiling shared framework when publishing to layout root

PublishToSharedLayoutRoot published the shared framework / targeting pack
by invoking arcade's PublishToDisk through a nested <MSBuild> call that
overrode OutputPath. OutputPath is a well-known global property, so the
override forks a new project configuration; resolving the framework file
list under that forked configuration rebuilds the member assemblies a
second time. The second compile deletes and recreates PDBs in a small
window, causing the intermittent MSB3030 'could not copy ... not found'
race that parallel consumers hit (dotnet/msbuild#12927).

Instead, run PublishToSharedLayoutRoot AfterTargets=Build (so the files
are already built and there is no circular dependency) and copy the
already-built FilesToPublish list to the layout root in the same
configuration, mirroring the Copy that arcade's PublishToDisk performs.
This removes the OutputPath fork and the duplicate compile. The
_PublishingToLayout differentiator is no longer needed because we no
longer reuse the OutputPath-differentiated configuration.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Clarify publishing process in Microsoft.AspNetCore.App.Runtime.sfxproj

Update comments to clarify the publishing process for the shared framework.

* Fix NU5118 by capturing layout file list into a private item

Option B ran GetFilesToPublish via DependsOnTargets in the primary
configuration. GetFilesToPublish captures _GetAllSharedFrameworkFiles
into the project-global _FilesToPackage item, which arcade's pack path
(_AddFilesToNuGetPackage) later appends to as well -- doubling every
entry (plain + R2R) and tripping NU5118.

Instead, call _GetAllSharedFrameworkFiles directly and capture its
TargetOutputs into a private item, then replicate arcade's
GetFilesToPublish RuntimePack/TargetingPack transform to compute the
layout destination. This avoids mutating the packaging items while still
reusing the already-built file list (no extra compile).

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

3 participants