Stream typemap fingerprints and PE output

[simonrozsival]

Dependent PR on top of #12599 (simonrozsival-incremental-typemap-emission).

An independent retry of optimization ideas an earlier experimental pass rejected. Every candidate was benchmarked parent-vs-candidate in alternating fresh processes against the real Microsoft.Android.Ref.36 Mono.Android.dll plus fixture workloads, on both the changed and unchanged incremental paths. Three changes cleared the bar; three did not and are not in this PR (details below).

Generated output is byte-identical to the #12599 base — all typemap DLLs and Java sources hash-compared in shared and per-assembly universe modes on both workloads.

Retained

1. Cache the fixed signatures repeated for every emitted peer

CreateInstance and the activation constructor have the same signature on every peer, but each emission rebuilt it through nested SRM signature callbacks. Cache both, and let PEAssemblyBuilder accept a pre-encoded BlobHandle, retaining the existing encoder and IL/max-stack paths.

2. Compute both typemap fingerprints in a single streaming pass

The content fingerprint (which seeds the emitted assembly's deterministic MVID) and the incremental-build fingerprint each walked the entire model into a BinaryWriter over a MemoryStream and then hashed the buffer. Those two buffered walks were the single largest allocation source in the generator.

Both are now serialised from one walk and streamed straight into SHA-256 through a small reusable buffer (FingerprintWriter). Fields shared by the two fingerprints are UTF-8 encoded once and appended to both hashes, so the byte stream each hash sees — and therefore every fingerprint value and generated MVID — is unchanged.

3. Return generated assemblies without a second buffer

Emitting an assembly serialised the PE image into a BlobBuilder and then copied every byte again into a MemoryStream. GeneratedAssembly.Content is now a read-only, seekable stream over the chunks the serialiser already produced (BlobBuilderStream), typed as Stream instead of MemoryStream.

That is all the build task needs: it hashes the stream and copies it to disk through Files.CopyIfStreamChanged, so atomic last-known-good replacement is unchanged and GenerateTrimmableTypeMap needed no edits.

Measurements

These tables isolate changes 2 and 3. The measurement parent is this branch with only change 1 applied — i.e. the signature-encoder caching is present on both sides, so it does not contribute to the deltas below and is not separately quantified here.

Real Microsoft.Android.Ref.36 Mono.Android.dll + Java.Interop.dll as framework inputs. fixtures = the repo test fixtures (8 962 peers, 405 JCWs, 5.3 MB output); bigapp = 1 500 synthetic user ACW types over Activity/Fragment/View/Service/listener bases (10 320 peers, 1 813 JCWs, 7.2 MB output).

Wall time — 12 alternating fresh-process rounds per cell, 3 timed iterations each (n = 36):

workload	path	parent median	candidate median	Δ	parent min	candidate min	Δ
fixtures	changed	180.5 ms	174.5 ms	−3.3 %	156.2 ms	136.6 ms	−12.5 %
fixtures	unchanged	108.2 ms	107.8 ms	−0.4 %	87.9 ms	86.0 ms	−2.2 %
bigapp	changed	291.0 ms	285.8 ms	−1.8 %	236.6 ms	228.0 ms	−3.6 %
bigapp	unchanged	172.2 ms	166.1 ms	−3.6 %	139.9 ms	138.0 ms	−1.4 %

No cell regresses. Total allocated bytes (GC.GetTotalAllocatedBytes(precise: true)) and peak committed GC bytes:

workload	path	universe	alloc parent → cand	Δ	peak parent → cand	Δ
fixtures	changed	shared	109.7 → 80.6 MB	−26.5 %	73.1 → 57.6 MB	−21.2 %
fixtures	changed	per-assembly	109.2 → 80.1 MB	−26.6 %	72.7 → 54.8 MB	−24.6 %
fixtures	unchanged	shared	62.9 → 46.7 MB	−25.8 %	52.9 → 44.5 MB	−15.9 %
fixtures	unchanged	per-assembly	62.4 → 46.2 MB	−26.0 %	53.0 → 44.2 MB	−16.6 %
bigapp	changed	shared	182.5 → 140.4 MB	−23.1 %	86.0 → 66.3 MB	−22.9 %
bigapp	changed	per-assembly	181.5 → 139.4 MB	−23.2 %	83.1 → 65.3 MB	−21.4 %
bigapp	unchanged	shared	112.6 → 88.6 MB	−21.3 %	42.9 → 48.8 MB	+13.8 %
bigapp	unchanged	per-assembly	111.5 → 87.6 MB	−21.4 %	42.6 → 48.2 MB	+13.1 %

The one non-improving cell is committed GC bytes on the bigapp unchanged path. It is a GC budget artifact, not retention: allocations there drop 24 MB, fewer gen0 collections run, and the collector keeps a larger gen0 budget committed. Live managed bytes after a forced full collection are 0.1 MB for both parent and candidate, and wall time on that cell improves 3.6 %.

Allocation traces (Microsoft-Windows-DotNETRuntime allocation sampling, bigapp changed path, 3 iterations) — top frames by sampled bytes:

parent	candidate
MetadataHelper.ComputeIncrementalFingerprint 49.0 MB	BlobBuilder..ctor 11.5 MB (SRM-internal)
MetadataHelper.WriteTypeRef 27.0 MB	BlobBuilder.AllocateChunk 5.7 MB (SRM-internal)
PEAssemblyBuilder.WritePE 21.5 MB	TypeMapAssemblyEmitter.EmitProxyType 4.5 MB
MetadataHelper.ComputeContentFingerprint 13.1 MB	AssemblyIndex.Build 4.4 MB
MetadataHelper.WriteNativeRegistration 9.0 MB	MergeCrossAssemblyAliases 3.7 MB

Every frame changes 2 and 3 target has dropped out of the profile entirely.

Attempted and rejected

Reverted because they did not clear the bar of a reproducible wall-time win or a substantial allocation/peak-memory win at neutral wall time.

Base-constructor-chain caching. Cached generic-substitution-independent registered-constructor metadata (and decoded parameter signatures) per declaring type on AssemblyIndex, applying generic substitutions at consumption, and hoisted the derived-constructor signature decode out of the base-constructor comparison loop. Consistently ~1 % faster in every statistic across five independent measurement sets, but that is under the run-to-run noise floor, allocations moved only 146.5 → 145.9 MB (−0.4 %), and peak committed regressed slightly because the cache retains arrays. A control build that removed CollectBaseConstructorChain outright showed the entire path is worth only ~4.5 % (250.0 → 243.4 ms median), so the ceiling here is small.

Metadata reading strategy. Compared PEStreamOptions.Default, PrefetchMetadata, PrefetchEntireImage, FileOptions.RandomAccess, and a 1-byte FileStream buffer on the 18.5 MB Mono.Android.dll. Medians across four alternating rounds were 249.1 / 248.6 / 254.6 / 251.3 / 249.9 ms — all within noise, with identical managed allocation. No real Mono.Android.dll win, so nothing changed.

Further SRM signature/IL callback allocations. Beyond the caching kept as change 1, I also tried delegate-free BeginSignature/EndSignature scratch encoding plus a BlobHandle overload of the exception-region EmitBody, converting the hottest per-method path (EmitUcoMethod, both its wrapper and callback signatures) to SRM's out-parameter Parameters overload. Byte-identical output — but only after reordering the two signature encodings, because interning them in the opposite order changes the blob heap layout. Worth 140.4 → 138.8 MB (−1.1 %) with no wall-time change. The remaining delegate families are spread across many call sites that each carry the same interning-order hazard, so the reward did not justify the fragility.

Testing

• Standalone Microsoft.Android.Sdk.TrimmableTypeMap.Tests: 806 passed, 0 failed (781 on the Skip unchanged trimmable typemap assembly emission��#12599 base, +25 new).
• New FingerprintWriterTests pin the streaming writer against BinaryWriter/MemoryStream byte-for-byte, including empty, non-ASCII, and values spanning the scratch and sink buffers, and re-run the previous buffered fingerprint serialisation verbatim over a model built from the real fixtures to assert both fingerprints are unchanged.
• New BlobBuilderStreamTests cover multi-chunk reads, partial reads across chunk boundaries, re-reads after rewind (the hash-then-copy pattern the build task uses), all three seek origins, argument validation, write rejection, and that each generated assembly stream still opens as a valid PE image.
• Generated output hash-compared against the Skip unchanged trimmable typemap assembly emission #12599 base: 4 DLLs + 405 Java sources (fixtures, shared and per-assembly universes) and 4 DLLs + 1 813 Java sources (bigapp) — all identical.
• The GenerateTrimmableTypeMap consumption pattern (Files.CopyIfStreamChanged + Dispose) was compiled against the new Stream-typed property to confirm the task needs no change.

No local SDK is available in this environment (bin/Debug/dotnet/dotnet and bin/Release/dotnet/dotnet are absent, and make prepare cannot run here — building Xamarin.Android.Build.Tasks fails on the Java.Interop javac bootstrap and the missing Android SDK/aapt2 bootstrap tasks). Full-build tests — Xamarin.Android.Build.Tests and Microsoft.Android.Sdk.TrimmableTypeMap.IntegrationTests — were therefore not run and should be covered by CI.

[Copilot]

Copilot review overview

🟡 Changes recommended

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review tier: Lite
Findings: 1 · 1 · 1

New issues introduced by this change (3)

Severity	Finding
	src/​Microsoft.Android.Sdk.TrimmableTypeMap/​TrimmableTypeMapTypes.cs — ❌ error — GeneratedAssembly is a public record and the Content type changed from…
	src/​Microsoft.Android.Sdk.TrimmableTypeMap/​Generator/​FingerprintWriter.cs — ⚠️ warning — FingerprintWriter.Write(...) currently silently ignores writes to the incremental…
	src/​Microsoft.Android.Sdk.TrimmableTypeMap/​Generator/​BlobBuilderStream.cs — 💡 suggestion — BlobBuilderStream.Read throws ArgumentOutOfRangeException(nameof(count)) for any…

What changed in this PR

This PR optimizes the trimmable typemap generator to reduce allocations by (1) streaming both typemap fingerprints from a single model walk directly into SHA-256 and (2) returning generated PE images as a read-only, seekable stream over the existing BlobBuilder chunks (avoiding an extra contiguous MemoryStream copy). It also expands unit test coverage to lock down byte-for-byte compatibility and new stream semantics.

Changes:

• Stream content + incremental fingerprints from one walk via a new FingerprintWriter, preserving the legacy BinaryWriter byte stream.
• Surface generated typemap assemblies as Stream (backed by BlobBuilderStream) instead of MemoryStream to avoid a second buffer copy.
• Add focused unit tests for pre-encoded signatures, streaming fingerprints, and BlobBuilderStream behavior.

File	Description
tests/​Microsoft.Android.Sdk.TrimmableTypeMap.Tests/​Generator/​TypeMapAssemblyGeneratorTests.cs	Adds coverage for EmitBody with pre-encoded signatures.
tests/​Microsoft.Android.Sdk.TrimmableTypeMap.Tests/​Generator/​TrimmableTypeMapGeneratorTests.cs	Updates byte comparisons to work with Stream-typed generated assemblies.
tests/​Microsoft.Android.Sdk.TrimmableTypeMap.Tests/​Generator/​FingerprintWriterTests.cs	New tests pin streaming fingerprints against legacy buffered serialization.
tests/​Microsoft.Android.Sdk.TrimmableTypeMap.Tests/​Generator/​BlobBuilderStreamTests.cs	New tests validate BlobBuilderStream read/seek semantics and PE readability.
src/​Microsoft.Android.Sdk.TrimmableTypeMap/​TrimmableTypeMapTypes.cs	Changes GeneratedAssembly.Content from MemoryStream to Stream and documents stream contract.
src/​Microsoft.Android.Sdk.TrimmableTypeMap/​TrimmableTypeMapGenerator.cs	Computes fingerprints once per model and uses GenerateToStream for emitted assemblies.
src/​Microsoft.Android.Sdk.TrimmableTypeMap/​Generator/​TypeMapAssemblyGenerator.cs	Introduces combined fingerprint computation and stream-returning generation path.
src/​Microsoft.Android.Sdk.TrimmableTypeMap/​Generator/​TypeMapAssemblyEmitter.cs	Accepts precomputed content fingerprint and adds EmitToStream path.
src/​Microsoft.Android.Sdk.TrimmableTypeMap/​Generator/​PEAssemblyBuilder.cs	Splits PE serialization into a reusable SerializePE() and adds CreatePEStream().
src/​Microsoft.Android.Sdk.TrimmableTypeMap/​Generator/​MetadataHelper.cs	Replaces buffered fingerprint computation with streaming FingerprintWriter + ModelFingerprints.
src/​Microsoft.Android.Sdk.TrimmableTypeMap/​Generator/​FingerprintWriter.cs	New streaming writer that reproduces BinaryWriter byte encoding into SHA-256.
src/​Microsoft.Android.Sdk.TrimmableTypeMap/​Generator/​BlobBuilderStream.cs	New read-only, seekable stream over BlobBuilder segments.