Conversation
Opt-in hook (?progressive URL param or localStorage.progressiveModels) that wires needle-tools' progressive LOD streaming onto every gltf-model loader. Models without needle LOD data load unchanged, and the library is dynamically imported so its module side effects (decoder-reachability fetch, eager loader construction, Needle global) never run outside the spike. The hook is chained into the component's ready promise so loadModel cannot race registration, and useNeedleProgressive only fills decoders the loader is missing, so A-Frame's DRACO/KTX2/meshopt setup wins. Pinned @needle-tools/gltf-progressive@3.6.1 (MIT, peer three >=0.160 satisfied by our 0.184.0). Webpack code-splits it into a lazy chunk; the main bundle is unchanged for users without the flag. Spike validation still requires a Needle Cloud account to process a >10MB GLB and a browser pass per the plan (texture-immutability, batching and clone-cache interactions, measure-load.mjs numbers). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TZrSKYqoJ6sPuzYjPN3F7F
Commits the approved implementation plan (spike gate, runtime integration, generationJobs/RAD-pattern pipeline, risks) plus current state of work, so a local session can pick the work up from the repo without this web session's conversation context. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TZrSKYqoJ6sPuzYjPN3F7F
Spike passed on a 76 MB Meshy GLB: 3.7 MB progressive initial file, 869 ms to first render vs 32.8 s download plus a >45 s parse freeze for the original, LOD refinement and texture swaps working through our gltf-model loader with no texture-immutability errors. Also documents the needle-cloud CLI usage, served URL pattern, and the KTX2 transcoder dependency the hook introduces.
…CDN URL (#1990 Phase 1) Replace the Phase 0 spike flag with URL-keyed behavior. Saved scenes persist only the gltf-model URL for a user asset, so isProgressiveModelUrl (hostname allowlist) is the single runtime signal that a model streams: - gltf-model hooks needle's extension onto its loader only for progressive srcs (chained into `ready`) and skips the clone-template cache for them, since the extension refines each parsed instance in place with per-instance LOD state. - batch-models gives progressive srcs a null batch key with a distinct skipReason, excluding them from deferral, grouping and late repack (a fixed-buffer BatchedMesh cannot follow runtime LOD swaps). - asset-fallback tries [optimizedSourceUrl, storageUrl] per assetId per session, so an unreachable off-bucket optimized variant falls back to the Firebase original (and a stale token still heals as before). The library stays a lazy webpack chunk; the main bundle grows by ~0.5 KB. The KTX2 transcoder keeps the library's Needle CDN default, fetched only when a Needle-hosted model loads. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01S3JLx2pGXN3eXQnz6vtGoU
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01S3JLx2pGXN3eXQnz6vtGoU
… Phase 2 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01S3JLx2pGXN3eXQnz6vtGoU
…to claude/nifty-brahmagupta-7l771t
…ssing (#1990 Phase 2) Owner-triggered, not automatic: the asset details modal gets a "Make streamable" button that calls the new requestProgressiveGlb callable, which writes a generationJobs doc (kind 'glb-progressive', provider 'cloudrun', tokenCost 0) and enqueues a Cloud Task to the new needle-uploader Cloud Run worker. The worker runs the pinned needle-cloud 2.7.0 CLI (optimize --progressive --usecase world --name <assetId>), resolves the served URL from `list --output json`, patches optimizedSource* on the asset doc and records the Needle content id in the private needleContent ledger. Deletion is now possible: needle-cloud 2.6+ has `delete <identifier>`, so the weekly asset GC asks the worker to trash a purged asset's copy and the ledger doubles as the orphan list when that fails. The reconciler's cloudrun re-enqueue is kind-aware (storagePath → needle-uploader, plyPath → rad-converter). Client: live job status in the modal (subscribes to the job doc, re-reads the asset on success), streaming label/size row, gallery "Optimizing…" badge covers the new kind, private assets are refused (public CDN). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PxguB3shKHAp3RTqnU4p5H
…llout (#1990) Plan doc: Phase 2 as built (manual trigger decision, worker contract, deletion resolved via needle-cloud 2.6+ delete, ledger statuses), what staging needs from Kieran, and the staging test checklist. Agent-context guides, the job-queue provider table and CLAUDE.md pick up the new kind, callable and worker. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PxguB3shKHAp3RTqnU4p5H
kfarr
changed the title
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stream large user-uploaded GLBs with needle-tools
gltf-progressive, processed by Needle Cloud, on the existing generationJobs/cloudrun job pipeline. Draft: built, not yet deployed to staging. The plan, as-built notes and the staging checklist live indocs/plans/1990-progressive-glb-streaming.md; this description is the resume point.Decisions
onCreatetrigger and no client-side size split (Needle processing is metered, the output lives on a third-party public CDN, and the auto threshold is still open).createProgressiveJob()is the seam an automatic path would call later.needle-cloud delete <identifier> --team <team>in CLI 2.6 after we asked (verified on 2.7.0, now pinned): moves the item to the trash, restorable 31 days. The weekly asset GC uses it through the worker.optimizedSourceUrloncloud.needle.tools,optimizationMetadata.format: 'needle-progressive',optimizedSourcePathremoved. No new asset-doc fields, no quota change,getServedUrluntouched.What's on the branch
Runtime (Phase 1, unchanged):
src/tested/progressive-models.js(isProgressiveModelUrlhost allowlist + idempotenthookProgressiveLoader),gltf-modelhooks needle's extension only for those srcs and skips the clone-template cache,batch-modelsexcludes them (progressive-streaming model),asset-fallbackretriesstorageUrlwhen the CDN copy fails. Library stays a lazy chunk; KTX2 transcoder from Needle's CDN.Pipeline (Phase 2, new):
public/functions/progressive-dispatch.js:requestProgressiveGlbcallable (owner-only; refuses non-mesh, deleted, source-less and private assets; one live job per asset) writes agenerationJobsdoc{ kind: 'glb-progressive', provider: 'cloudrun', tokenCost: 0 }and enqueues a Cloud Task (queueglb-progressive, samerad-task-invokerSA). Pure rules inprogressive-rules.js.needle-uploader/Cloud Run worker (Node 22,needle-cloud@2.7.0pinned, token from Secret Manager): download original →optimize --progressive true --usecase world --name <assetId>→ served URL fromlist --output json→ patch the asset doc →needleContent/{assetId}ledger → terminal job status.{ action: 'delete' }trashes a copy for the GC.deploy.sh+ README with a curl one-shot.case 'cloudrun're-enqueues by kind;asset-gc.jsasks the worker to delete the Needle copy on purge, ledger statuseslive → delete-requested → deleted | delete-failed | orphaned(anything notdeletedis the manual purge list).firestore.rules:needleContenthas no client access.assetsService.requestProgressiveVariant+watchAssetJob; modal button, live job status, streaming labels (en/es/pt-BR/fr); gallery "Optimizing…" badge covers the kind.test/core/progressive-rules.test.js,test/core/needle-cli.test.js, streaming cases intest/shared/assets/utils.test.js.Spike results (Phase 0, 2026-09-11)
76.2 MB / 1.67M-tri Meshy building: original 32.8 s download then >45 s parse freeze; progressive 3.7 MB / 869 ms to first render, refined to full mesh on close-up, no texture-immutable errors. Details in the plan doc.
Next: deploy to staging (dev-3dstreet)
optimizeis gated on it), team id, fresh read/write token → Secret Managerneedle-cloud-tokencd needle-uploader && ./deploy.sh dev-3dstreet us-central1 <team>+ one-time IAM listed in the scriptPROGRESSIVE_SERVICE_URLS['dev-3dstreet']; deployrequestProgressiveGlb,reconcileGenerationJobs,purgeSoftDeletedAssets, rules, hostinglist --output jsonstill returnsurl, and Needle's failure wording (decides retry vs skip; isolated inneedle-cli.js)🤖 Generated with Claude Code
https://claude.ai/code/session_01PxguB3shKHAp3RTqnU4p5H