Preserve hand-written entries when regenerating container configs - #4476
Open
pinin4fjords wants to merge 2 commits into
Open
pinin4fjords wants to merge 2 commits into
pinin4fjords wants to merge 2 commits into
Conversation
generate_container_configs() previously deleted and rewrote each conf/containers_*.config file purely from a meta.yml scan, silently discarding any hand-added overrides for modules without per-platform container metadata. This implements the marker convention the Seqera Containers Part 2 blog post describes but that never shipped: content above `// AUTOGENERATED CONFIG BELOW THIS POINT - DO NOT EDIT` is preserved, only the generated section below it is replaced. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
5 tasks
Codecov Report❌ Patch coverage is
☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
generate_container_configs()(behindnf-core pipelines lint --fix container_configsand bulkmodules update/remove/patch) unconditionally deleted and rewrote everyconf/containers_*.configfile from a freshmeta.ymlscan, with no way to keep hand-written entries for modules that don't carry per-platformcontainers:metadata yet.// AUTOGENERATED CONFIG BELOW THIS POINT - DO NOT EDIT), but it was never actually implemented when the generation feature shipped in Command to generate pipeline container config files #3955 - I found this while trying to adopt the scheme in nf-core/rnaseq and hit real data loss risk from it.generate_container_configs()full scan andupdate_module_container_config()single-module update); only the generated section below the marker is replaced. Files with no generated entries but existing manual content are kept rather than deleted. Files without a marker at all (i.e. anything written by a pre-this-PR version of the tool) are treated as fully generated, matching current behaviour, so nothing changes for existing pipelines until they add manual content above a marker themselves.container_configslint docstring to document the convention.Test plan
uv run pytest tests/pipelines/test_container_configs.py tests/pipelines/lint/test_container_configs.py- 14 passeduv run ruff check/uv run ruff format --checkon changed files - cleanuv run mypyon changed files - no new errors (pre-existing errors elsewhere in the codebase are unrelated to this change)devcheckout that the unrelatedtests/modules/failures in this sandbox (network access to GitHub/GitLab) are pre-existing and not caused by this change🤖 Generated with Claude Code