Backend: Includes not recursively expanded for required pipeline configuration
<!--- Please read this! Before opening a new issue, make sure to search for keywords in the issues filtered by the "regression" or "bug" label: - https://gitlab.com/gitlab-org/gitlab/issues?label_name%5B%5D=regression - https://gitlab.com/gitlab-org/gitlab/issues?label_name%5B%5D=bug and verify the issue you're about to submit isn't a duplicate. ---> ### Summary When using nested includes in a required pipeline configuration template, the nested includes are not expanded. This was [reported](https://gitlab.zendesk.com/agent/tickets/136966) (Zendesk, internal use only) by a 90-seat premium customer. ### Steps to reproduce 1. Create a project `templates/templates` with the following in `gitlab-ci/required_pipeline.yml`: ``` include: - project: "templates/nested-include" ref: master file: "include.yml" ``` 2. Create a project `templates/nested-include` with the following in `include.yml`: ``` setup_job: stage: test script: - echo "Running $CI_JOB" ``` 3. Set `templates/templates` as the instance-wide template repository, and set `required_pipeline` as the required pipeline configuration file. 4. Create a new project with `.gitlab-ci.yml` file. Observe that CI lint fails with "This GitLab Ci configuration is invalid: include config contains unknown keys: project, ref" ### Example Project N/A ### What is the current *bug* behavior? The validation error occurs because the nested include is not expanded. ### What is the expected *correct* behavior? The nested include should be expanded. ### Relevant logs and/or screenshots (Paste any relevant logs - please use code blocks (```) to format console output, logs, and code as it's tough to read otherwise.) ### Output of checks This bug happens on v12.4.0-ee. ### Possible fixes The problem occurs here: https://gitlab.com/gitlab-org/gitlab/blob/master/ee/lib/ee/gitlab/ci/config_ee.rb#L18 It does not process includes. More details: https://gitlab.com/gitlab-org/gitlab/-/issues/35345#note_441520840 ### User Impact The CI linter fails due to the inability for includes to be expanded and therefore making variables inaccessible. The result is errors noting that the configuration is invalid due to unknown keys. The keys are there but not available to be read. ## Proposal See: - https://gitlab.com/gitlab-org/gitlab/-/issues/35345#note_441520840 - https://gitlab.com/gitlab-org/gitlab/-/issues/35345#note_1559592195 ## MR Implementation | Description | MR / Issue | | ------ | ------ | | | | | | |
issue