[Nexthop][NPI] "asic_config_v3" part 1 - basic skeleton#1198
[Nexthop][NPI] "asic_config_v3" part 1 - basic skeleton#1198marif-nexthop wants to merge 4 commits into
Conversation
a6438c6 to
6ec15e0
Compare
6ec15e0 to
8e1c685
Compare
Implements `asic_config_v3` design. Add `fboss/lib/asic_config_v3/` Python package skeleton along with a fully defined CMake build target. This creates the basic infra of the data-driven ASIC config generator (`asic_config_v3`) that will replace the code-driven `asic_config_v2` path. The `gen.py` entry point is included with an empty `_GENERATOR_REGISTRY` so the `fboss-asic-config-v3-gen` target is buildable immediately; registry entries for concrete ASIC families will be added in follow-up commits. Ran `fboss/lib/asic_config_v3/run-helper.sh`.
…transceiver_mapping' in gen target
c51a41f to
c2baba4
Compare
joseph5wu
left a comment
There was a problem hiding this comment.
This base PR looks pretty straightforward to me.
But please make sure not to include unused files or dependencies
| "fboss/lib/platform_mapping_v2/asic_vendor_config.py" | ||
| "fboss/lib/platform_mapping_v2/gen.py" | ||
| "fboss/lib/platform_mapping_v2/helpers.py" | ||
| "fboss/lib/platform_mapping_v2/integrated_transceiver_mapping.py" | ||
| "fboss/lib/platform_mapping_v2/platform_mapping_v2.py" | ||
| "fboss/lib/platform_mapping_v2/port_profile_mapping.py" | ||
| "fboss/lib/platform_mapping_v2/profile_settings.py" | ||
| "fboss/lib/platform_mapping_v2/read_files_utils.py" | ||
| "fboss/lib/platform_mapping_v2/si_settings.py" | ||
| "fboss/lib/platform_mapping_v2/static_mapping.py" |
There was a problem hiding this comment.
Let's only include them when you actually need them
Your current PR doesn't need any platform_mapping_v2 files.
I would like to decouple these two tools
There was a problem hiding this comment.
Done. "Part 1" now declares only the skeleton's stdlib-only sources (init.py, base_generator.py, gen.py, generators/init.py) with no platform_mapping_v2 files and no thrift DEPENDS. The platform_mapping_v2 sources and the thrift dependencies will be added in Parts 3 and 4, in the same PRs as the code that actually imports them. Verified the fboss-asic-config-v3-gen target still builds and runs from a clean build.
| platform_config_python | ||
| switch_config_python | ||
| transceiver_python | ||
| phy_python | ||
| platform_mapping_config_python | ||
| fboss_common_python | ||
| python-pyyaml::python-pyyaml |
There was a problem hiding this comment.
Same comments for the dependency. asic config shouldn't use transceiver_python etc.
Please only uses the necessary dependency
The skeleton in Part 1 only uses the stdlib (json, os, sys, copy, abc, typing) and imports nothing from platform_mapping_v2 or any thrift python target. Drop the platform_mapping_v2 sources and all thrift DEPENDS (platform_config_python, switch_config_python, transceiver_python, phy_python, platform_mapping_config_python, fboss_common_python, python-pyyaml) so the asic_config tool is not coupled to platform_mapping_v2. These sources and dependencies will be reintroduced in the later parts, in the same PRs as the code that actually imports them. Verified the fboss-asic-config-v3-gen target still builds and runs from a clean build.
|
@joseph5wu has imported this pull request. If you are a Meta employee, you can view this in D108936778. |
|
@joseph5wu merged this pull request in 12b2252. |
Summary: **Pre-submission checklist** - [x] I've ran the linters locally and fixed lint errors related to the files I modified in this PR. You can install the linters by running `pip install -r requirements-dev.txt && pre-commit install` - [x] `pre-commit run` Implements `asic_config_v3` design. Add strict JSON schemas for the Broadcom `XGS` ASIC config, the platform config, and the shared vendor common JSONs. Include a validation script that scans the `asic_config_v3` directory tree and validates every matching file against its schema. Pull Request resolved: #1199 Test Plan: Install required package `python3 -m pip install jsonschema` Run schema validation ``` python3 fboss/lib/asic_config_v3/validation/validate_asic_configs.py Vendor common configs (vendor_common.schema.json): No files found. Broadcom XGS ASIC configs (broadcom_xgs_asic_config.schema.json): No files found. Platform configs (platform_config.schema.json): No files found. 0 passed, 0 failed ``` _Note: There are no json files to validate at this point. Will be added in the next PR._ ---- **This PR is created on top of** - #1131 - #1149 - #1198 **so it contains diff from these PRs. When these PRs will merge, this PR will have a shorter diff.** Reviewed By: srikrishnagopu Differential Revision: D109061608 Pulled By: joseph5wu fbshipit-source-id: 52ca32802e0db6b7dd3ac5c97a53ee262a4f15fe
Summary: **Pre-submission checklist** - [x] I've ran the linters locally and fixed lint errors related to the files I modified in this PR. You can install the linters by running `pip install -r requirements-dev.txt && pre-commit install` - [x] `pre-commit run` Implements `asic_config_v3` design. In this PR, introduce the `asic_config_v3` Broadcom XGS generator class along with the shared XGS SDK and SAI common JSONs, and per-ASIC JSONs for Tomahawk 5 and Tomahawk 6. Registered both ASICs in gen.py's `_GENERATOR_REGISTRY` and extend the CMake target SOURCES with the new generator file. After this commit the `asic_config_v3` package can generate XGS configs for any platform whose `asic_config.json` references `tomahawk5` or `tomahawk6`. No platforms are registered yet but will be added in the following PRs. Pull Request resolved: #1220 Test Plan: Running `fboss/lib/asic_config_v3/run-helper.sh` is no-op as there are no platforms added yet. The platforms will be added in the following PRs. Schema validation passes for all files. ``` $ python3 ./fboss/lib/asic_config_v3/validation/validate_asic_configs.py Vendor common configs (vendor_common.schema.json): vendors/broadcom/xgs/sai_common.json: PASSED vendors/broadcom/xgs/sdk_common.json: PASSED Broadcom XGS ASIC configs (broadcom_xgs_asic_config.schema.json): vendors/broadcom/xgs/asics/tomahawk5.json: PASSED vendors/broadcom/xgs/asics/tomahawk6.json: PASSED Platform configs (platform_config.schema.json): No files found. 4 passed, 0 failed ``` ---- **This PR is created on top of** - #1131 - #1149 - #1198 - #1199 **so it contains diff from these PRs. When these PRs will merge, this PR will have a shorter diff.** Reviewed By: daiwei1983 Differential Revision: D109338944 Pulled By: joseph5wu fbshipit-source-id: 454fd40e05ba8504a5929ceee1efdd328e8c2b68
Summary: **Pre-submission checklist** - [x] I've ran the linters locally and fixed lint errors related to the files I modified in this PR. You can install the linters by running `pip install -r requirements-dev.txt && pre-commit install` - [x] `pre-commit run` Implements `asic_config_v3` design. In this PR, add platform configurations for the four core XGS platforms which were originally generated by `asic_config_v2`: - montblanc - icecube800bc - icecube800banw - wedge800bact After this commit the `asic_config_v3` generator produces four XGS output files which should be identical to the `asic_config_v2` output. **Note: After review comments, changed variant name `base` to `default`.** v2: montblanc_base.yml v3: montblanc_defaultyml v2: wedge800bact_base.yml v3: wedge800bact_default.yml Pull Request resolved: #1221 Test Plan: Ran `fboss/lib/asic_config_v3/run-helper.sh` to generate `asic_config_v3` output: ``` Completed build for fboss-asic-config-v3-gen.GEN_PY_EXE ... Writing to /var/FBOSS/fboss/fboss/lib/asic_config_v3/generated_asic_configs/icecube800banw_default.yml ... Writing to /var/FBOSS/fboss/fboss/lib/asic_config_v3/generated_asic_configs/wedge800bact_default.yml ... Writing to /var/FBOSS/fboss/fboss/lib/asic_config_v3/generated_asic_configs/montblanc_default.yml ... Writing to /var/FBOSS/fboss/fboss/lib/asic_config_v3/generated_asic_configs/icecube800bc_default.yml Configs have been written to specified output directory $ ls -1 fboss/lib/asic_config_v3/generated_asic_configs icecube800banw_default.yml icecube800bc_default.yml montblanc_default.yml wedge800bact_default.yml ``` The generated output files are byte-identical with `asic_config_v2` output ``` $ diff -r fboss/lib/asic_config_v3/generated_asic_configs/ fboss/lib/asic_config_v2/generated_asic_configs/ <no diff> ``` Schema validation also passed for all files ``` $ python3 ./fboss/lib/asic_config_v3/validation/validate_asic_configs.py Vendor common configs (vendor_common.schema.json): vendors/broadcom/xgs/sai_common.json: PASSED vendors/broadcom/xgs/sdk_common.json: PASSED Broadcom XGS ASIC configs (broadcom_xgs_asic_config.schema.json): vendors/broadcom/xgs/asics/tomahawk5.json: PASSED vendors/broadcom/xgs/asics/tomahawk6.json: PASSED Platform configs (platform_config.schema.json): platforms/icecube800banw/asic_config.json: PASSED platforms/icecube800bc/asic_config.json: PASSED platforms/montblanc/asic_config.json: PASSED platforms/wedge800bact/asic_config.json: PASSED 8 passed, 0 failed ``` **This PR is created on top of** - #1131 - #1149 - #1198 - #1199 - #1220 **so it contains diff from these PRs. When these PRs will merge, this PR will have a shorter diff.** Reviewed By: daiwei1983 Differential Revision: D109735092 Pulled By: joseph5wu fbshipit-source-id: 936409ca309759f8feab3c29046ca0d8ad645189
Pre-submission checklist
pip install -r requirements-dev.txt && pre-commit installpre-commit runSummary
Implements
asic_config_v3design.Add
fboss/lib/asic_config_v3/Python package skeleton along with a fully defined CMake build target. This creates the basic infra of the data-driven ASIC config generator (asic_config_v3) that will replace the code-drivenasic_config_v2path.The
gen.pyentry point is included with an empty_GENERATOR_REGISTRYso thefboss-asic-config-v3-gentarget is buildable immediately; registry entries for concrete ASIC families will be added in follow-up commits.Test Plan
Ran
fboss/lib/asic_config_v3/run-helper.sh.Note:
fboss/lib/asic_config_v3/run-helper.shwill not generate any output until "asic_config_v3" part 4 PR.This PR is created on top of
so it contains diff from it. When that PR will merge, this PR will have a shorter diff.