[Nexthop] Simplify run_test.py arguments

[anna-nexthop]

Pre-submission checklist

• I've ran the linters locally and fixed lint errors related to the files I modified in this PR. You can install the linters by running pip install -r requirements-dev.txt && pre-commit install
• pre-commit run

Summary

Simplify the command-line interface for run_test.py by
removing redundant options and consolidating related functionality.

Changes

Removed Options

• Remove --sai-bin and --hw-agent-bin-path option (install WIP binaries in the absolute path instead)
• Remove --asic option (consolidated into --enable-production-features)

Updated Options

• Consolidate --enable-production-features to take ASIC as argument
  instead of being a boolean flag
• Update all test binaries to use absolute paths (/opt/fboss/bin/*)

Bug Fixes

• Fix typo: producition_features -> production_features

Documentation

• Update documentation with simplified command examples
• Add test configuration variables documentation
• Simplify T0/T1/T2 test examples in documentation

Testing

Integration Test Results

Device: fboss101
Build ID: 23679
Test Suite: 11 targeted integration tests
Results: ✅ 11/11 PASSED (100%)

Click to expand full test results

=========================================
Starting Integration Tests for PR https://github.com/facebook/fboss/pull/408
DUT: fboss101
Results Directory: /tmp/pr408_integration_results_20260203_155518
=========================================

========================================
Test 1: Verify --enable-production-features takes ASIC argument
Command: ssh fboss101 -c 'cd /opt/fboss && ./bin/run_test.py sai_agent --help 2>&1 | grep -A3 "enable-production-features"'
========================================
✓ PASSED

Output:
                             [--enable-production-features ASIC]
                             [--platform_mapping_override_path [PLATFORM_MAPPING_OVERRIDE_PATH]]
                             [--agent-run-mode [{mono,multi_switch}]]
                             [--num-npus {1,2}]
--
  --enable-production-features ASIC
                        Enable filtering by production features for the
                        specified ASIC
  --platform_mapping_override_path [PLATFORM_MAPPING_OVERRIDE_PATH]

========================================
Test 2: Test --enable-production-features with valid ASIC (tomahawk5)
Command: ssh fboss101 -c 'cd /opt/fboss && timeout 30 ./bin/run_test.py sai_agent --list_tests --config ./share/hw_test_configs/meru400biu.agent.materialized_JSON --enable-production-features tomahawk5 2>&1 | tail -10'
========================================
✓ PASSED

Output:
  VerifyPortRateTraffic/1  # GetParam() = 100000
  VerifyPortRateTraffic/2  # GetParam() = 200000
  VerifyPortRateTraffic/3  # GetParam() = 400000
test/AgentHwPtpTcProvisionTests.
  VerifyPtpTcDelayRequest/TWENTYFIVEG  # GetParam() = 25000
  VerifyPtpTcDelayRequest/FIFTYG  # GetParam() = 50000
  VerifyPtpTcDelayRequest/HUNDREDG  # GetParam() = 100000
  VerifyPtpTcDelayRequest/TWOHUNDREDG  # GetParam() = 200000
  VerifyPtpTcDelayRequest/FOURHUNDREDG  # GetParam() = 400000

========================================
Test 3: Test --enable-production-features with invalid ASIC shows error
Command: ssh fboss101 -c 'cd /opt/fboss && timeout 10 ./bin/run_test.py sai_agent --list_tests --config ./share/hw_test_configs/meru400biu.agent.materialized_JSON --enable-production-features invalid_asic_xyz 2>&1 || true' | grep "Error: ASIC"
========================================
✓ PASSED

Output:
Error: ASIC 'invalid_asic_xyz' not found in production features file.

========================================
Test 4: Verify --asic option is removed
Command: ssh fboss101 -c 'cd /opt/fboss && ./bin/run_test.py sai_agent --help 2>&1' | grep -c "\-\-asic"
========================================
✓ PASSED

Output:
0

========================================
Test 5: Verify --sai-bin option is removed from global options
Command: ssh fboss101 -c 'cd /opt/fboss && ./bin/run_test.py --help 2>&1' | grep -c "\-\-sai-bin"
========================================
✓ PASSED

Output:
0

========================================
Test 6: Verify --hw-agent-bin-path option is removed
Command: ssh fboss101 -c 'cd /opt/fboss && ./bin/run_test.py sai_agent --help 2>&1' | grep -c "\-\-hw-agent-bin-path"
========================================
✓ PASSED

Output:
0

========================================
Test 7: Verify run_test.py references absolute paths for test binaries
Command: ssh fboss101 -c 'sudo cat /opt/fboss/bin/run_test.py | grep -c "/opt/fboss/bin/sai_test-sai_impl"'
========================================
✓ PASSED

Output:
1

========================================
Test 8: Verify typo fix: production_features (not producition_features)
Command: ssh fboss101 -c 'sudo cat /opt/fboss/bin/run_test.py' | grep -c "producition_features"
========================================
✓ PASSED

Output:
0

========================================
Test 9: Verify run_test.py has valid Python syntax
Command: ssh fboss101 -c 'sudo cp /opt/fboss/bin/run_test.py /tmp/run_test_syntax_check.py && python3 -m py_compile /tmp/run_test_syntax_check.py && echo "Syntax check passed" && sudo rm -f /tmp/run_test_syntax_check.py*'
========================================
✓ PASSED

Output:
Syntax check passed

========================================
Test 10: Verify run_test.py can be imported as module
Command: ssh fboss101 -c 'cd /opt/fboss/bin && python3 -c "import sys; sys.path.insert(0, \".\"); import run_test" && echo "Import successful"'
========================================
✓ PASSED

Output:
Import successful

========================================
Test 11: Verify --production-features option removed from qsfp subcommand
Command: ssh fboss101 -c 'cd /opt/fboss && ./bin/run_test.py qsfp --help 2>&1' | grep -c "\-\-production-features"
========================================
✓ PASSED

Output:
0

=========================================
Test Summary
=========================================
Total Tests: 11
Passed: 11
Failed: 0
=========================================
Results saved to: /tmp/pr408_integration_results_20260203_155518

All tests passed!

[AarjunC]

We should not remove the bin path arg. Instead can we make it optional? If not provided, use the default binary path, but user can still have an option to use different binary.

[anna-nexthop]

@AarjunC Can you give some context why you would like to keep the bin option? It was already optional, but made it confusing why you might need to provide it. Also, why only for the hw and sai test binaries? If it's just for testing purposes, why not just overwrite the binary at the default path?

[meta-codesync]

@AarjunC has imported this pull request. If you are a Meta employee, you can view this in D100650164.

[AarjunC]

Please check review comments.

[facebook-github-tools]

@anna-nexthop has updated the pull request. You must reimport the pull request before landing.

[facebook-github-tools]

@anna-nexthop has updated the pull request. You must reimport the pull request before landing.

[AarjunC]

test_binary_name != "qsfp_hw_test" will fail. _get_test_binary_name() returns "/opt/fboss/bin/qsfp_hw_test".

Can we make /opt/fboss/bin as a const value?

Enhancement suggestion:- Can we log error in _get_test_binary_name() if file is not found? This will be helpful if vendor scp fboss in location other than /opt/

[anna-nexthop]

@AarjunC This check in the abstract class seems really fragile and I see that more than just the QSFP hw test runner doesn't need a conf file. What about if I update this check that if a conf_file is provided (isn't None or an empty string), then the path exists check is ran and the updated error message to be printed is that the provided path {conf_file} is not found.

For when the test binary isn't available, the issue is very obvious and looks like this (ran it locally that doesn't have that /opt/ directory available):

I'm hesitant to create yet another const value of the base path since the script is already so obfuscated, but can make the change if you feel strongly about it.

[anna-nexthop]

@AarjunC Applied my suggestion that should resolve your comment about the fragility of the qsfp_hw_test binary check in the latest commit.

[facebook-github-tools]

@anna-nexthop has updated the pull request. You must reimport the pull request before landing.

[meta-codesync]

@AarjunC merged this pull request in 2d931a5.