Describe the bug
This may be either a bug or a feature request.
In this case it is not uncommon for clusters to have network access restricted on actual compute nodes but allow download access on specific restricted nodes.
After creating the sif image and attempting to run on a firewalled machine, the following error occurs:
./pgap.py --no-internet -r -o mg37_results ./test_genomes/MG37/input.yaml
--no-internet flag enabled, not checking remote versions.
<urlopen error [Errno 111] Connection refused>
Failed to update ./pgap.py, ignoring
Something has gone wrong, please manually download: https://github.com/ncbi/pgap/raw/prod/scripts/pgap.py
When running a node that does have internet access it works as expected:
./pgap.py --no-internet -r -o mg37_results ./test_genomes/MG37/input.yaml
--no-internet flag enabled, not checking remote versions.
Output will be placed in: /dir/pgap/mg37_results.1
INFO: Using cached SIF image
Not after a successful run, there is ~29GB of data in a folder input-2022-08-11.build6275, however it still doesn't run when you attempt to rerun.
If the above is expected, could you consider an offline mode to running the pipeline after the data has been downloaded once. Better yet, split the data download from the rest of the pipeline and allow that to be run separately?
To Reproduce
See above
Expected behavior
Once the sif image has been created, no more internet access is required. Using the --no-internet option to me implies that it is possible to run without a network connection once the image is created elsewhere.
Software versions (please complete the following information):
- OS: SuSE 15SP3
- pgap.py --version: PGAP version 2022-08-11.build6275 is up to date.
- Docker (or other container runner) version: apptainer version 1.0.1
Log Files
error happens before debug output starts
Additional context
Describe the bug
This may be either a bug or a feature request.
In this case it is not uncommon for clusters to have network access restricted on actual compute nodes but allow download access on specific restricted nodes.
After creating the sif image and attempting to run on a firewalled machine, the following error occurs:
When running a node that does have internet access it works as expected:
Not after a successful run, there is ~29GB of data in a folder input-2022-08-11.build6275, however it still doesn't run when you attempt to rerun.
If the above is expected, could you consider an offline mode to running the pipeline after the data has been downloaded once. Better yet, split the data download from the rest of the pipeline and allow that to be run separately?
To Reproduce
See above
Expected behavior
Once the sif image has been created, no more internet access is required. Using the --no-internet option to me implies that it is possible to run without a network connection once the image is created elsewhere.
Software versions (please complete the following information):
Log Files
error happens before debug output starts
Additional context