Quick Start¶
Installation¶
Install subway in one of the following approaches
pip install hpcsubway
git clone https://github.com/Rails-on-HPC/subway.git . && python setup.py install
git clone https://github.com/Rails-on-HPC/subway.git . && export $SUBWAY_PATH=$(pwd)/bin
For the third approach, you don’t need to install things at all, which might be prefect for HPC setups.
Support platforms: subway is compatible with python 3.5 to 3.8 in linux and macos.
Workflow for a subway project¶
Just
subway init
in the empty project folder.Then you can copy your program binaries into this folder.
Modify configurations by
subway config edit
for this project. See config.json.Probably you may need to add some template files.
Write your own
main.py
asentry_point
of the project.Now
subway run
to get everything running. (You can also submitsubway run
to task managers like slurm)Analyse the results by powerful
subway query
command.
Writting entry point py¶
The entry point of the project, often named as main.py
, is the engine and monitor of the whole project,
which is also the underlying executable when subway run
.
If you initialize subway project by subway init, then a ready to go template of main.py
has already been in the project folder.
The file demo is as follows:
#! /usr/bin/python
import os
from subway import cons
cons.work_dir = os.path.dirname(os.path.abspath(__file__))
from subway.config import conf, history
from subway.workflow import main_once, main_rt
from subway.plugins import DebugSub, DebugChk
if __name__ == "__main__":
main_rt(DebugChk(), DebugSub())
The ingredients include:
Shebang in the first line, change it to your python path
The following several lines setup
work_dir
variable, you should keep them as they are.Import what you may need from subway, such as
conf
,history
, as well as main loop functionmain_rt
, together with checker and submitter classes you want to customize.Implement your own checker and submitter if needed.
Run
main_rt
with your checker and submitter as arguments.
Also note the file must have executable permission which can be set by chmod +x main.py
.