|
Condor pool consist of computational resources of the PC in Computer Centre's PC laboratory of around 81 Intel Pentium CPUs.
1. User Account Requirement
User must have HPCPOWER account before submitting programs to condor pool in PC laboratory. User requires HPCPOWER account for submitting and manipulating job at the Condor pool.
2. Program General Requirement
Each condor job should be limited to around 1.5 hours. Surveies on the usage in the PC laboratory were conducted and a normal session in the PC laboratory is found to be at around 1.5 hr. If the user submit some condor jobs that take longer than that, the job will likely to be terminated when the user switches off the PC power. The job will be automatically resubmitted to other free PCs and start from the beginning, but it will be a waste of resources.
The program must be self-containing, i.e. source code for all subroutines must be already included (no IMSL calls)
3. Program Input Requirement
The ideal case will be each condor job taking regular pattern set of input. e.g.:
Massive Monte-carlo type simulation, where each job will run on 1 or 1 set of random no. seed. A program (or a shell script) will be necessary to execute on hpcpower that generates thousands of job command strings like:
my_program 1 1 1.2345
my_program 1 2 2.345
...
If the parameters bear some regular pattern, Condor contains facilities to automatically increment the counter (but it always has to start with zero)
4. Program Output Requirement
The output filename for different condor job must be distinct, and the name of the output files must bear some relationships with the parameters fed. If it cannot be done, then you have to create an output directory (distincted name, bearing some relations with the parameters fed).
5. Program Output Collection
It is necessary to write an data collection program in the hpcpower. This program will open the output files one-by-one, extract the individual results and manipulate them to get the final result. In some sense it behaves like the "mpi-reduce" subroutine, but the mpi routines are real-time and the data are via inter-processor communication, vs this data collection program will perform manipulating on data in disk files.
6. Compiling program for Condor Pool
It is necessary to make the program become standalone executable file on Windows XP/Vista platform (32-bit) before submitting it to condor pool. The followings are the basic steps for compiling Fortran prograam to be executable on Windows platform:
i. Gather all the necessary fortran program files (including header and library files)
ii. Open the 'Command Prompt'. Type "ifortvars" to set the environmental variables for compilation.
>ifortvars
iii. Check the environmental variable if not sure.
>echo %f90%
>echo %f90flags%
iv. After checking that environmental variables %f90% and %f90flags% are not empty, compile the program by this command:
>%f90% %f90flags% my_program.f -o my_program.exe
In case successful then it will generate the executable file 'my_program.exe'. You may trial run it and debug it on WINHPC master node if necessary.
It is highly recommended to SCP all the necessary program, DLL and input files to a PC in Run Run Shaw PC laboratory to test if the program is executable in PC laboratory.
7. Submitting and Manipulating job at Condor Pool
After making your program become standalone executable file on Windows platform, upload all necessary program, DLL and input files to hpcpower.
In case the program requires multiple DLL files it is recommended to use 7-zip to compress all the DLL files into a single zip file. Afterward in the condor batch script use 7-zip to decompress the DLL files at the execution host.
i. Prepare condor control file: It is necessary to give several runtime parameters for condor about your program. A sample condor control file is located at /share/condor/hello.con. Remind to change several parameters such as 'Executable','transfer_input_files' and 'Queue'.
ii. Submitting condor control file: Type 'condor_submit my_control_file'
iii. Checking job status: Type 'condor_q'
iv. Deleting job: Type 'condor_rm MY_JOB_ID'.
For further enquries please email to the HPC team (hpc@cc.hku.hk).
Condor Online Manuals
|
