Wirawan Purwanto
dfb9db6a60
* Added tool to get a detailed picture of GPU utilization on a SLURM cluster.
...
Tested on Wahab cluster.
2 years ago
Wirawan Purwanto
3aa1688f8e
* Added basic README.
2 years ago
Wirawan Purwanto
4af174ea34
* Containers: Added simplistic tool to dump info about Python
...
inside the container.
2 years ago
Wirawan Purwanto
ce695cd672
* Created initial tool to scan user dirs for the size of their trash folders.
2 years ago
Wirawan Purwanto
c4601a5a30
* (WIP) Minor improvement + documentation.
2 years ago
Wirawan Purwanto
5edd528511
* Saved a sample revised ipython module loader, to be used with
...
"fix3" version of lmod_python_fix.py.
Sigs:
# -rw------- 1 wpurwant users 291 2020-06-09 20:09 000-odurc-lmod.py.rev3
# 70e84e7eaaf28da297fa342c8318fe2b 000-odurc-lmod.py.rev3
3 years ago
Wirawan Purwanto
4a1b5d0a69
* Saved developmental notebooks to devise/test the "lmod_python_fix.py".
...
These were done on Wahab with legacy (lmod-based) Python suite.
Sigs:
# -rw-r--r-- 1 wpurwant users 56588 2020-06-09 17:35 debug_ipython_start_1.ipynb
# -rw-r--r-- 1 wpurwant users 20395 2020-06-09 17:34 debug_ipython_start_2.ipynb
# -rw-r--r-- 1 wpurwant users 21965 2020-06-09 19:20 debug_ipython_start_3.ipynb
# -rw-r--r-- 1 wpurwant users 10717 2020-06-09 19:23 debug_ipython_start_4_vanilla.ipynb
# -rw-r--r-- 1 wpurwant users 37041 2020-06-09 19:59 debug_ipython_start_5.ipynb
# b061fbc819806bc18d48287a9654bcc5 debug_ipython_start_1.ipynb
# bc262fc0d3cd290da1f46a242a6fcccb debug_ipython_start_2.ipynb
# 5f289c909e967096712b3196a19979b7 debug_ipython_start_3.ipynb
# b5f0cd9251e325b0c7e0222ba4cfc037 debug_ipython_start_4_vanilla.ipynb
# abc440a885da0872d5bc215adfc3edd7 debug_ipython_start_5.ipynb
3 years ago
Wirawan Purwanto
0ad73f50d6
* Saved README.txt of the lmod_python fix (notes written to Minhao, 2020-06-09).
3 years ago
Wirawan Purwanto
d159d57578
* Third attempt to fix lmod "module" for ipython ("fix3")
...
(dated: 2020-06-09).
* Supports both Turing and Wahab.
* Allows both addition and deletion of paths from sys.path.
* Author's note: Later on, I discovered that these complex steps do not appear
to be necessary, somehow the lmod Python commands were able to take care
of additions and deletions through the modification of os.environ['PYTHONPATH']
alone, so I am not sure if this fix is absolutely needed. (?)
Sigs:
# -rw------- 1 wpurwant users 3298 2020-06-09 20:07 lmod_python_fix3.py
# 78abc7bae90bf23a42aa40285175ca48 lmod_python_fix3.py
3 years ago
Wirawan Purwanto
232cc5cafb
* Second attempt to fix lmod "module" for ipython ("fix2")
...
(dated: 2020-06-09).
WIP: Prevent indefinite lengthening of sys.path but not complete
fix yet.
Sigs:
# -rw------- 1 wpurwant users 1290 2020-06-09 19:28 lmod_python_fix2.py
# e218a4ac73762e0380c4bbe2290906b1 lmod_python_fix2.py
3 years ago
Wirawan Purwanto
5c0b516026
* First attempt to fix lmod "module" for ipython ("fix1")
...
(dated: 2020-06-09)
Sigs:
# -rw------- 1 wpurwant users 915 2020-06-09 17:20 lmod_python_fix1.py
# bb42b2bde01f2e235eb205fe555fab9e lmod_python_fix1.py
3 years ago
Wirawan Purwanto
7c02045fa1
* Saved the original lmod_python.py and
...
~/.ipython/profile_default/startup/000-odurc-lmod.py
from Wahab cluster on the "legacy" lmod-based ipython/jupyter
session. Last update: 2020-06-09.
Sigs:
# -rw-r--r-- 1 wpurwant users 179 2020-06-03 15:49 000-odurc-lmod.py
# -rw-r--r-- 1 wpurwant users 843 2020-03-17 14:02 lmod_python.py
# 8718a2d7a7d6d152701e6c31254f7168 000-odurc-lmod.py
# 24a740e83656875bd39f942529378414 lmod_python.py
3 years ago
Wirawan Purwanto
9edb1e040d
* Imported tools to dump information about files in a container.
...
The "dump-info" tool works only for Debian-based distro for now.
3 years ago
Wirawan Purwanto
e828a16a49
* Expanded support to Wahab.
...
Last updated 2021-03-02.
3 years ago
Wirawan Purwanto
cc95430e06
* Added lmod module for OPENIB-based nwchem installation on Wahab.
4 years ago
Wirawan Purwanto
f9d350aa7c
* Added make-based install procedure for nwchem 7.0.0.
4 years ago
Wirawan Purwanto
9e7c5b3312
* Modularize & tidied up the nwbuild05 script.
4 years ago
Wirawan Purwanto
b3f1877f73
* Initial version of nwchem "build05": Using "OPENIB" backend.
4 years ago
Wirawan Purwanto
67c05e7efd
* Added small build notes for nwchem on Wahab.
4 years ago
Wirawan Purwanto
08dfff3ce4
* Modifications for building armci-mpi on the Wahab's install location.
4 years ago
Wirawan Purwanto
8cf081b4b1
* Copied armci-mpi build script from ODU container repo.
...
This is taken from commit #dd7cf94ecdb1c1368fed2458c4f45bd95e1a140c
from ODU container repo. No changes were made at all.
4 years ago
Wirawan Purwanto
b97592a8c7
* Copied "build04" nwchem build script from ODU container repo.
...
This is taken from commit #a278c9cb246fc267f93018e607856c1141942d32
from ODU container repo. No changes were made at all.
4 years ago
Wirawan Purwanto
d68c1cc608
* Wahab: Imported comsol-51, last modified 2021-02-11 to accommodate
...
launching from another SLURM job (e.g. OOD virtual desktop interface).
4 years ago
Wirawan Purwanto
0331ea6ed3
* Committed: Original user-invocable "qe-<VERSION>" from Wahab.
...
Signatures:
# -rwxr-xr-x 1 wpurwant users 520 2020-03-10 14:39:05 qe-6.3
# -rwxr-xr-x 1 wpurwant users 514 2020-03-10 14:29:54 qe-6.4
# -rwxr-xr-x 1 wpurwant users 531 2020-03-10 14:28:29 qe-6.4-intel
# bda38c3efc9bb0f83db351924e06da43 qe-6.3
# 2b2e9ec9c641dd60206776a5a80c75a3 qe-6.4
# 083455403fa47897c4d6e7ce6d591f9a qe-6.4-intel
# 8d29ad9caa756b55747e58a06c39aa59576a896b qe-6.3
# 1443cdb47e4340be1106d4e80ba3dc47717d9ca1 qe-6.4
# 1bf39ffc828cdc784658ea510fbda0602d34e237 qe-6.4-intel
4 years ago
Wirawan Purwanto
e9597af157
* Minor typographical improvements.
4 years ago
Wirawan Purwanto
dda0e1b7ad
* ARCHIVE: Imported some tools from my stats work back in 2016.
...
Warning: These tools are obsolete, and may or may not work.
Orig-cwd: TURING:/home/wpurwant/hpc-explore/turing-hw/tests20160826
Sigs:
# -rwx------ 1 wpurwant users 214 2016-09-13 11:58:21 show-disabled-nodes2.awk
# -rwx------ 1 wpurwant users 111 2016-08-26 13:10:14 show-disabled-nodes.sh
# -rwx------ 1 wpurwant users 122 2016-08-26 10:25:16 show-node-summary.sh
# -rwx------ 1 wpurwant users 153 2016-08-26 12:06:25 show-non-full-nodes.sh
# 71c64a7e6cd9df34ca6a9348ebc44ab1 show-disabled-nodes.sh
# 8a3570829d132c526cc44adf72212887 show-disabled-nodes2.awk
# ebbc0642cd29a1575670c86629d89c72 show-node-summary.sh
# 5eee777a8380333654d3a06e3482f38e show-non-full-nodes.sh
# dfb36573c6d758bc675f70247ff0befe9ddbe365 show-disabled-nodes.sh
# d1995fc91d402c9dcc30adc0b461e2beaa25ab67 show-disabled-nodes2.awk
# 3267d169b476396d66f6127e1343bd7c23b37e1f show-node-summary.sh
# 18673a78914d9cce47c95d4fd5146805fc227416 show-non-full-nodes.sh
4 years ago
Wirawan Purwanto
9429ad3697
* Added capability to parse & create scratch dirs.
4 years ago
Wirawan Purwanto
7dc03821a8
* g09slurm initial update: correctly parse nprocshared / nproc in link0
...
command in a case-insensitive manner.
4 years ago
Min Dong
19eacfbf54
* Committed: Original user-invocable "g09slurm" from Wahab
...
and the "g09.slurm" backend script.
Signatures:
# -rwxr-xr-x 1 wpurwant users 394 2020-09-10 01:06:20 g09.slurm
# -rwxr-xr-x 1 wpurwant users 1120 2019-09-16 15:24:20 g09slurm
# 387f828b04454678b23b05af61e4f334 g09.slurm
# 9ee4bfcfad766165a81239cb13d7ef30 g09slurm
# e0dbc8b24aeecbbecf7255a3e3f84dc3fc3a422f g09.slurm
# d5a33e554766c41f17f6839242c5c7f4d60e7fe4 g09slurm
g09.slurm is originally located at
/cm/shared/apps/gaussian/g09revD.01/script/g09/script/g09.slurm
4 years ago
Wirawan Purwanto
19c833c3ff
* sinfo-report-node-stats.sh: Simple tool to report status of compute
...
nodes based on SLURM's "sinfo" output.
4 years ago
Wirawan Purwanto
18d79dd34b
* On custom sacct1.sh, include job (most recent) state as well.
4 years ago
Wirawan Purwanto
d384d0320d
* Added explicitly headless option (--headless).
...
* Increased wait time to 5 minutes.
5 years ago
Wirawan Purwanto
c2a5ae8863
* slurm: Modified fields to print by default.
5 years ago
Wirawan Purwanto
32b82db7a3
* slurm: Added custom sacct wrapper script which contains my preferences.
5 years ago
Wirawan Purwanto
db2ca075ed
* Must re-add SLURM if it is not loaded.
6 years ago
Wirawan Purwanto
a65338a8bf
* Added "wgo" (what's going on) to check the status of processes
...
on login node. Currently supposed to be used for Turing only.
6 years ago
Wirawan Purwanto
1387997010
* Prints an error message and quit in case Jupyter
...
did not start after 2 minutes.
6 years ago
Wirawan Purwanto
ba6c9f53ed
* Added accommodation for Anaconda as well as site-provided python.
...
* Added support for fully headless mode (connect via local browser).
* Added some safeguards against job failing to start due to executable
not found, etc.
6 years ago
Wirawan Purwanto
1c8a5da492
* Imported launch_jupyter from Turing.
...
Originally furnished by Min Dong, 2019-03-01 14:30 EST.
6 years ago
Wirawan Purwanto
28eb7a0d98
* sq: Customizable squeue wrapper: introduce new defaults / default
...
behavior on squeue.
6 years ago
Wirawan Purwanto
94e0aa9490
* jupyter-anaconda2: A script that will start Jupyter notebook process
...
for Anaconda2 distribution.
Note: for now it is a script that has to be submitted in the compute
node. I will upgrade this to become a self-submitting script eventually.
6 years ago
Wirawan Purwanto
95034685ff
* interact: Tool to allocate an interactive session on a regular
...
compute node under SLURM.
6 years ago
Wirawan Purwanto
dbce662c5a
* interact-gpu: Tool to allocate an interactive session on a GPU
...
compute node under SLURM.
6 years ago
Wirawan Purwanto
82ea3bc689
* Archived: SGE version of pwscf-5.3 script.
7 years ago
Wirawan Purwanto
9c82c4d465
* Update bash module support: with recent changes on Turing,
...
'module' seems to be supported out of the box for bash.
If 'module' environment is detected, we skip the initiation step.
7 years ago
Wirawan Purwanto
68c0e70d4d
* Added "regular" runsas which runs with more limited memory.
8 years ago
Wirawan Purwanto
27b8ccd6ae
* Added runsas-himem from earlier consultation this year.
8 years ago
Wirawan Purwanto
df6facce86
* pwscf: Ad-hoc fix for Turing after 2016 upgrade.
...
We force using the old (TCL) module system since the new module
system (LMOD) always executes itself whenever a bash batch script
is executed on Turing right now.
8 years ago
Wirawan Purwanto
739d765f53
* Added convenience for gathering & analyzing CPUs on the cluster.
...
* Documentation update.
8 years ago
Wirawan Purwanto
aa597b907c
* In hoststats subcommand: Also print node status flags if they exist.
8 years ago