python/20210705.run-python-jupyter.md

@@ -1,101 +0,0 @@
-# Question: Running Python Code on Jupyter Notebook
-
-<!--
-    Published URL: https://forum.hpc.odu.edu/t/running-a-python-code-on-hpc-through-jupyter/85
-    Note: The version on this doc is slightly different from what published
-    there as of 2022-05-19.
--->
-
-I have Python code that I would like to run on HPC.
-Can I run it through the Jupyter notebook?
-If so, how can I?
-
-<!-- was taken from Wirawan's WORK SCRAP NOTES -->
-
-Yes, but we recommend that you launch the script through the HPC job scheduler,
-called “SLURM”.
-Let's call the script you want to run `SCRIPT.py`
-*(include the full path if the script is not the same directory
-as where you want to run this)*.
-Here is a basic SLURM job script that may work for you:
-
-```
-#!/bin/bash
-#SBATCH -p gpu
-#SBATCH --gres gpu:1
-enable_lmod
-module load container_env tensorflow-gpu/2.4.0
-crun.tensorflow-gpu python3 SCRIPT.py
-```
-
-**Important notes:**
-
-  * Don't worry why we recommend using the `tensorflow`
-    [container](https://wiki.hpc.odu.edu/Containers/Container-Intro).
-    This container has a installation of Python 3.7 that has fairly
-    complete set of basic libraries used by Python users, such as
-    `numpy`, `scipy`, `pandas`, `scikit-learn`, and of course,
-    TensorFlow.
-  * Replace `SCRIPT.py` with your actual script file name.
-  * We assume that your workload needs a GPU to run your calculation.
-    If you do not, please remove the `#SBATCH -p` and `#SBATCH --gres` lines
-    and replace the words `tensorflow-gpu` with `tensorflow-cpu` is everywhere.
-
-Create this script using a text editor, save it to a file (say, `JOB.sh`).
-Then you will submit the script
-from the UNIX shell interface of the cluster by typing:
-
-```
-$ sbatch JOB.sh
-```
-
-(The `$` at the beginning of the line represents
-the shell prompt--do not type that.)
-If successful, there will be a message printed
-`Submitted batch job NNNNNN`
-on the terminal,
-where `NNNNNN` is an integer called job ID or job number.
-The output file will be `slurm-NNNNNN.out`.
-
-Here are a few documentations to help understand this process:
-
-  * Documentation on Python on ODU HPC:
-    <https://wiki.hpc.odu.edu/Software/Python>
-
-  * General workflow of using SLURM job scheduler:
-    <https://wiki.hpc.odu.edu/slurm#general-workflow-of-using-slurm>
-
-If you prefer video introduction to SLURM, please take a look at this short video to understand what job scheduler is:
-
-**"Working with SLURM"**<br/>
-<https://odumedia.mediaspace.kaltura.com/playlist/dedicated/1_8eqsb16m/1_oy7ls1o0>
-
-(This is part of the
-[Intro to HPC](https://wiki.hpc.odu.edu/Training/HPC-Intro)
-training videos.)
-
-
------
-
-*Back to your original question*:
-Yes, you can run the script from a notebook.
-Once you opened a (blank) notebook, you will enter this statement in a new Python cell:
-
-```
-%load SCRIPT.py
-```
-
-Press `<Shift+Enter>` once, the code will be loaded into that same cell.
-Press `<Shift+Enter>` once more, the code will execute.
-But please note that this mode of execution can be awkward, i.e.
-we have to open a Jupyter session,
-`%load` the script in order to run it.
-The maximum time to run the script is under 24 hours (the max limit of Jupyter on HPC).
-This mode is useful if you plan to interactively test or troubleshoot a script,
-but less useful when you have a bunch of calculations to do,
-or calculation that you expect to run for a long time.
-
-*If you still have issue or question after reading this response:*
-Can you share the script with us so we may understand the requirements of your script?
-
-