* Added a notes concerning software options for job scheduler on Windows.

Gathered from CaRCC discussion.
Wirawan Purwanto 7 months ago
parent fa6d73a842
commit 1d9a91f698
  1. 53

@ -0,0 +1,53 @@
Job Scheduler for Windows
Original question
From: Deb McCaffrey <dmccaff4@asu.edu><br/>
Sent: Monday, December 4, 2023 3:25 PM<br/>
To: people-network@carcc.org<br/>
Subject: [CaRCC people] Scheduling on Windows?
Does anyone know of a way to do job scheduling on Windows? We have some researchers dealing with resource contention on their Windows server. They're just manually communicating their jobs right now.
## Several options:
### 1: HTCondor
"HTCondor offers Windows functionality. You can find more information here: https://htcondor.readthedocs.io/en/latest/platform-specific/microsoft-windows.html" (Carrie Brown)
"If you've got any questions about HTCondor on Windows, feel free to reach out to htcondor-users@cs.wisc.edu. We'd love to hear feedback!" (Brian Lin, OSG)
### 2: HyperShell
"Actually we have something that may work.
At Purdue we've developed a workflow automation tool for high-throughput computing called HyperShell (https://hyper-shell.readthedocs.io, https://github.com/glentner/hyper-shell).
It's been in use for a few years now, is cross-platform, and has a few different modes of use.
To use it as a persistent scheduler, it can have a local SQLite database file on the Windows server, or a remote Postgres instance, and it will schedule tasks on all connect executors.
It is not a resource manager though, however many concurrent tasks you say can run - will, so the workload itself needs to be told not to use more than you intend.
I've used this to orchestrate large numbers of tasks over dozens of Windows Server instances."
(Geoffrey R Lentner <glentner@purdue.edu>)
### Other places to look at:
A handy list of job scheduling software, and some such as Apache Airflow, are open source and run on Windows.
It remains to be seen whether any other them have the features your researchers need. (David Paul Reddy)
Considerations for Windows jobs
"...it's good to get a sense of what sort of jobs researchers running on this Windows server.
* Do they involve a GUI interface and interactive sessions?
* Are they pleasantly parallel tasks with low core requirements or bigger multi-core OpenMP or MPI ones?
* How long do the jobs normally last and how many is each individual trying to run?" (Matthew T. West, Research IT, University of Exeter)