From 1d9a91f698d598e04a068358a2c22f295f11083b Mon Sep 17 00:00:00 2001 From: Wirawan Purwanto Date: Sat, 9 Dec 2023 14:59:41 -0500 Subject: [PATCH] * Added a notes concerning software options for job scheduler on Windows. Gathered from CaRCC discussion. --- windows/20231208.Windows-job-scheduler.md | 53 +++++++++++++++++++++++ 1 file changed, 53 insertions(+) create mode 100644 windows/20231208.Windows-job-scheduler.md diff --git a/windows/20231208.Windows-job-scheduler.md b/windows/20231208.Windows-job-scheduler.md new file mode 100644 index 0000000..e310272 --- /dev/null +++ b/windows/20231208.Windows-job-scheduler.md @@ -0,0 +1,53 @@ +Job Scheduler for Windows +========================= + + +Original question +----------------- + +From: Deb McCaffrey
+Sent: Monday, December 4, 2023 3:25 PM
+To: people-network@carcc.org
+Subject: [CaRCC people] Scheduling on Windows? + +Does anyone know of a way to do job scheduling on Windows? We have some researchers dealing with resource contention on their Windows server. They're just manually communicating their jobs right now. + + +## Several options: + +### 1: HTCondor + +"HTCondor offers Windows functionality. You can find more information here: https://htcondor.readthedocs.io/en/latest/platform-specific/microsoft-windows.html" (Carrie Brown) + +"If you've got any questions about HTCondor on Windows, feel free to reach out to htcondor-users@cs.wisc.edu. We'd love to hear feedback!" (Brian Lin, OSG) + + +### 2: HyperShell + +"Actually we have something that may work. +At Purdue we've developed a workflow automation tool for high-throughput computing called HyperShell (https://hyper-shell.readthedocs.io, https://github.com/glentner/hyper-shell). +It's been in use for a few years now, is cross-platform, and has a few different modes of use. +To use it as a persistent scheduler, it can have a local SQLite database file on the Windows server, or a remote Postgres instance, and it will schedule tasks on all connect executors. +It is not a resource manager though, however many concurrent tasks you say can run - will, so the workload itself needs to be told not to use more than you intend. +I've used this to orchestrate large numbers of tasks over dozens of Windows Server instances." +(Geoffrey R Lentner ) + + + +### Other places to look at: + +A handy list of job scheduling software, and some such as Apache Airflow, are open source and run on Windows. + +https://en.wikipedia.org/wiki/List_of_job_scheduler_software + +It remains to be seen whether any other them have the features your researchers need. (David Paul Reddy) + + +Considerations for Windows jobs +------------------------------- + +"...it's good to get a sense of what sort of jobs researchers running on this Windows server. + + * Do they involve a GUI interface and interactive sessions? + * Are they pleasantly parallel tasks with low core requirements or bigger multi-core OpenMP or MPI ones? + * How long do the jobs normally last and how many is each individual trying to run?" (Matthew T. West, Research IT, University of Exeter)