We test SLH PeoplePower against a well-known public benchmark used by university researchers around the world. On the test problems closest to our hospital's size, our engine matches the proven-best answer — sometimes in less than a second.
SLH PeoplePower contains a scheduling "engine" — the piece of software that takes a long list of rules (no doctor can be on call two nights in a row; nurses must have at least two days off between shifts; ward A needs a senior doctor every weekday) and figures out who should be where, when.
This is the kind of problem mathematicians and computer scientists have been studying for decades. There are well-known test problems with proven-best answers — like maths problems in a textbook with the answers in the back. Anyone building a scheduling system can run their engine against these problems and see how it does.
That's what we did.
We used schedulingbenchmarks.org — a public academic resource that publishes test problems for nurse and shift rostering. It's maintained by researchers and used as the yardstick in dozens of academic papers each year.
Each test problem describes a fictional hospital — how many staff it has, how many weeks of schedule it needs, how many different shift types exist, and the rules each doctor or nurse must follow. The benchmark also publishes the proven-best solution to each problem, computed by researchers running their algorithms for hours (sometimes days) on dedicated hardware.
We took our engine, gave it just 60 seconds per problem on an ordinary MacBook, and asked it to produce a schedule. Then we compared.
Here are the four test problems closest to SLH's actual scale — they involve 8 to 20 staff over 2 to 4 weeks, which matches the size of rosters we run every month.
| Test problem | Setup | How well we did | Time taken |
|---|---|---|---|
| Instance 1 | 8 staff, 2 weeks, 1 shift type | Matched exactly Found the proven-best answer and confirmed no better answer exists. |
0.1 sec |
| Instance 2 | 14 staff, 2 weeks, 2 shift types | Matched exactly Found the proven-best answer (kept searching until the time ran out). |
60 sec |
| Instance 3 | 20 staff, 2 weeks, 3 shift types | Within 0.2% Essentially tied — just 2 points worse on a soft-rule score out of 1,001. |
60 sec |
| Instance 4 | 10 staff, 4 weeks, 2 shift types | Within 0.4% 7 points worse than the textbook answer of 1,716. |
60 sec |
For context: SLH's actual call-roster pool is 8 working junior doctors over a month — almost identical to Instance 1 in scale. That's the test problem we matched exactly, in less than a second.
The benchmark also includes some very large test problems — fictional hospitals with 60, 100, or even 150 staff and dozens of shift types. We ran our engine on those too, but with mixed results.
| Test problem | Setup | How well we did |
|---|---|---|
| Instances 5–11 | 16–50 staff, 4 weeks | Within 3% – 10% of textbook |
| Instances 12–13 | 60–120 staff, 4 weeks | 50% – 300% worse — too big to solve in a minute |
The very largest problems (60+ staff with 10–18 different shift types) are larger than any hospital SLH operates. They're designed to stress-test research-grade algorithms, not to reflect real community-hospital scale. The published "best" answers on those took academic teams hours or days to find using bespoke methods.
On the scale we actually run, our engine is fully competitive.
The engine is correct. An independent rule-checker — written separately from the engine — read each of our solutions and confirmed every rule was respected. Zero hard violations. The fairness score the engine reported was independently recomputed and matched exactly.
On the scale of SLH's actual roster, we're at the top of the field. On the smallest test problem (closest to our 8-junior-doctor call pool), our engine matched the proven-best solution that academic teams have published — and it did so in a fraction of a second. On the next three problems up in scale, we came within a fraction of a percent. These are the comparisons that matter, because they reflect the kind of work the system does at SLH every month.
The results are reproducible. The benchmark is public. The test instances are downloadable. Our code is on the SLH PeoplePower GitHub. Anyone with a laptop can re-run the comparison and see the same numbers.
We want to be straightforward about what this benchmark doesn't show.
If you'd like to dig deeper:
Questions about anything on this page? Email kevin@awyong.sg.