Free GPU Cluster Capacity Planning Template for Infra PMs (Excel & Notion)

The template is only useful if it forces a capacity decision, not a prettier forecast.

In one weekly review, the infra PM had a clean Notion page, a polished Excel tab, and three engineers still arguing about why the training queue kept backing up at the end of the day. The sheet was not the issue. The issue was that nobody had written down what the cluster was supposed to protect: training throughput, inference latency, or research flexibility. That is the real job of a capacity planning template. Not reporting, but forcing a tradeoff. Not an inventory tracker, but an operating contract. Not a dashboard for comfort, but a document that makes someone own the next shortage.

What should a GPU capacity template actually tell me?

A good GPU template tells you where the constraint is before the outage proves it for you.

The first counter-intuitive truth is that average utilization is the least interesting number in the file. I have sat in debriefs where a team proudly pointed to “healthy” average usage while the actual pain was a 9 p.m. queue spike that stalled training jobs until morning. The template that matters shows reserved capacity, burst capacity, queue policy, maintenance buffer, and the lead time for adding hardware. If those fields are not visible on one page, the template is decorative. Not a forecast, but a decision log. Not a utilization report, but a failure map.

The second counter-intuitive truth is that the template should make underuse visible, not just overuse. Infra leaders often hide behind the language of “efficiency,” but a cluster that is always packed is usually one maintenance event away from pain. In a planning review with a hiring manager who owned shared compute, the debate was not whether to squeeze harder; it was whether the cluster had any slack for a model retrain that missed its slot. The best template exposes the slack explicitly: which GPUs are pinned to production, which are preemptible, which are reserved for experiments, and which are actually fungible across teams. If you cannot answer that in one glance, you do not have a plan.

The third counter-intuitive truth is that ownership matters more than precision. I have seen teams argue for an extra decimal place in a demand model while nobody could say who would approve a new reservation policy. That is how planning turns into theater. A useful template names the owner for each assumption: incoming demand, job priority, storage throughput, network headroom, and procurement timeline. The point is not mathematical purity. The point is to make every assumption attach to a human who can defend it when the queue backs up.

Why does Excel still beat Notion for some teams?

Excel beats Notion whenever the template has to calculate, reconcile, or survive argument.

Notion is better when the artifact is a shared operating memo. Excel is better when the artifact must survive a tense review with finance, infra, and the hiring manager asking for a number that can be challenged. I have watched teams move a model into Notion because it looked cleaner, then quietly rebuild the formulas in Excel after the first budget meeting. That is not a tool preference. That is organizational psychology. People use the medium that can absorb conflict. Not X, but Y: not the prettiest interface, but the most defensible one.

The template should usually live in both places for different reasons. Excel is the calculation layer: demand by team, reserved versus shared pool, expected arrivals, hardware lead times, and what happens if a delivery slips by two weeks. Notion is the narrative layer: assumptions, owners, policy, and decisions from the last review. If you collapse both into one place, you either lose rigor or lose readability. In one debrief, the PM had all the numbers in Notion and all the explanations in a slide deck. Nobody trusted either. The spreadsheet was where the truth belonged. The page was where the argument belonged.

Use this line when the room starts drifting into aesthetics: “I do not care where the template lives. I care whether it lets us trace a shortage back to the exact assumption that created it.” That sentence cuts through the usual tool debate because it reframes the issue from interface to accountability. A team that cannot explain its assumptions in writing will not suddenly become honest because the page has a nicer sidebar. The template is not trying to impress leadership. It is trying to survive a bad quarter.

Which assumptions break GPU plans in practice?

The assumptions that break GPU plans are rarely about the GPUs.

The hidden failure is usually storage, network, or queue behavior. I watched one planning discussion where everyone obsessed over a new batch of accelerators, but the real issue was that data staging could not keep up with model starts. The queue looked full, but the bottleneck was a transfer path no one had put in the template. That is why the best capacity sheet includes not just GPU count, but data ingress, model artifact size, checkpoint frequency, and the policy for jobs that miss their launch window. Not hardware only, but the path around the hardware. Not peak demand only, but the longest blocking step.

The second hidden failure is fragmentation. A cluster with enough total capacity can still be unusable if the free GPUs are split across the wrong memory sizes, topology, or reservation boundaries. In one hiring committee-style discussion with an infra lead, the debate was whether the team needed more machines or better bin-packing discipline. The answer was neither. The cluster needed a template that showed what type of workload could run on what type of GPU, and which workloads could not be co-located without causing churn. If the sheet treats every GPU as interchangeable, it is lying. A capacity plan that ignores topology is not a plan. It is wishful accounting.

The third hidden failure is policy drift. Teams write down one reservation rule, then override it every time a senior researcher complains. After a month, the template still looks correct and the cluster is still broken. That is the part executives miss. The problem is not the worksheet. The problem is the unofficial policy. I have heard a director say, “We do not have a capacity issue; we have a compliance issue disguised as capacity.” That was accurate. If the template does not record exception handling, it will lie by omission. Every exception should be visible enough that the next person can see what rule was broken and why.

A useful script for the meeting is: “Before we add more hardware, show me the constraint that will disappear when the hardware lands.” If nobody can answer that cleanly, the request is not a capacity plan. It is a hope.

How do I use the template in a stakeholder review?

Use the template to force a choice, not to invite another round of vague agreement.

In a real review, the dangerous moment comes when everyone nods at the same worksheet and leaves with different interpretations. The infra PM’s job is to collapse those interpretations into one decision. I have seen a hiring manager push back on a plan because it was accurate and still useless. The numbers were right, but the question of priority was missing. Should the cluster favor product inference, research training, or internal experimentation? If the template cannot support that answer, it is not review-ready. Not a data artifact, but a governance artifact. Not “here is the situation,” but “here is the rule.”

Use direct language in the room. “This template says we can serve training and inference on the same pool, but it does not say who gets preempted when both compete.” “This plan assumes procurement lands before the next model cycle; what is the fallback if it slips?” “If we approve this reservation, which team loses flexibility?” Those are not soft questions. They are the actual job. The strongest infra PMs do not present a spreadsheet and wait for consensus. They present a tradeoff and ask leadership to own it.

There is also a script for resisting cosmetic revisions: “I am willing to rename fields, but I am not willing to hide the constraints.” That line matters because organizations often try to solve uncomfortable tradeoffs with prettier language. A good capacity template refuses that move. It keeps the friction visible. In the debrief room, that visibility is what separates a serious operating team from a team that is merely busy.

If you want one practical test, use this one: can the template explain, in under one minute, what happens when demand increases, supply slips, or policy changes? If not, it is not a planning template. It is a record of hope.

When is a template the wrong answer?

A template is the wrong answer when the team lacks ownership, not data.

If no one can approve reservations, enforce queue policy, or sign off on exceptions, then the template becomes ceremonial. I have seen teams build immaculate planning files while the real decisions kept happening in hallway conversations. That is the classic failure mode. Not a tooling problem, but a power problem. Not a missing spreadsheet, but missing authority. A template can surface the issue, but it cannot replace governance.

The second case where the template fails is when the org is pretending that every workload is the same. Training, inference, evaluation, and experimentation do not behave like one blended pool unless leadership is willing to accept cross-contamination in priority. A template that flattens those differences will look elegant and perform badly. In one review, the PM tried to unify all workloads into a single forecast. The hiring manager stopped the meeting and said, “You just turned three operating models into one fantasy model.” That was the right call. The template should separate classes of demand, not average them away.

The third case is when leadership wants a comfort object. Some leaders ask for a sheet because they want to feel that the problem is being managed. A serious PM does not feed that instinct. The right response is, “This template will not solve the shortage by itself. It will show whether the shortage is structural, temporary, or political.” That is the correct level of honesty. If the organization is not prepared to act on the answer, producing the template is wasted effort.

Preparation Checklist

• Build the Excel model first if the template needs formulas, scenario toggles, or reconciliation across teams.
• Use Notion as the decision log: assumptions, owners, last review date, and policy changes belong there.
• Separate training, inference, and evaluation into different demand lines. Do not merge them for convenience.
• Record reserved, shared, and preemptible capacity as distinct buckets. If they are mixed, the plan is already broken.
• Add non-GPU constraints to the sheet: storage ingress, network headroom, and lead time for hardware arrival.
• Prepare one sentence that states the tradeoff in plain language. Example: “This plan prioritizes production inference over experimental jobs during peak demand.”
• Work through a structured preparation system (the PM Interview Playbook covers tradeoff framing, capacity assumptions, and debrief examples that map well to infra PM reviews).

Mistakes to Avoid

• BAD: “We have enough GPUs on paper, so we are covered.” GOOD: “We have enough total GPUs, but the free pool is fragmented across the wrong workload types.”
• BAD: “Notion is the source of truth, and Excel is just a backup.” GOOD: “Excel computes the numbers; Notion records the decisions and owners.”
• BAD: “We will revisit policy after the next purchase.” GOOD: “We will document the current exception path now, because policy drift is what breaks the plan before procurement does.”

FAQ

1. Do I need both Excel and Notion? Yes. Excel is the planning engine; Notion is the decision record. If you force both jobs into one tool, you usually lose either the math or the accountability. The template needs both to survive a real review.

2. Can this template work for inference clusters too? Yes, but only if you separate inference latency requirements from training throughput. If you treat them as the same workload, the template will hide the real tradeoff and leadership will make the wrong call.

3. What is the biggest sign the template is too weak? It cannot answer a shortage question quickly. If the sheet cannot tell you what gets delayed, who gets priority, and what assumption failed, the template is cosmetic, not operational.amazon.com/dp/B0GWWJQ2S3).