How we caught a broken modpack before any customer paid for it

Last night at 03:41 UTC our test runner gave up on Lucky World Invasion. It had spent 25 minutes spinning on a fresh Hetzner box, stuck in the queued phase. We pulled the plug, flipped the pack to broken, and by 03:45 nobody could order it from the signup form.

That’s the whole point of running these tests.

Why “verified” tags lie

Most hosts ship a list of one-click modpacks and trust whatever metadata the author shipped. Pack says it runs on a server, so it gets a green tick. The tick stays green forever.

Modpacks aren’t frozen. The author can ship a client-only mod by mistake. A dependency can drop server support. CurseForge can change distribution rules and break the download URL. A pack that ran fine last month can quietly stop working.

The customer is usually the one who finds out. Usually after they’ve paid for the month.

What the sweep does

Every night between 03:00 and 05:00 UTC we grab the three packs that haven’t been re-checked in a week. For each one we boot a clean Hetzner VM, run the exact install path you’d get as a customer, and watch the server come up.

If it boots, the pack stays green. If install times out or the port never opens, we flag it broken. Test VMs get torn down either way so we’re not paying for empty boxes.

The signup form reads that table before it takes anyone’s card. Broken packs literally can’t be ordered.

What happened to Lucky World Invasion

The pack had been sitting in our database tagged “verified” since 16 May. Static metadata, nobody had touched it. Last night the install just never finished. SSH timed out, the server never bound a port.

Rough timeline:

• 03:14 Isle of Berk passed.
• 03:16 Lucky World Invasion started provisioning.
• 03:26 install was still stuck in the queued phase.
• 03:41 SSH timed out, pack flipped to broken.
• 03:44 third pack passed.
• 03:45 sweep done, all test VMs gone.

A customer trying to order Lucky World Invasion today gets a clear “this pack can’t be hosted right now” message instead of a 30-day commitment to a server that won’t boot.

The next sweep that finds it stale will re-run the test. If the upstream fixes whatever broke, the tag flips back to verified on its own. Nobody on our side has to push a button.

You can see the count

Pass and fail for the week is on our status page. Same number we look at internally. If you’re shopping around and another host claims their modpacks “just work”, ask them to show you the equivalent number.