Sleeping automated tests

Building automated test suites for high-volume financial systems taught me that slow tests don’t just waste time. They erode trust in the entire test infrastructure. The most common culprit: sleeping the test runner.

Consequences #

The consequences to sleeping automated tests become more severe as the test suite grows.

Inflate continuous delivery execution time #

If the tests take too long to execute, many tests end up being reclassified so they’re run daily or weekly, not through continuous delivery to prevent artificial bottlenecks in release management.

Bugs are no longer found on commit. They’re found manually by developers or QA during the day, even though you have an automated test covering the use case. Developers find the bug manually and wonder why the automated test missed it.

Worst case: someone authors a new test for a scenario that was already covered.

Queue execution of other tests #

If the testing infrastructure is busy running tests that are sleeping, other tests become queued. The bottleneck is resolved by either removing sleep or increasing infrastructure size — only one of those costs nothing.

Increase execution cost #

Cloud providers charge by execution time. A pipeline sleeping through 10-second waits 50 times per test run is a billing problem, not just a performance problem.

Another common trap: “unlimited” execution under a single pipeline. When that pipeline is busy, teams buy additional pipelines. If the busy pipeline is sleeping, they’ve paid for nothing.

Flakiness (false positives) #

The most common, and most severe consequence. The test fails for a reason outside the scope of the test case — not because the assertion failed, but because the sleep duration was wrong. Fast hardware, cold infrastructure, slow DNS: any of these shifts the timing enough to break the test.

Flaky tests trained by sleep timeouts are a maintenance trap. The “fix” is always to increase the sleep value, which makes the next failure more expensive.

What does sleeping look like? #

Thread.Sleep(1000);

or

await sleep(1000);

Why is this wrong? #

What does the system under test do? Are the operations time-based? Does it provide a service level agreement (SLA) that guarantees execution time is neither faster nor slower than a constant?

In my experience, the target system always has a variable execution time. Even if the system has an SLA of 30 seconds, wait for the completion event — don’t sleep.

If you need to bound how long a test runs, configure a test timeout. That’s a hard limit on total test duration, not a guess about how long each operation takes.

Common reasons for variability:

1. One-time initialization process
2. Recent software deployment
3. Non-cached DNS request
4. New TLS connection
5. Connection pooling with an empty pool
6. Cache miss
7. Internet latency
8. Application recycled after a period of inactivity

These reasons compound. A test that passes consistently in development can fail in CI because the pipeline hits a cold pool and a DNS miss at the same time.

This seems odd, why would anyone choose to sleep a test? #

For request/response API testing, it’s uncommon. Issuing a request and waiting for the response — via a blocking call or async/await — is natural. The problem surfaces in more complex scenarios.

Concurrent requests #

When a test issues concurrent requests, you want to coordinate their completion before asserting or continuing. For example, to efficiently wait for any number of HTTP requests in .NET:

var urls = new string[] { … };

// Issue a concurrent async request for each URL
var requests = urls.Select(httpClient.GetAsync).ToArray();

// Wait for all requests to complete
Task.WaitAll(requests);

If all requests complete in 1 second, we wait 1 second. If the slowest takes 17 seconds, we wait 17 seconds — no longer, no shorter.

Browser-based UI testing #

This is the most common case. The test author guesses a constant pause duration between actions. For example:

1. Navigate to URL
2. Sleep 5 seconds (waiting for page to load)
3. Find username text box
4. Enter username
5. Find password text box
6. Enter password
7. Sleep 1 second
8. Ensure there is no validation error message
9. Click login
10. Sleep 5 seconds
11. Assert login successful

Every sleep value here is a guess. The correct approach waits for observable state:

1. Navigate to URL
2. Wait for page ready (e.g., wait for username field to be visible and interactive)
3. Enter username
4. Enter password
5. Wait for validation state (no error message present)
6. Click login
7. Wait for post-login element (e.g., dashboard header visible)
8. Assert login successful

Browser test frameworks expose event-based wait APIs. Use them. The event-based version is shorter, faster, and doesn’t break when the network is slower than expected.

I don’t want the test to run forever #

Test runners support a test timeout — a hard limit on how long the test is allowed to execute. Configure that at the test level.

Use this instead of building timeouts into every action within the test.

Anti-bot feature requires sleeping #

Some applications detect and block automated tools. This is not an argument to sleep the tests.

The “anti-automation” feature should be implemented outside the core application or behind a feature toggle. For the full test suite, run against a deployment with the feature toggle disabled. This is correct for continuous delivery.

One or more smoke tests can hit the application with the feature enabled to verify that blocking works — and simulated human interactions captured from production can be replayed for that specific scenario.

The bottom line #

Sleep is a guess. Every test that sleeps is encoding an assumption about timing that will eventually be wrong. Replace sleep with event-based waiting, and test timeouts for hard bounds. The result is a test suite that’s faster, cheaper, and trustworthy.