An ASP.NET Core application often depends on external services to complete a request. It might call a payment gateway, retrieve customer information from another API, or send data to a third-party service. When these dependencies become slow or temporarily unavailable, the application's response time and reliability can suffer.
A common mistake is to send an HTTP request and wait for it to finish without defining how long the application should wait or what should happen when the request fails.
Retries and timeouts help address these problems, but they must be configured carefully. Retrying every failure can overload an already struggling service, while excessively long timeouts can consume application resources. Retrying a non-idempotent operation can also create duplicate transactions.
ASP.NET Core provides IHttpClientFactory for managing HTTP clients, while the standard resilience libraries provide policies for timeouts, retries, and other transient failures.
This article explains how to configure outbound HTTP requests, apply retries and timeouts, handle common failures, and choose settings suitable for production applications.
Why Outbound HTTP Requests Need Resilience
An HTTP request can fail for several reasons:
The remote service is temporarily unavailable.
A network connection is interrupted.
The remote service responds more slowly than expected.
The service returns a temporary error such as HTTP 503.
The application reaches a configured timeout.
The remote service rejects the request because of authentication or validation errors.
Not all failures should trigger a retry. A temporary service outage might justify another attempt, but an HTTP 400 response usually indicates invalid input that will not become valid by repeating the request.
A resilient HTTP client needs to distinguish transient failures from permanent ones and place a limit on the time and number of attempts.
Configure IHttpClientFactory
IHttpClientFactory provides a central place to configure HTTP clients, manage their underlying handlers, and apply logging and resilience policies.
Install the standard HTTP resilience package:
dotnet add package Microsoft.Extensions.Http.ResilienceThis package provides resilience integration for HttpClient, including support for retry and timeout strategies.
Next, register a named client in Program.cs.
using Microsoft.Extensions.Http.Resilience;
using Polly;
var builder = WebApplication.CreateBuilder(args);
builder.Services
.AddHttpClient("CatalogApi", client =>
{
client.BaseAddress =
new Uri("https://catalog.example.internal/");
client.Timeout = Timeout.InfiniteTimeSpan;
})
.AddStandardResilienceHandler(options =>
{
options.TotalRequestTimeout.Timeout =
TimeSpan.FromSeconds(10);
options.AttemptTimeout.Timeout =
TimeSpan.FromSeconds(3);
options.Retry.MaxRetryAttempts = 2;
options.Retry.Delay =
TimeSpan.FromMilliseconds(300);
options.Retry.BackoffType =
DelayBackoffType.Exponential;
options.Retry.UseJitter = true;
});
var app = builder.Build();
app.MapGet("/", () => "API is running.");
app.Run();The example configures a client for a fictional catalog service. Replace the base address with the actual service endpoint.
The important settings are:
AddHttpClientregisters a client managed by the factory.Timeout.InfiniteTimeSpandisables theHttpClient-level timeout so that the resilience handler controls the request deadline.TotalRequestTimeoutlimits the overall time spent on the outbound operation, including retry attempts and their delays.AttemptTimeoutlimits an individual attempt.MaxRetryAttempts = 2allows up to two retries after the initial attempt.Exponential backoff increases the delay between retries.
Jitter adds variation to retry delays, reducing the chance that many clients retry simultaneously.
The total timeout must be large enough to accommodate the attempt timeouts and retry delays. The outer deadline still takes precedence if the operation takes too long.
The standard resilience handler also includes other strategies, such as a circuit breaker and rate limiting. Review the defaults and adjust them to the dependency's behavior rather than assuming every service needs identical settings.
Understand How Retry and Timeout Policies Work
A retry policy repeats an operation after a qualifying failure. A timeout policy limits how long an operation can continue.
These strategies solve different problems.
Strategy | Purpose | Example |
|---|---|---|
Attempt timeout | Limits the duration of one attempt | Stop waiting after three seconds |
Total request timeout | Limits the entire operation | End the operation after ten seconds |
Retry | Repeats eligible transient failures | Retry a temporary HTTP 503 response |
Circuit breaker | Temporarily blocks calls to an unhealthy dependency | Stop sending requests after repeated failures |
Rate limiter | Limits concurrent or incoming operations | Prevent excessive outbound concurrency |
Retries do not make a remote service available. They provide another opportunity for a temporary failure to recover. If the dependency remains unhealthy, repeated attempts can increase load and make the problem worse.
Use the Configured Client in an ASP.NET Core Service
Once the client is registered, inject IHttpClientFactory into the service that calls the remote API.
using System.Net;
using System.Net.Http.Json;
public sealed class CatalogService
{
private readonly IHttpClientFactory _httpClientFactory;
public CatalogService(
IHttpClientFactory httpClientFactory)
{
_httpClientFactory = httpClientFactory;
}
public async Task<ProductDto?> GetProductAsync(
int productId,
CancellationToken cancellationToken = default)
{
if (productId <= 0)
{
throw new ArgumentOutOfRangeException(
nameof(productId));
}
var client =
_httpClientFactory.CreateClient("CatalogApi");
using var response = await client.GetAsync(
$"api/products/{productId}",
cancellationToken);
if (response.StatusCode == HttpStatusCode.NotFound)
{
return null;
}
response.EnsureSuccessStatusCode();
return await response.Content
.ReadFromJsonAsync<ProductDto>(
cancellationToken: cancellationToken);
}
}
public sealed record ProductDto(
int Id,
string Name,
decimal Price);The service uses the named client so that all requests inherit its configured resilience behavior.
The cancellation token allows the calling operation to stop waiting when its work is no longer needed. For example, an incoming HTTP request may be cancelled because the client disconnected.
The code treats HTTP 404 as a missing product rather than an exceptional failure. Other unsuccessful status codes are passed to EnsureSuccessStatusCode, which raises an exception.
The resilience handler can retry eligible failures before the service receives the final response. The application should still handle the final failure appropriately.
Which HTTP Failures Should Be Retried?
A retry policy should focus on failures that may resolve without changing the request.
Failure | Usually retry? | Reason |
|---|---|---|
HTTP 400 Bad Request | No | The request is usually invalid |
HTTP 401 Unauthorized | No | Credentials or authentication must be corrected |
HTTP 404 Not Found | No | The requested resource may not exist |
HTTP 408 Request Timeout | Sometimes | The server may have timed out processing the request |
HTTP 429 Too Many Requests | Sometimes | Respect the server's retry guidance and rate limits |
HTTP 500 Internal Server Error | Depends | The error may be transient or caused by a persistent defect |
HTTP 502 Bad Gateway | Often | A gateway may be experiencing a temporary upstream failure |
HTTP 503 Service Unavailable | Often | The service may be temporarily unavailable |
Network interruption | Often | A subsequent attempt may succeed |
The standard resilience handler uses predefined behavior for eligible HTTP responses and transient failures. Confirm the installed package's defaults and configure the retryable status codes and exceptions when the dependency requires a different policy.
For HTTP 429 responses, respect Retry-After when provided and supported by the configured strategy. Avoid retrying faster than the server permits.
Common Mistakes When Configuring Retries
Retrying operations that are not idempotent
Consider an API that charges a customer's payment method. The first request may reach the payment provider and complete successfully, but the response may be lost because of a network interruption.
If the application blindly sends the same charge request again, the customer could be charged twice.
For payment creation, order creation, and similar operations, use the provider's idempotency mechanism when available. Reuse the same idempotency key for retries of the same logical operation.
Do not assume that a POST request is safe to retry merely because the HTTP client can repeat it.
Setting an excessive timeout
A long timeout can keep requests waiting while consuming sockets, memory, and other resources. Under heavy traffic, slow dependencies can contribute to thread-pool pressure and growing request queues.
Choose timeouts based on the application's response-time objective and the dependency's expected behavior.
Retrying every exception
Authentication failures, invalid requests, and programming errors usually require a different response than a temporary network failure.
Keep retry conditions narrow and avoid repeating operations that cannot succeed without a change in input or configuration.
Ignoring cancellation
Do not remove cancellation support from application services. Propagate the caller's CancellationToken through outbound operations so that cancelled work does not continue unnecessarily.
Best Practices for Production Applications
Set an end-to-end deadline
Choose a total outbound timeout that fits within the incoming request's overall deadline. If the application has only a limited time to respond, it cannot spend that entire period independently on every downstream service.
For workflows involving multiple services, allocate time budgets to each dependency rather than allowing every call to consume the full request deadline.
Use bounded retries with backoff
Keep the retry count small and increase delays between attempts when appropriate. Jitter helps distribute retry traffic over time.
Avoid stacking multiple independent retry policies without calculating their combined effect. If several layers retry the same operation, the total number of downstream attempts can grow rapidly.
Monitor the dependency
Track request duration, timeout frequency, HTTP status codes, retry counts, and final failure rates. These measurements help determine whether the problem comes from the application, the network, or the downstream service.
Avoid recording authorization headers, access tokens, or sensitive request and response bodies in logs.
Test failure scenarios
Use a test server or controlled mock to simulate slow responses, HTTP 503 responses, connection failures, and successful responses after a transient error.
Verify that the configured number of attempts is respected, the overall deadline is enforced, and cancellation stops unnecessary work.
Advantages and Disadvantages
Advantages | Disadvantages |
|---|---|
Transient failures can recover without immediate application failure. | Retries increase request volume and may add latency. |
Timeouts prevent requests from waiting indefinitely. | Aggressive timeouts can fail legitimate slow operations. |
Centralized configuration improves consistency. | Shared defaults may not suit every downstream service. |
Backoff and jitter help reduce synchronized retry traffic. | Poorly coordinated policies can amplify outages. |
Metrics make dependency failures easier to investigate. | Monitoring and testing add implementation effort. |
Summary
Retries and timeouts are important parts of reliable outbound HTTP communication in ASP.NET Core. IHttpClientFactory provides a centralized way to configure clients, while Microsoft.Extensions.Http.Resilience supplies strategies for controlling request duration and handling transient failures.
Start with a clear total request deadline, a shorter per-attempt timeout, and a small retry count. Use backoff and jitter, respect server rate limits, propagate cancellation, and monitor downstream failures.
Most importantly, retry only when another attempt is appropriate. For operations that can create payments, orders, or other persistent changes, use idempotency protections before enabling automatic retries. A carefully configured client can improve resilience without turning a temporary dependency problem into a larger outage.
Windows Hosting Recommendation
0 comments:
Post a Comment