We have a simple app using FastAPI that execute requests to another endpoint but we are seeing poor performance on percentiles P85, P90 and P99 running around 200/rps – we only see slow responses inside our code because same metrics on API Gateway is not possible to see the same latency problem. Our P85 is around 50ms, P90 jumps to 90ms and P99 is around 200ms.

Why ClientSession in async context manager freeze until timeout approximately once in 3-4 runs?

I’m trying to create a payment microservice with following layout: