httpx.RemoteProtocolError: can't handle event type ConnectionClosed when role=SERVER and state=SEND_RESPONSE #1478
Description
Checklist
- The bug is reproducible against the latest release and/or
master. - There are no similar issues or pull requests to fix it yet.
Describe the bug
When issuing a large amount of requests with an async client via an SSH http proxy, the client loses connection and throws an exception. This doesn't happen on localhost.
To reproduce
My RPC adapter:
class AsyncJSONRPC(object):
def __init__(self,
rpc_uri: str = 'http://localhost/'):
self.rpc_uri = rpc_uri
self._client = self._get_httpx_client()
async def close(self):
await self._client.aclose()
async def __aenter__(self):
return self
async def __aexit__(self, *excinfo):
await self.close()
def _get_httpx_client(self) -> httpx.AsyncClient:
"""Return an async httpx client pointing to ethereum_client_rpc_uri."""
return httpx.AsyncClient(base_url=self.rpc_uri)
async def json_rpc_request(self, body: dict) -> httpx.Response:
"""Return the response to the request."""
response = await self._client.post(
url=self.rpc_uri,
json=body
)
return response
async def do_thing(self) -> int:
body = {'jsonrpc': '2.0'}
response = await self.json_rpc_request(body)
return response.json()['result']
class ThingDoer(object):
async def do_many_things(self):
things = []
async with AsyncJSONRPC() as json_rpc:
for i in range(1, 100000):
things.append(json_rpc.do_thing())
await asyncio.gather(*things)
def run(self):
start_time = time.monotonic()
asyncio.run(self.do_many_things())
print(f"Time Taken:{time.monotonic() - start_time}")
worker = ThingDoer()
worker.run()Expected behavior
The client should handle an arbitrary number of requests and then exit.
Actual behavior
Client throws remoteProtocolError: can't handle event type ConnectionClosed when role=SERVER and state=SEND_RESPONSE with more than 10k requests, and more then 1k with http2=True.
Debugging material
unhandled exception during asyncio.run() shutdown
task: <Task finished name='Task-28345' coro=<AsyncJSONRPC.do_thing() done, defined at /home/me/Projects/test/__init__.py:43> exception=RemoteProtocolError("can't handle event type ConnectionClosed when role=SERVER and state=SEND_RESPONSE")>
Traceback (most recent call last):
File "/home/bapperson/.cache/pypoetry/virtualenvs/test-hevE6Mis-py3.9/lib/python3.9/site-packages/httpx/_exceptions.py", line 326, in map_exceptions
yield
File "/home/bapperson/.cache/pypoetry/virtualenvs/test-hevE6Mis-py3.9/lib/python3.9/site-packages/httpx/_client.py", line 1502, in _send_single_request
(status_code, headers, stream, ext,) = await transport.arequest(
File "/home/bapperson/.cache/pypoetry/virtualenvs/test-hevE6Mis-py3.9/lib/python3.9/site-packages/httpcore/_async/connection_pool.py", line 218, in arequest
response = await connection.arequest(
File "/home/bapperson/.cache/pypoetry/virtualenvs/test-hevE6Mis-py3.9/lib/python3.9/site-packages/httpcore/_async/connection.py", line 106, in arequest
return await self.connection.arequest(method, url, headers, stream, ext)
File "/home/bapperson/.cache/pypoetry/virtualenvs/test-hevE6Mis-py3.9/lib/python3.9/site-packages/httpcore/_async/http11.py", line 72, in arequest
) = await self._receive_response(timeout)
File "/home/bapperson/.cache/pypoetry/virtualenvs/test-hevE6Mis-py3.9/lib/python3.9/site-packages/httpcore/_async/http11.py", line 133, in _receive_response
event = await self._receive_event(timeout)
File "/home/bapperson/.cache/pypoetry/virtualenvs/test-hevE6Mis-py3.9/lib/python3.9/site-packages/httpcore/_async/http11.py", line 169, in _receive_event
event = self.h11_state.next_event()
File "/usr/lib/python3.9/contextlib.py", line 135, in __exit__
self.gen.throw(type, value, traceback)
File "/home/bapperson/.cache/pypoetry/virtualenvs/test-hevE6Mis-py3.9/lib/python3.9/site-packages/httpcore/_exceptions.py", line 12, in map_exceptions
raise to_exc(exc) from None
httpcore.RemoteProtocolError: can't handle event type ConnectionClosed when role=SERVER and state=SEND_RESPONSEEnvironment
- OS: Linux host 5.10.14-arch1-1
- Python version: 3.9.1
- HTTPX version: 0.16.1
- Async environment: asyncio, no interest in using trio.
- HTTP proxy: Does not happen on localhost, happens when proxying connection with SSH tunnel
- Custom certificates: No
Additional context
I was just testing throughput and encountered this. Does not happen with short lived clients. It's talking to a custom json rpc, but I don't see this issue with requests, or the non-async version of the client.