相关文章推荐
鬼畜的番茄  ·  Flash受限的GUI开发| ...·  7 月前    · 
儒雅的手电筒  ·  音樂發燒友專用的Hi-Res 專業術語| ...·  7 月前    · 
灰常酷的口罩  ·  《魔都精兵的奴隶》动画化；《杜鹃的婚约》公布 ...·  2 年前    · 
想出国的钥匙扣  ·  用新的人工智能驱动的微软必应和Edge重塑搜 ...·  2 年前    · 
刚毅的火车  ·  小米平板6/6 Pro ...·  2 年前    · 
小百科  ›  Issue 41345: Remote end closed connection without response - Python tracker
post python3
逼格高的山寨机
2 年前

This issue tracker has been migrated to GitHub , and is currently read-only .
For more information, see the GitHub FAQs in the Python's Developer Guide.

Created on 2020-07-20 11:43 by Michal Arbet , last changed 2022-04-11 14:59 by admin . This issue is now closed .

Files File name Uploaded Description mycap.pcap Michal Arbet , 2020-07-20 11:43 I'm not sure if this is really python bug, but I hope that you can check the issue. Issue is that from time to time i'm getting exception from python when sending request to server which has http keepalive option turned on. Requests send a request but in few miliseconds apache2 server is closing persistent connection by sending FIN packet which generate traceback. I can reproduce it by following simple script. #!/usr/bin/python3 import requests from time import sleep import logging logging.basicConfig(level=logging.DEBUG) s = requests.Session() s.verify = False # self-signed cert counter = 0 txt = "test" while True: counter = counter + 1 s.post(' http://localhost ', data={counter:txt}) sleep(5) Everything is working fine, but from time to time I get following traceback. When FIN is received right after request was sent. michalarbet@pixla :~/work$ ./request_test.py DEBUG:urllib3.connectionpool:Starting new HTTP connection (1): localhost:80 DEBUG:urllib3.connectionpool: http://localhost:80 "POST / HTTP/1.1" 200 0 DEBUG:urllib3.connectionpool:Resetting dropped connection: localhost DEBUG:urllib3.connectionpool: http://localhost:80 "POST / HTTP/1.1" 200 0 DEBUG:urllib3.connectionpool:Resetting dropped connection: localhost DEBUG:urllib3.connectionpool: http://localhost:80 "POST / HTTP/1.1" 200 0 DEBUG:urllib3.connectionpool:Resetting dropped connection: localhost DEBUG:urllib3.connectionpool: http://localhost:80 "POST / HTTP/1.1" 200 0 DEBUG:urllib3.connectionpool:Resetting dropped connection: localhost DEBUG:urllib3.connectionpool: http://localhost:80 "POST / HTTP/1.1" 200 0 DEBUG:urllib3.connectionpool:Resetting dropped connection: localhost DEBUG:urllib3.connectionpool: http://localhost:80 "POST / HTTP/1.1" 200 0 DEBUG:urllib3.connectionpool:Resetting dropped connection: localhost DEBUG:urllib3.connectionpool: http://localhost:80 "POST / HTTP/1.1" 200 0 DEBUG:urllib3.connectionpool:Resetting dropped connection: localhost DEBUG:urllib3.connectionpool: http://localhost:80 "POST / HTTP/1.1" 200 0 DEBUG:urllib3.connectionpool:Resetting dropped connection: localhost DEBUG:urllib3.connectionpool: http://localhost:80 "POST / HTTP/1.1" 200 0 DEBUG:urllib3.connectionpool:Resetting dropped connection: localhost DEBUG:urllib3.connectionpool: http://localhost:80 "POST / HTTP/1.1" 200 0 DEBUG:urllib3.connectionpool:Resetting dropped connection: localhost DEBUG:urllib3.connectionpool: http://localhost:80 "POST / HTTP/1.1" 200 0 DEBUG:urllib3.connectionpool:Resetting dropped connection: localhost DEBUG:urllib3.connectionpool: http://localhost:80 "POST / HTTP/1.1" 200 0 DEBUG:urllib3.connectionpool:Resetting dropped connection: localhost DEBUG:urllib3.connectionpool: http://localhost:80 "POST / HTTP/1.1" 200 0 DEBUG:urllib3.connectionpool:Resetting dropped connection: localhost DEBUG:urllib3.connectionpool: http://localhost:80 "POST / HTTP/1.1" 200 0 DEBUG:urllib3.connectionpool:Resetting dropped connection: localhost DEBUG:urllib3.connectionpool: http://localhost:80 "POST / HTTP/1.1" 200 0 DEBUG:urllib3.connectionpool:Resetting dropped connection: localhost DEBUG:urllib3.connectionpool: http://localhost:80 "POST / HTTP/1.1" 200 0 DEBUG:urllib3.connectionpool:Resetting dropped connection: localhost DEBUG:urllib3.connectionpool: http://localhost:80 "POST / HTTP/1.1" 200 0 DEBUG:urllib3.connectionpool:Resetting dropped connection: localhost DEBUG:urllib3.connectionpool: http://localhost:80 "POST / HTTP/1.1" 200 0 DEBUG:urllib3.connectionpool:Resetting dropped connection: localhost DEBUG:urllib3.connectionpool: http://localhost:80 "POST / HTTP/1.1" 200 0 DEBUG:urllib3.connectionpool:Resetting dropped connection: localhost DEBUG:urllib3.connectionpool: http://localhost:80 "POST / HTTP/1.1" 200 0 DEBUG:urllib3.connectionpool:Resetting dropped connection: localhost DEBUG:urllib3.connectionpool: http://localhost:80 "POST / HTTP/1.1" 200 0 DEBUG:urllib3.connectionpool:Resetting dropped connection: localhost DEBUG:urllib3.connectionpool: http://localhost:80 "POST / HTTP/1.1" 200 0 DEBUG:urllib3.connectionpool:Resetting dropped connection: localhost DEBUG:urllib3.connectionpool: http://localhost:80 "POST / HTTP/1.1" 200 0 DEBUG:urllib3.connectionpool:Resetting dropped connection: localhost DEBUG:urllib3.connectionpool: http://localhost:80 "POST / HTTP/1.1" 200 0 DEBUG:urllib3.connectionpool:Resetting dropped connection: localhost DEBUG:urllib3.connectionpool: http://localhost:80 "POST / HTTP/1.1" 200 0 DEBUG:urllib3.connectionpool:Resetting dropped connection: localhost DEBUG:urllib3.connectionpool: http://localhost:80 "POST / HTTP/1.1" 200 0 DEBUG:urllib3.connectionpool:Resetting dropped connection: localhost DEBUG:urllib3.connectionpool: http://localhost:80 "POST / HTTP/1.1" 200 0 DEBUG:urllib3.connectionpool:Resetting dropped connection: localhost DEBUG:urllib3.connectionpool: http://localhost:80 "POST / HTTP/1.1" 200 0 DEBUG:urllib3.connectionpool:Resetting dropped connection: localhost DEBUG:urllib3.connectionpool: http://localhost:80 "POST / HTTP/1.1" 200 0 DEBUG:urllib3.connectionpool:Resetting dropped connection: localhost DEBUG:urllib3.connectionpool: http://localhost:80 "POST / HTTP/1.1" 200 0 DEBUG:urllib3.connectionpool:Resetting dropped connection: localhost DEBUG:urllib3.connectionpool: http://localhost:80 "POST / HTTP/1.1" 200 0 DEBUG:urllib3.connectionpool:Resetting dropped connection: localhost DEBUG:urllib3.connectionpool: http://localhost:80 "POST / HTTP/1.1" 200 0 DEBUG:urllib3.connectionpool:Resetting dropped connection: localhost DEBUG:urllib3.connectionpool: http://localhost:80 "POST / HTTP/1.1" 200 0 DEBUG:urllib3.connectionpool:Resetting dropped connection: localhost DEBUG:urllib3.connectionpool: http://localhost:80 "POST / HTTP/1.1" 200 0 DEBUG:urllib3.connectionpool:Resetting dropped connection: localhost DEBUG:urllib3.connectionpool: http://localhost:80 "POST / HTTP/1.1" 200 0 DEBUG:urllib3.connectionpool:Resetting dropped connection: localhost DEBUG:urllib3.connectionpool: http://localhost:80 "POST / HTTP/1.1" 200 0 DEBUG:urllib3.connectionpool:Resetting dropped connection: localhost DEBUG:urllib3.connectionpool: http://localhost:80 "POST / HTTP/1.1" 200 0 DEBUG:urllib3.connectionpool:Resetting dropped connection: localhost DEBUG:urllib3.connectionpool: http://localhost:80 "POST / HTTP/1.1" 200 0 DEBUG:urllib3.connectionpool:Resetting dropped connection: localhost DEBUG:urllib3.connectionpool: http://localhost:80 "POST / HTTP/1.1" 200 0 DEBUG:urllib3.connectionpool:Resetting dropped connection: localhost DEBUG:urllib3.connectionpool: http://localhost:80 "POST / HTTP/1.1" 200 0 DEBUG:urllib3.connectionpool:Resetting dropped connection: localhost DEBUG:urllib3.connectionpool: http://localhost:80 "POST / HTTP/1.1" 200 0 Traceback (most recent call last): File "/usr/lib/python3/ dist-packages/urllib3/connectionpool.py ", line 665, in urlopen httplib_response = self._make_request( File "/usr/lib/python3/ dist-packages/urllib3/connectionpool.py ", line 421, in _make_request six.raise_from(e, None) File "<string>", line 3, in raise_from File "/usr/lib/python3/ dist-packages/urllib3/connectionpool.py ", line 416, in _make_request httplib_response = conn.getresponse() File "/usr/lib/python3.8/ http/client.py ", line 1332, in getresponse response.begin() File "/usr/lib/python3.8/ http/client.py ", line 303, in begin version, status, reason = self._read_status() File "/usr/lib/python3.8/ http/client.py ", line 272, in _read_status raise RemoteDisconnected("Remote end closed connection without" http.client.RemoteDisconnected: Remote end closed connection without response During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/michalarbet/.local/lib/python3.8/site-packages/requests/adapters.py", line 439, in send resp = conn.urlopen( File "/usr/lib/python3/ dist-packages/urllib3/connectionpool.py ", line 719, in urlopen retries = retries.increment( File "/usr/lib/python3/ dist-packages/urllib3/util/retry.py ", line 400, in increment raise six.reraise(type(error), error, _stacktrace) File "/usr/lib/python3/ dist-packages/six.py ", line 702, in reraise raise value.with_traceback(tb) File "/usr/lib/python3/ dist-packages/urllib3/connectionpool.py ", line 665, in urlopen httplib_response = self._make_request( File "/usr/lib/python3/ dist-packages/urllib3/connectionpool.py ", line 421, in _make_request six.raise_from(e, None) File "<string>", line 3, in raise_from File "/usr/lib/python3/ dist-packages/urllib3/connectionpool.py ", line 416, in _make_request httplib_response = conn.getresponse() File "/usr/lib/python3.8/ http/client.py ", line 1332, in getresponse response.begin() File "/usr/lib/python3.8/ http/client.py ", line 303, in begin version, status, reason = self._read_status() File "/usr/lib/python3.8/ http/client.py ", line 272, in _read_status raise RemoteDisconnected("Remote end closed connection without" urllib3.exceptions.ProtocolError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "./request_test.py", line 26, in <module> s.post(' http://localhost ', data={counter:txt}) File "/home/michalarbet/.local/lib/python3.8/site-packages/requests/sessions.py", line 578, in post return self.request('POST', url, data=data, json=json, **kwargs) File "/home/michalarbet/.local/lib/python3.8/site-packages/requests/sessions.py", line 530, in request resp = self.send(prep, **send_kwargs) File "/home/michalarbet/.local/lib/python3.8/site-packages/requests/sessions.py", line 643, in send r = adapter.send(request, **kwargs) File "/home/michalarbet/.local/lib/python3.8/site-packages/requests/adapters.py", line 498, in send raise ConnectionError(err, request=request) requests.exceptions.ConnectionError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response')) Apache using keepalive. /etc/apache2/apache2.conf:KeepAliveTimeout 5 /etc/apache2/apache2.conf:MaxKeepAliveRequests 100 /etc/apache2/apache2.conf:KeepAlive On Sending also pcap file from different reproduce test, but behaviour is same. 
I'm using  Ubuntu 20.04 with dist apache2 and default settings :
michalarbet@pixla:~/work$ dpkg -l | grep apache
ii  apache2                                    2.4.41-4ubuntu3
          amd64        Apache HTTP Server
ii  apache2-bin                                2.4.41-4ubuntu3
          amd64        Apache HTTP Server (modules and other binary files)
ii  apache2-data                               2.4.41-4ubuntu3
          all          Apache HTTP Server (common files)
ii  apache2-utils                              2.4.41-4ubuntu3
          amd64        Apache HTTP Server (utility programs for web servers)
ii  libapache2-mod-php                         2:7.4+75
         all          server-side, HTML-embedded scripting language (Apache
2 module) (default)
ii  libapache2-mod-php7.4                      7.4.3-4ubuntu2.2
         amd64        server-side, HTML-embedded scripting language (Apache
2 module)
michalarbet@pixla:~/work$ uname -a
Linux pixla 5.4.0-40-generic #44-Ubuntu SMP Tue Jun 23 00:01:04 UTC 2020
x86_64 x86_64 x86_64 GNU/Linux
When reproducing issue, be patient , you have to wait several iterations
before it will fail.
Some bug reports and discussion related to :
https://bugs.python.org/issue41345
https://github.com/psf/requests/issues/4664
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=899406
po 20. 7. 2020 v 14:54 odesílatel Ilia Androshchuk <report@bugs.python.org>
napsal:
> Ilia Androshchuk <an7e@outlook.com> added the comment:
> Hi Michal,
> Which version of operating system and apache are you running?
> I'll try to reproduce
> ----------
> nosy: +an7e
> _______________________________________
> Python tracker <report@bugs.python.org>
> <https://bugs.python.org/issue41345>
> _______________________________________
DEBUG:urllib3.connectionpool:Resetting dropped connection: localhost
DEBUG:urllib3.connectionpool:http://localhost:80 "POST / HTTP/1.1" 200 3138
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 665, in urlopen
    httplib_response = self._make_request(
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 421, in _make_request
    six.raise_from(e, None)
  File "<string>", line 3, in raise_from
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 416, in _make_request
    httplib_response = conn.getresponse()
  File "/usr/lib/python3.8/http/client.py", line 1332, in getresponse
    response.begin()
  File "/usr/lib/python3.8/http/client.py", line 303, in begin
    version, status, reason = self._read_status()
  File "/usr/lib/python3.8/http/client.py", line 272, in _read_status
    raise RemoteDisconnected("Remote end closed connection without"
http.client.RemoteDisconnected: Remote end closed connection without response
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/requests/adapters.py", line 439, in send
    resp = conn.urlopen(
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 719, in urlopen
    retries = retries.increment(
  File "/usr/lib/python3/dist-packages/urllib3/util/retry.py", line 400, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/usr/lib/python3/dist-packages/six.py", line 702, in reraise
    raise value.with_traceback(tb)
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 665, in urlopen
    httplib_response = self._make_request(
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 421, in _make_request
    six.raise_from(e, None)
  File "<string>", line 3, in raise_from
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 416, in _make_request
    httplib_response = conn.getresponse()
  File "/usr/lib/python3.8/http/client.py", line 1332, in getresponse
    response.begin()
  File "/usr/lib/python3.8/http/client.py", line 303, in begin
    version, status, reason = self._read_status()
  File "/usr/lib/python3.8/http/client.py", line 272, in _read_status
    raise RemoteDisconnected("Remote end closed connection without"
urllib3.exceptions.ProtocolError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "test1.py", line 14, in <module>
    s.post('http://localhost', data={counter:txt})
  File "/usr/lib/python3/dist-packages/requests/sessions.py", line 581, in post
    return self.request('POST', url, data=data, json=json, **kwargs)
  File "/usr/lib/python3/dist-packages/requests/sessions.py", line 533, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/lib/python3/dist-packages/requests/sessions.py", line 646, in send
    r = adapter.send(request, **kwargs)
  File "/usr/lib/python3/dist-packages/requests/adapters.py", line 498, in send
    raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))
I'm using same dist and settings:
lerfi@UbuntuVM:~/work/psf/41345$ uname -a
Linux UbuntuVM 5.4.0-40-generic #44-Ubuntu SMP Tue Jun 23 00:01:04 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
erfi@UbuntuVM:~/work/psf/41345$ dpkg -l | grep apache
ii  apache2                                    2.4.41-4ubuntu3                       amd64        Apache HTTP Server
ii  apache2-bin                                2.4.41-4ubuntu3                       amd64        Apache HTTP Server (modules and other binary files)
ii  apache2-data                               2.4.41-4ubuntu3                       all          Apache HTTP Server (common files)
ii  apache2-utils                              2.4.41-4ubuntu3                       amd64        Apache HTTP Server (utility programs for web servers)
pip3 requests==2.22.0
it looks like there's a problem with this urllib3 package
Thank you very much for help, I created pull request for urllib3 -> https://github.com/urllib3/urllib3/pull/1911 Can u just confirm that this could be fixed in urllib3 as it is in pull request ? Have I fixed it by good way ? Thanks. Michal Arbet ( kevko ) pá 24. 7. 2020 v 20:29 odesílatel Guido van Rossum < report@bugs.python.org > napsal: > Guido van Rossum < guido@python.org > added the comment: > This surely is some application corner case. Closing. > ---------- > nosy: +gvanrossum > resolution: -> not a bug > stage: -> resolved > status: open -> closed > _______________________________________ > Python tracker < report@bugs.python.org > > < https://bugs.python.org/issue41345 > > _______________________________________ 
Previous report about Requests to the Python bug tracker: Issue 33620.
I suspect this is an unavoidable race condition with trying a POST (or other non-idempotent) request on an idle HTTP connection. I think it has to be up to the higher-level application or user to decide if it is safe to retry a POST request. Otherwise you risk e.g. accidentally ordering two pizzas because the server received two POST requests but something interfered with the response of the first response. On the other hand, I noticed some browsers seem to automatically retry a POST once if it is interrupted, which makes me uneasy.
A concrete example of the problem is a firmware upload that triggers a reboot. If the reboot is too quick and prevents the POST response being sent, I found that a web browser will repeat the firmware upload once more after my firmware boots up again.
If it is not safe to retry the POST request, other options would be to avoid the server thinking the connection is stale:
* always do the POST request on a fresh HTTP connection
* "ping" the connection with a dummy request (e.g. OPTIONS, HEAD, or GET) immediately before the POST request
Another option that comes to mind is to try using the 100 Continue mechanism, but this is not a general solution and depends on the application.
2022-04-11 14:59:33adminsetgithub: 85517 2020-07-28 05:47:14martin.pantersetnosy: + martin.panter
messages: + msg374466
2020-07-27 08:04:37Michal Arbetsetmessages: + msg374365 2020-07-24 18:28:51gvanrossumsetstatus: open -> closed

nosy: + gvanrossum
messages: + msg374200

resolution: not a bug
stage: resolved 2020-07-21 13:18:28an7esetmessages: + msg374066 2020-07-20 23:17:21an7esetmessages: + msg374034 2020-07-20 13:48:46Michal Arbetsetmessages: + msg374008 2020-07-20 12:54:48an7esetnosy: + an7e
messages: + msg374004
2020-07-20 11:43:46Michal Arbetcreate
 
推荐文章
鬼畜的番茄  ·  Flash受限的GUI开发| TouchGFX Documentation
7 月前
儒雅的手电筒  ·  音樂發燒友專用的Hi-Res 專業術語| Sony Hong Kong
7 月前
灰常酷的口罩  ·  《魔都精兵的奴隶》动画化；《杜鹃的婚约》公布新PV，最近一些 ...
2 年前
想出国的钥匙扣  ·  用新的人工智能驱动的微软必应和Edge重塑搜索，你的网络副驾驶 ...
2 年前
刚毅的火车  ·  小米平板6/6 Pro 正式发布：首销价1899 元起、11 英寸2.8K 护眼屏 ...
2 年前
今天看啥   ·   Py中国   ·   codingpro   ·   小百科   ·   link之家   ·   卧龙AI搜索
删除内容请联系邮箱 2879853325@qq.com
小百科 - 百科知识指南
© 2024 ~ 沪ICP备11025650号