相关文章推荐
Traceback (most recent call last):
  File "/home/ubuntu/workspace/dcard/venv/lib/python3.5/site-packages/requests/packages/urllib3/response.py", line 435, in _update_chunk_length
    self.chunk_left = int(line, 16)
ValueError: invalid literal for int() with base 16: b''
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "/home/ubuntu/workspace/dcard/venv/lib/python3.5/site-packages/requests/packages/urllib3/response.py", line 226, in _error_catcher
    yield
  File "/home/ubuntu/workspace/dcard/venv/lib/python3.5/site-packages/requests/packages/urllib3/response.py", line 486, in read_chunked
    self._update_chunk_length()
  File "/home/ubuntu/workspace/dcard/venv/lib/python3.5/site-packages/requests/packages/urllib3/response.py", line 439, in _update_chunk_length
    raise httplib.IncompleteRead(line)
http.client.IncompleteRead: IncompleteRead(0 bytes read)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "/home/ubuntu/workspace/dcard/venv/lib/python3.5/site-packages/requests/models.py", line 660, in generate
    for chunk in self.raw.stream(chunk_size, decode_content=True):
  File "/home/ubuntu/workspace/dcard/venv/lib/python3.5/site-packages/requests/packages/urllib3/response.py", line 340, in stream
    for line in self.read_chunked(amt, decode_content=decode_content):
  File "/home/ubuntu/workspace/dcard/venv/lib/python3.5/site-packages/requests/packages/urllib3/response.py", line 514, in read_chunked
    self._original_response.close()
  File "/usr/lib/python3.5/contextlib.py", line 77, in __exit__
    self.gen.throw(type, value, traceback)
  File "/home/ubuntu/workspace/dcard/venv/lib/python3.5/site-packages/requests/packages/urllib3/response.py", line 244, in _error_catcher
    raise ProtocolError('Connection broken: %r' % e, e)
requests.packages.urllib3.exceptions.ProtocolError: ('Connection broken: IncompleteRead(0 bytes read)', IncompleteRead(0 bytes read))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "spider.py", line 40, in main
    poster.get(callback=lambda posts, forum=forum_name: store_to_db(posts, forum))
  File "/home/ubuntu/workspace/dcard-spider/dcard/posts.py", line 51, in get
    return PostsResult(self.ids, bundle, callback)
  File "/home/ubuntu/workspace/dcard-spider/dcard/posts.py", line 73, in __init__
    self.results = self.format(bundle, callback)
  File "/home/ubuntu/workspace/dcard-spider/dcard/posts.py", line 97, in format
    comments = comments.get() if comments else []
  File "/usr/lib/python3.5/multiprocessing/pool.py", line 608, in get
    raise self._value
  File "/usr/lib/python3.5/multiprocessing/pool.py", line 119, in worker
    result = (True, func(*args, **kwds))
  File "/usr/lib/python3.5/multiprocessing/pool.py", line 44, in mapstar
    return list(map(*args))
  File "/home/ubuntu/workspace/dcard-spider/dcard/posts.py", line 60, in _serially_get_comments
    _comments = client.get(comments_url, params=params)
  File "/home/ubuntu/workspace/dcard-spider/dcard/utils.py", line 31, in get
    response = self.req_session.get(url, **kwargs)
  File "/home/ubuntu/workspace/dcard/venv/lib/python3.5/site-packages/requests/sessions.py", line 480, in get
    return self.request('GET', url, **kwargs)
  File "/home/ubuntu/workspace/dcard/venv/lib/python3.5/site-packages/requests/sessions.py", line 468, in request
    resp = self.send(prep, **send_kwargs)
  File "/home/ubuntu/workspace/dcard/venv/lib/python3.5/site-packages/requests/sessions.py", line 608, in send
    r.content
  File "/home/ubuntu/workspace/dcard/venv/lib/python3.5/site-packages/requests/models.py", line 737, in content
    self._content = bytes().join(self.iter_content(CONTENT_CHUNK_SIZE)) or bytes()
  File "/home/ubuntu/workspace/dcard/venv/lib/python3.5/site-packages/requests/models.py", line 663, in generate
    raise ChunkedEncodingError(e)
requests.exceptions.ChunkedEncodingError: ('Connection broken: IncompleteRead(0 bytes read)', IncompleteRead(0 bytes read))
 

requests.exceptions.ChunkedEncodingError: (‘Connection broken: IncompleteRead(0 bytes read)’, IncompleteRead(0 bytes read))报错原因,服务器http协议1.0,而python搞的是1.1,解决办法就是指定客户端http协议版本,毕竟咱们无法去改服务器的代码。

解决办法:
  • python3
import http.client
http.client.HTTPConnection._http_vsn = 10
http.client.HTTPConnection._http_vsn_str = 'HTTP/1.0'
  • python2
import httplib
httplib.HTTPConnection._http_vsn = 10
httplib.HTTPConnection._http_vsn_str = 'HTTP/1.0'
					
报错:http.client.IncompleteRead: IncompleteRead(180224 bytes read, 39396 more exp
我搜索了一下百度,没有发现类似的答案,不过在stackoverflow有类似的疑问。等会后面有链接。 可能出现这个问题的原因:这里执行urllib的read()函数时候,它会捕获任何不完整的读取异常。因此出现了报错。 我们可以不让它捕获异常,因此当读取链接的时候我们可以用try / catch来抛出异常。 我之前的一段代...
(已解决)pip._vendor.urllib3.exceptions.ProtocolError: (“Connection broken: ConnectionResetError(10054,
使用urllib.request解析很长的页面,会报异常,这是因为服务器分片了,我们只要把剩下的内容拦截下来就可以了。 import urllib.request url = "https://www.iqilu.com" with urllib.request.urlopen(url) as f: res = f.read().decode('utf-8')
python2使用requests发生报错 环境:服务端使用的是IIS 报错:ChunkedEncodingError: (‘Connection broken: IncompleteRead(0 bytes read)’, IncompleteRead(0 bytes read)) 使用此博客的第4中方式:https://blog.csdn.net/wangzuxi/article/d...
使用urllib3上传出现的问题 Connection broken: IncompleteRead(1394 bytes read)', IncompleteRead(1394 bytes read) 原因:网上说此问题出现的原因是服务器的请求只支持HTTP/1.0,urllib3中使用了chunk,但是chunk在HTTP/1.1中才会有,传输还没完成连接就中断了,所以报错 # 引入...
python3 用urllib时,出现http.client.IncompleteRead: IncompleteRead(34144 bytes read) 错误代码: 原出错代码: html = urllib.request.urlopen(request).read().decode('utf-8') 解决方法: html = urllib.request.ur...
 
推荐文章