exception - Downloading large file in python error: Compressed file ended before the end-of-stream marker was reached -

i downloading compressed file internet:

with lzma.open(urllib.request.urlopen(url)) file:     line in file:         ... 

after having downloaded , processed a large part of file, error:

file "/usr/lib/python3.4/lzma.py", line 225, in _fill_buffer raise eoferror("compressed file ended before " eoferror: compressed file ended before end-of-stream marker reached

i thinking might caused internet connection drops or server not responding time. if case, there anyway make keep trying, until connection reestablished, instead of throwing exception. don't think problem file, have manually downloaded many files same website manually , decompressed it. have been able download , decompress smaller files python. file trying download has compressed size of 20 gb.

from urllib.urlopen docs:

one caveat: read() method, if size argument omitted or negative, may not read until end of data stream; there no way determine entire stream socket has been read in general case.

maybe lzma.open trips on huge size/connection errors/timeout because of above.