bug `make download` will hang forever if networking flush/unstable
Username: Hugo Almeida
Origin: https://bugs.openwrt.org/index.php?do=details&task_id=1967
Supply the following if possible:
- "all", Device problem occurs on
- "master branch", Software versions of OpenWrt/LEDE release, packages, etc.
- "said below", Steps to reproduce
when i call make download
, it hangs/dead,
after debug (tools: htop
and strace
, and read the sources),
i found it because of blocking socket.
details below:
- the python script
scripts/dl_github_archive.py
# https://github.com/openwrt/openwrt/blob/master/scripts/dl_github_archive.py#L402
fileobj = urllib2.urlopen(req, context=sslcontext)
# this will make a blocked socket connection
# if no timeout param, it will use global setting, which is None, it means blocked forever.
# this is bad smell, especially unstable networking.
# ref: https://docs.python.org/3/library/socket.html?highlight=socket#socket-timeouts
running situation:
`
/mnt/op/build/scripts/dl_github_archive.py \
--dl-dir="/mnt/op/build/dl" \
--url="https://github.com/kvalo/ath10k-firmware.git" \
--version="d366b80d41309531c0c98b26625f23e318f33e31" \
--subdir="ath10k-firmware-2018-10-10-d366b80d" \
--source="ath10k-firmware-2018-10-10-d366b80d.tar.xz" \
--hash="6285b9f9aa5a541b8258750f36955a6b307903a87cedd842b73dbfdd2bef8986"
`
which bloked/dead because of networking issue.
if patch that line, like this
`
fileobj = urllib2.urlopen(req, timeout=30, context=sslcontext)
`
it wont block forever even networking lost.
- as a proof, the perl script
scritps/download.pl
has timeout and retry param.
# https://github.com/openwrt/openwrt/blob/master/scripts/download.pl#L85-L86
? (qw(curl -f --connect-timeout 20 --retry 5 --location --insecure), shellwords($ENV{CURL_OPTIONS} || ''), $url)
: (qw(wget --tries=5 --timeout=20 --no-check-certificate --output-document=-), shellwords($ENV{WGET_OPTIONS} || ''), $url)
so, add timeout and retry option for make download
please,
and apply to all process of downloading stuff operation (
include the 2 scripts above and others, e.g git clone
etc.
).