#2656 closed defect (fixed)

RSB urlib user agent is blocked by some sites, change to something else.

Reported by: Chris Johns Owned by: Chris Johns
Priority: high Milestone: 4.11
Component: unspecified Version: 4.11
Severity: major Keywords: RSB
Cc: Blocked By:
Blocking:

Description

The libjpeg does not accept the user agent urlib and urllib2 uses. It is silly because it can worked around by submitting a false user agent string.

The work around requires switching to urlib2 in python2.

Change History (4)

comment:1 Changed on Mar 17, 2016 at 3:20:13 AM by Amar Takhar

Try Mozilla/5.0 (Windows NT 6.1; WOW64; rv:43.0) Gecko/20100101 Firefox/43.0

Which is Firefox 43 on Windows.

comment:2 Changed on Mar 17, 2016 at 4:22:42 AM by Chris Johns

So adding a user-agent for http://www.ijg.org/files/jpegsrc.v9a.tar.gz stops the 403 error but it now breaks downloading from Source Forge.

Last edited on Mar 17, 2016 at 4:23:09 AM by Chris Johns (previous) (diff)

comment:3 Changed on Mar 17, 2016 at 4:52:58 AM by Chris Johns

Using Wget/1.16.3 (freebsd10.1) worked. Silly really when you think about it.

comment:4 Changed on Mar 17, 2016 at 5:53:15 AM by Chris Johns <chrisj@…>

Resolution: fixed
Status: newclosed

In 31f33047a278b4ade2d0fb4964ebac4e7209b105/rtems-source-builder:

sb: Change urlib to urllib2 on Python2 and add user agent support.

Some sites block the standard python user agent. Change to Wget
as it seems to make the tested sites behave.

Limit the size of long URLs when logging to the user.

Closes #2656.

Note: See TracTickets for help on using tickets.