[OE-core] FETCHCMD drop breaks build when append is used (from patch b259bd31eb)

[Andre McCurdy]

On Sat, Aug 31, 2019 at 11:07 AM Andrey Zhizhikin <andrey.z at gmail.com> wrote:
> On Sat, Aug 31, 2019 at 2:18 PM Adrian Bunk <bunk at stusta.de> wrote:
> > Disallowing appends could cause huge problems for a user or layer that
> > has to append local options (e.g. proxy) building a recipe like libedit
> > that has to change the User-Agent.

Clearly we need to have a solution for libedit. I don't see it's
related to user or layer specific issues of setting a password or a
proxy though.

There seem to be three independent cases where appending to the
fetcher commands in bitbake.conf has been used:

1) Setting usernames/passwords for private mirrors, sstate servers,
etc. For that I believe the recommended solution is .netrc. Not only
is putting passwords on the command line a bad idea in general, but it
also won't work consistently for fetchers where download() and
checkstatus() are implemented differently - e.g. for wget, download()
calls wget (and therefore respects FETCHCMD_wget) but checkstatus() is
implemented directly in python (and ignores FETCHCMD_wget completely).
To see the effects, try setting up a password protected http or https
sstate server... it will work if you put your credentials in .netrc
but not if you add them to FETCHCMD_wget.

2) Configuring a proxy server for users behind a firewall. For that I
believe the recommended solution is the various *_proxy environment
variables?

3) Forcing a custom User-Agent. I don't know what the best solution is
here but it feels like the wget fetcher should either handle this
internally (e.g. set a more compatible User-Agent by default?
Automatic retries with various User-Agents?) or provide a documented
API which specifically sets the User-Agent in cases where the default
really does need to be over-ridden.

In the end the bitbake fetchers are abstractions and if users need to
force their own options directly into the final command line (ie
bypassing the abstraction) then it suggests the abstraction is
incomplete. It feels like a slippery slope if we start to encourage or
rely on doing that.