[OE-core] Modifying SRC_URI from anonymous python

[Richard Purdie]

On Thu, 2016-11-17 at 13:28 -0800, Andre McCurdy wrote:
> I have a supplier who provides recipes which set SRC_URI to their
> private git servers. To make those same recipes usable by others (ie
> me), some anonymous python is used to transform the default SRC_URI
> (elements which contain private git URLs are replaced, patches and
> other files are left as-is).
> 
> This apparently worked in OE 2.0 but from 2.1 onwards the anonymous
> python which modifies SRC_URI races with the anonymous python in
> base.bbclass which parses SRC_URI to determine additional
> do_fetch/do_unpack dependencies and whether or not to call
> fetch2.get_srcrev().

I suspect we made some changes around that time to ensure determinism
in the order certain things happened.

>  Specifically I get failures because
> fetch2.get_srcrev() sees the original SRC_URI and tries to resolve
> AUTOREV from a repo to which I don't have access.
> 
> The proposed solution from the supplier is this patch to
> base.bbclass:
> 
> @@ -598,7 +598,7 @@ python () {
>              d.appendVarFlag('do_unpack', 'depends', '
> file-native:do_populate_sysroot')
> 
>      if needsrcrev:
> -        d.setVar("SRCPV", "${@bb.fetch2.get_srcrev(d)}")
> +        d.setVar("SRCPV", "${@bb.fetch2.get_srcrev(d)}",
> parsing=True)
> 
>      set_packagetriplet(d)


parsing=True is a bitbake internal thing, its not documented and
shouldn't be used outside the bitbake codebase. In hindsight, naming
could have been better.

> After reading the setVar source I'm not very clear how or why this
> works, but it looks dubious. What is parsing=True intended to do?

Its internal bitbake data store stuff, don't use or rely on it. It
might happen to work here but its just luck.

> Is it documented somewhere that modifying SRC_URI from anonymous
> python isn't allowed? I've now seen two suppliers both independently
> run into the same problem when updating to OE 2.1.

We have an open bug related to anonymous python ordering. Its hard to
fix easily as something can indicate it wants to run last, but then
everything wants to run last. As Chris mentions, you can use an earlier
event handler as one way to work around it.

I'd also note that anonymous python runs in parsed order. Since
base.bbclass is "up front", its hard to run things before it though :/.

Cheers,

Richard