[OE-core] Long delays with latest bitbake (was: [PATCH 1/7] insane.bbclass: in file-rdeps do not look into RDEPENDS recursively)

[richard.purdie at linuxfoundation.org]

On Fri, 2019-08-16 at 17:00 +0200, Martin Jansa wrote:
> > Will try to bump BB_NUMBER_THREADS from 8 to 72.
> 
> I've tried to remove icecc.bbclass inherit (because it does things
> while parsing and RP probably doesn't have it inherited), but that
> didn't save much time.
> 
> All 3 tests were with bitbake
> 18d4a31fdcec1f0e5d2199d6142f0ce833fca1a7
> 94m19.081s  8 BB_NUMBER_THREADS and icecc
> 82m59.574s  8 BB_NUMBER_THREADS no icecc
> 68m3.556s    72 BB_NUMBER_THREADS no icecc
> 
> Still don't know how to get to sub 10min world runs RP was seeing,
> but at least it's as slow as it was before runqeueu changes (or even
> a bit faster in lastest master).

Just thinking out loud, other things which can influence timings:

Is SSTATE_DIR on NFS or local disk?
Are sstate mirrors configured?
Is there an existing build or not, if so, how much is valid?
Underlying filesystem of the build?

Your build seems especially slow at executing through the tasks which
is effectively a test on how fast the system can fork() and return in
some ways. It would be interesting to block dry-run on the server side,
skip the fork and see how the numbers compare.

My build manages some parts of the tasklist faster than others, perhaps
because there are more "free" tasks to execute at some points in the
task graph than others.

Also, I have some patches which improve performance a bit which I'm
still testing.

Cheers,

Richard