[OE-core] Long delays with latest bitbake (was: [PATCH 1/7] insane.bbclass: in file-rdeps do not look into RDEPENDS recursively)

[Peter Kjellerstedt]

I’m seeing similar results with our builds (Poky + some layers from OpenEmbedded + our own layers; no hash equivalency). Here are some examples of builds from today (with approximate timings added manually by me):

Initialising tasks: 100% |#######################################| Time: 0:00:08
<1 minute without any output>
Checking sstate mirror object availability: 100% |###############| Time: 0:00:23
Sstate summary: Wanted 4151 Found 0 Missed 4151 Current 943 (0% match, 18% complete)
NOTE: Executing Tasks
<3 minutes without any output>
NOTE: Setscene tasks completed
<9 minutes without any output>
No currently running tasks (512 of 12540)   4% |#                              |

After that I aborted the build and removed the tmp directory. The next build started without any of the above delays. However, the display of the then executed setscene tasks by knotty is anything but useful now after the latest changes to bitbake (mostly showing nothing, and occasionally showing a few lines saying that it is running task 0 of 0 which are immediately removed again). Anyway, after that build had run for a couple of thousand tasks it failed with a build error. When I had fixed that and started the build again, I got these timings:

Initialising tasks: 100% |#######################################| Time: 0:00:08
Checking sstate mirror object availability: 100% |###############| Time: 0:00:18
<1.5 minutes without any output>
Sstate summary: Wanted 3870 Found 1 Missed 3869 Current 1224 (0% match, 24% complete)
NOTE: Executing Tasks
<2 minutes without any output>
NOTE: Setscene tasks completed

That build eventually completed normally. Finally, when I started the build again to basically do nothing, I got this:

Initialising tasks: 100% |#######################################| Time: 0:00:08
<3 minutes without any output>
NOTE: Executing Tasks
NOTE: Setscene tasks completed

Comparing that build to a corresponding do-nothing build with Thud, the time difference matches those three minutes where I have no idea what bitbake is doing now that it didn’t need to do before…

Hopefully these time degradations can be solved, because the current state of bitbake is barely usable. We also need to look into possible ways to improve the cooker output when it is running the setscene tasks so it makes some kind of sense again.

//Peter

From: openembedded-core-bounces at lists.openembedded.org <openembedded-core-bounces at lists.openembedded.org> On Behalf Of Alexander Kanavin
Sent: den 12 augusti 2019 21:50
To: Khem Raj <raj.khem at gmail.com>
Cc: OE-core <openembedded-core at lists.openembedded.org>
Subject: Re: [OE-core] [PATCH 1/7] insane.bbclass: in file-rdeps do not look into RDEPENDS recursively

On Mon, 12 Aug 2019 at 21:01, Khem Raj <raj.khem at gmail.com<mailto:raj.khem at gmail.com>> wrote:

Richard, another issue I saw was slowness in world builds when I had
this enabled
I was on master-next for both core and bitbake. It was spending a lot
of time in ensuring
the build is not needed so much that the build took almost same time
than the build with
out it where the build without it will build everything.

world builds seem to have regressed (preparation time wise) even without having hash equivalency. Just oe-core is bearable, but adding meta-oe layers makes it spent many minutes after initializing tasks but before any tasks actually begin.

Alex
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openembedded.org/pipermail/openembedded-core/attachments/20190812/7de31c1e/attachment-0001.html>