[OE-core] [PATCH 3/3] rm_work.bbclass: clean up sooner

[Patrick Ohly]

On Wed, 2017-02-15 at 19:32 +0100, Martin Jansa wrote:
> Are all changes necessary for this to work already in master?

Yes.

> Yesterday I've noticed that rm_work for some components which are
> early in the dependency (like qtbase) are executed relatively late
> (together with do_package_qa).

Could do_rm_work run before do_package_qa? rm_work.bbclass doesn't know
that, and therefore schedules do_rm_work after do_package_qa.

If yes, then adding a list of tasks that can be ignored would be
trivial. This can be a variable, so a recipe can even add their own
ones, if necessary.

> So I've tried very naive way to find out if the rm_work tasks are
> executed sooner or not just by comparing Task IDs in build of the same
> image built from scratch (without sstate) with Dizzy, Morty and
> current master.

Interesting, I hadn't thought of testing it like that.

> If we dismiss the strange case in rm_work.tasks.master.qemux86 then it
> seems to perform at least as good as old completion BB_SCHEDULER.
> 
> 
> But I wanted to ask if there is something else we can do or you were
> planing to do, because IIRC you shared some longer analysis of what
> could be improved here and I'm not sure if you managed to implement it
> all.

The other ideas that I mentioned at some point didn't pan out as
intended. In particular allowing do_rm_work tasks to run when the normal
task limit was reached didn't have a big effect and the implementation
was a hack, so I dropped that.

> It feels to me that rm_work has high priority, but still it's
> "blocked" by e.g. do_package_qa which gets executed late and then
> immediately followed by rm_work.

That should be easy to change, perhaps like this (untested):

RM_WORK_TASKS_WHITELIST = "do_build do_package_qa"

        deps = set(bb.build.preceedtask('do_build', True, d))
	whitelist = d.getVar('RM_WORK_TASKS_WHITELIST').split()
        deps.difference_update(whitelist)
        # In practice, addtask() here merely updates the dependencies.
        bb.build.addtask('do_rm_work', 'do_build', ' '.join(deps), d)


-- 
Best Regards, Patrick Ohly

The content of this message is my personal opinion only and although
I am an employee of Intel, the statements I make here in no way
represent Intel's position on the issue, nor am I authorized to speak
on behalf of Intel on this matter.