[OE-core] [PATCH 1/11] binutils: Add do_check task for executing binutils test suite

[richard.purdie at linuxfoundation.org]

On Wed, 2019-08-28 at 05:06 +0000, Nathan Rossi wrote:
> Create the do_check task to the binutils-cross include. This task can
> be
> used to execute the binutils test suite for the cross target
> binutils.
> By default this executes all the check targets of the binutils
> Makefile,
> this can however be changed by setting MAKE_CHECK_TARGETS to the
> desired
> test suite target (e.g. check-gas).
> 
> The binutils test suites do not require any target execution, as such
> the check target can be run without QEMU or a target device. However
> since the binutils tests do rely on a C compiler there is dependence
> on
> both gcc and libc in order to run the tests.
> 
> Signed-off-by: Nathan Rossi <nathan at nathanrossi.com>

Looking at this a bit further we have a problem with gcc-cross and
binutils-cross.

The trouble is we extracted all "target" dependencies away from the
cross recipes. This means we build *-cross once per target arch but the
runtime pieces are build once per target tune.

The tests are very much target tune specific so shouldn't be in the
cross recipes.

This is going to cause worlds of pain when we switch between machines
that use the same arch but different tunes and the sysroot has files
from a previous different tune.

I suspect this will manifest itself should we run the signature checks
but there real underlying problem here people will run into that we'll
have to solve before this can merge.

There is a secondary issue later in the series with adding the test
results to "selftest".

On the autobuilder we run "selftest" once and assume its machine
independent. These results clearly aren't. This gives problems for both
running the tests and processing the results.

Equally, "cross" toolchain tests aren't really "runtime" but are
machine/target specific. FWIW, injecting the test results the way you
are doing is good and the right thing to do but I think we may need a
different class of test, or we inject them as a special case of runtime
test, or runtime ptest (a cross runtime ptest result?).

I'm happy to try and help figure out how to address some of this if I
can help somehow. I am conscious the release deadlines are pressing too
:(.

Cheers,

Richard