[OE-core] [RFC PATCH] Add gnu testsuite execution for OEQA

[Khem Raj]

On Tue, Jul 9, 2019 at 4:48 AM Nathan Rossi <nathan at nathanrossi.com> wrote:
>
> On Tue, 9 Jul 2019 at 21:16, Burton, Ross <ross.burton at intel.com> wrote:
> >
> > This is awesome!
> >
> > On Sat, 6 Jul 2019 at 12:39, Nathan Rossi <nathan at nathanrossi.com> wrote:
> > >         | gcc          | g++          | libstdc++   | binutils    | gas         | ld          | glibc
> > > x86-64  |   589/135169 |   457/131913 |     1/13008 |     0/  236 |     0/ 1256 |   166/ 1975 |  1423/ 5991
> > > arm     |   469/123905 |   365/128416 |    19/12788 |     0/  191 |     0/  872 |   155/ 1479 |    64/ 5130
> > > aarch64 |   460/130904 |   364/128977 |     1/12789 |     0/  190 |     0/  442 |   157/ 1474 |    76/ 5882
> > > powerpc | 18336/116624 |  6747/128636 |    33/12996 |     0/  187 |     1/  265 |   157/ 1352 |  1218/ 5110
> > > mips64  |  1174/134744 |   401/130195 |    22/12780 |     0/  213 |    43/ 7245 |   803/ 1634 |  2032/ 5847
> > > riscv64 |   456/106399 |   376/128427 |     1/12748 |     0/  185 |     0/  257 |   152/ 1062 |    88/ 5847
> >
> > So what I'm really interested in is what those numbers look like for
> > e.g. x86-64 on real hardware: are the test suites always failing a
> > bit, or are these indicative of problems we've introduced?
>

these numbers id PASS/FAIL are not usual, usually it should be only
handful of failures, but this could be
some simple basic missing piece, since dejaGNU is quite picky at times.

> Just to clarify, by real hardware are you interested in the tests
> running against a system running with a oe distro + kernel. Or just
> running on e.g. the build host?
>
> As for the failing tests, do not put to much emphasis on the values
> provided. I am sure there are a number of tests that are failing due
> to minor configuration issues that simply need a deeper look at the
> test result and output to fix the issue. And the large number of
> failures for powerpc are due to qemu user failing due to illegal
> instruction.
>
> The goal would be to get the results similar to what other
> users/distro get, by reviewing results posted here:
> https://gcc.gnu.org/ml/gcc-testresults/.
> And for glibc the results from: https://sourceware.org/glibc/wiki/Release/2.29
>
> Thanks,
> Nathan
> --
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core at lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core