[OE-core] [RFC PATCH 0/5] revive runtime/cases/_ptest.py

[Robert Yang]

Hello,

These patches can make ptest test case work, RP has suggested we write a tool to
do the regression check on ptest result, I think that the use case is like:

$ bitbake <image> -ctestiamge # Suppose we add ptest to default test cases in the future
# Upgrade a recipe form V1.0 to V1.1
$ bitbake <image> -ctestiamge # Run the test again

Then the regression check tool can report what's different (passed, failed,
skipped) between V1.0 and V1.1.

Currently, I'm not sure about where to save the ptest results, I saved it to
${WORKDIR}/testimage/ptest_log atm, e.g.:
tmp/work/qemux86-poky-linux/core-image-minimal/1.0-r0/testimage/ptest_log

But it will be removed after -cclean, then no regression check can be made, so
I'd like to save the ptest results to DEPLOY_DIR_IMAGE if no objections, and
make -cclean not remove them. (Or only keep the latest 2 results).

And I'm not sure where to add the regression check/tool, maybe one of:
1) Add a separate tool in oe-core/scripts, this can make it easy to do regression
   check among different build directories, and runtime/cases/_ptest.py can invoke it.
2) Add it to runtime/cases/_ptest.py directly
3) Add it to buildhistory

I prefer the first one, please feel free to give your comments.

// Robert

The following changes since commit ef68005a8c527e9b1d05b7769f0ec8ebe9ec3f91:

  webkitgtk: Upgrade to 2.16.5 (2017-07-17 13:49:04 +0100)

are available in the git repository at:

  git://git.openembedded.org/openembedded-core-contrib rbt/ptest
  http://cgit.openembedded.org/openembedded-core-contrib/log/?h=rbt/ptest

Robert Yang (5):
  oeqa/targetcontrol.py: simplify checking for qemu_use_kvm
  runtime/cases/_ptest.py: revive it
  oeqa/utils/logparser.py: add skip status
  runtime/cases/_ptest.py: add skip status
  runtime/cases/_ptest.py: rename it to ptest.py

 meta/lib/oeqa/runtime/cases/_ptest.py | 103 ----------------------------------
 meta/lib/oeqa/runtime/cases/ptest.py  |  77 +++++++++++++++++++++++++
 meta/lib/oeqa/targetcontrol.py        |   5 +-
 meta/lib/oeqa/utils/logparser.py      |   5 +-
 4 files changed, 82 insertions(+), 108 deletions(-)
 delete mode 100644 meta/lib/oeqa/runtime/cases/_ptest.py
 create mode 100644 meta/lib/oeqa/runtime/cases/ptest.py

-- 
2.11.0