[OE-core] [PATCH 5/8] oeqa/sdkext/devtool.py: don't reset when the test is failed

[Paul Eggleton]

On Tue, 13 Dec 2016 14:39:46 Robert Yang wrote:
> On 12/13/2016 02:33 PM, Paul Eggleton wrote:
> > On Tue, 13 Dec 2016 14:01:08 Robert Yang wrote:
> >> On 12/13/2016 12:48 PM, Paul Eggleton wrote:
> >>> On Wed, 16 Nov 2016 22:19:34 Robert Yang wrote:
> >>>> The contents are helpful to debug when the error happens.
> >>> 
> >>> In this case removing this is OK because we're operating on an eSDK that
> >>> the tests install. However, this wouldn't be appropriate for the devtool
> >>> oe- selftest tests since they are operating on the user's build system
> >>> itself - not that you probably have any intention of changing those, I
> >>> just wanted to note that.
> >> 
> >> Sorry, I don't quite understand, does selftest uses
> >> oeqa/sdkext/devtool.py,
> >> please ?
> > 
> > No, this is for bitbake -c testsdkext.
> > 
> >> Even if it uses devtool.py, I still think that we need keep the
> >> error contents for debugging when the error happens. It's very hard to
> >> debug when there is no error log or no workspace.
> > 
> > I mean the equivalent oe-selftest tests in oeqa/selftest/devtool.py.
> > 
> > Actually, I've realised something that applies here as well - leaving the
> > files around would be OK if we stopped on the first failure, but we don't
> > -
> > the next test proceeds after it. How will you be able to rely on what's in
> > the workspace if several other tests have run afterwards - not to mention
> > ensure those tests aren't disrupted by the leftover files?
> 
> That's a problem, but we really need consider debugging, otherwise it's
> painful when test are failed but nothing left. How about run test cases
> in different workspaces ?

Currently when I need to debug a test, I tend to either run the test commands 
manually, or comment out the cleanup. If we really want to improve debugging 
then we ought to do it properly - have proper teardown infrastructure and then 
provide a mode that disables it.

Cheers,
Paul

-- 

Paul Eggleton
Intel Open Source Technology Centre