[OE-core] [PATCH 5/8] oeqa/sdkext/devtool.py: don't reset when the test is failed

[Robert Yang]

On 12/13/2016 04:07 PM, Paul Eggleton wrote:
> On Tue, 13 Dec 2016 14:39:46 Robert Yang wrote:
>> On 12/13/2016 02:33 PM, Paul Eggleton wrote:
>>> On Tue, 13 Dec 2016 14:01:08 Robert Yang wrote:
>>>> On 12/13/2016 12:48 PM, Paul Eggleton wrote:
>>>>> On Wed, 16 Nov 2016 22:19:34 Robert Yang wrote:
>>>>>> The contents are helpful to debug when the error happens.
>>>>>
>>>>> In this case removing this is OK because we're operating on an eSDK that
>>>>> the tests install. However, this wouldn't be appropriate for the devtool
>>>>> oe- selftest tests since they are operating on the user's build system
>>>>> itself - not that you probably have any intention of changing those, I
>>>>> just wanted to note that.
>>>>
>>>> Sorry, I don't quite understand, does selftest uses
>>>> oeqa/sdkext/devtool.py,
>>>> please ?
>>>
>>> No, this is for bitbake -c testsdkext.
>>>
>>>> Even if it uses devtool.py, I still think that we need keep the
>>>> error contents for debugging when the error happens. It's very hard to
>>>> debug when there is no error log or no workspace.
>>>
>>> I mean the equivalent oe-selftest tests in oeqa/selftest/devtool.py.
>>>
>>> Actually, I've realised something that applies here as well - leaving the
>>> files around would be OK if we stopped on the first failure, but we don't
>>> -
>>> the next test proceeds after it. How will you be able to rely on what's in
>>> the workspace if several other tests have run afterwards - not to mention
>>> ensure those tests aren't disrupted by the leftover files?
>>
>> That's a problem, but we really need consider debugging, otherwise it's
>> painful when test are failed but nothing left. How about run test cases
>> in different workspaces ?
>
> Currently when I need to debug a test, I tend to either run the test commands
> manually, or comment out the cleanup. If we really want to improve debugging
> then we ought to do it properly - have proper teardown infrastructure and then
> provide a mode that disables it.

Thanks, I will drop the patch atm.

// Robert

>
> Cheers,
> Paul
>