[bitbake-devel] [PATCH 1/1] bitbake: cookerdata: Avoid double exceptions for bb.fatal()

[Robert Yang]

On 5/14/19 7:02 PM, Robert Yang wrote:
> 
> 
> On 5/12/19 4:28 PM, Richard Purdie wrote:
>> On Thu, 2019-05-09 at 16:03 +0800, Robert Yang wrote:
>>> The bb.fatal() raises BBHandledException() which causes double exceptions,
>>> e.g.:
>>>
>>> Add 'HOSTTOOLS += "hello"' to conf/local.conf:
>>> $ bitbake -p
>>> [snip]
>>> During handling of the above exception, another exception occurred:
>>> [snip]
>>> ERROR: The following required tools (as specified by HOSTTOOLS) appear to be 
>>> unavailable in PATH, please install them in order to proceed:
>>>    hello
>>>
>>> Use "raise" rather than "raise bb.BBHandledException" to fix the double
>>> exceptions.
>>>
>>> [YOCTO #13267]
>>>
>>> Signed-off-by: Robert Yang <liezhi.yang at windriver.com>
>>> ---
>>>   bitbake/lib/bb/cookerdata.py | 4 +++-
>>>   1 file changed, 3 insertions(+), 1 deletion(-)
>>>
>>> diff --git a/bitbake/lib/bb/cookerdata.py
>>> b/bitbake/lib/bb/cookerdata.py
>>> index f8ae410..585edc5 100644
>>> --- a/bitbake/lib/bb/cookerdata.py
>>> +++ b/bitbake/lib/bb/cookerdata.py
>>> @@ -301,7 +301,9 @@ class CookerDataBuilder(object):
>>>               if multiconfig:
>>>                   bb.event.fire(bb.event.MultiConfigParsed(self.mcdata
>>> ), self.data)
>>> -        except (SyntaxError, bb.BBHandledException):
>>> +        except bb.BBHandledException:
>>> +            raise
>>> +        except SyntaxError:
>>>               raise bb.BBHandledException
>>>           except bb.data_smart.ExpansionError as e:
>>>               logger.error(str(e))
>>
>> Hi Robert,
>>
>> This doesn't sound right, where is this exception being printed a
>> second time? The point of "BBHandledException" is to say "don't print
>> any further traces for this exception". If this fixes the bug, it means
>> something somewhere is printing a trace for a second time when it
>> should pass through BBHandledException?
> 
> Hi RP,
> 
> I found another serious problem when tried to raise BBHandledException. There
> is a deadlock when a recipe is failed to parse, e.g.:
> 
> $ echo helloworld >> meta/recipes-extended/bash/bash_4.4.18.bb
> $ bitbake -p
> meta/recipes-extended/bash/bash_4.4.18.bb:42: unparsed line: 'helloworld'
> [hangs]
> 
> Then bitbake hangs, we can use Ctrl-C to break it, but the sub processes
> are still existed, and we need kill them manually, otherwise we can't start
> bitbake again.

BTW, things becomes much better after the following two patches are merged:
bitbake: knotty: Fix for the Second Keyboard Interrupt
bitbake: cooker: Cleanup the queue before call process.join()

Now we hardly can reproduce the problem:
echo helloworld >> meta/recipes-extended/bash/bash_4.4.18.bb
$ while true; do kill-bb; rm -fr bitbake-cookerdaemon.log 
tmp/cache/default-glibc/qemux86-64/x86_64/bb_cache.dat* ; bitbake -p; done

It's not easy to hang any more, but still hangs sometimes, I tried to debug it,
but didn't find the root cause, the ui/knotty.py can't get event from server,
and goes into a dead loop.

             event = eventHandler.waitEvent(0)
             if event is None:
                 if main.shutdown > 1:
                     break
                 termfilter.updateFooter()
                 event = eventHandler.waitEvent(0.25)
                 if event is None:
                     continue

The main.shutdown is always 0 when it hangs.

// Robert

> 
> I think that there are two problems:
> 1) The cooker.py:parse_generator() raises an exception which makes
>     one of result_queue or parser_quit have a deadlock, then
>     causes the shutdown():process.join() hanged.
> 
> 2) The sub processes, parser_quit and result_queue are running dependently,
>     we can't make sure that cooker.py:Parser():realrun() can get data from
>     parser_quit, so there is a race issue, the Parser's subprocess maybe already
>     exited before parser_quit sends data to it.
> 
> I think that we can move part of parse_next()'s code into parse_generator()
> to fix the deadlock problem, and we may need change the working way of
> parser_quit. But I'm not sure, I need your suggestions before work on it.
> 
> // Robert
> 
>>
>> Cheers,
>>
>> Richard
>>
>>