[bitbake-devel] [PATCH] layerindexlib: Fix parsing of recursive layer dependencies

[Mark Hatle]

[YOCTO #13447]

When running bitbake-layers layerindex-fetch from 'master', there is a
circular dependency between meta-oe and meta-python.  This triggered a maximum
recursion depth exception.

To fix the exception, as we walk down a branch (depth first search), we track
the layers we've already seen.  If we are about to recurse into a layer we've
already seen we report a warning and then stop recursion.

Signed-off-by: Mark Hatle <mark.hatle at windriver.com>
---
 lib/layerindexlib/__init__.py | 21 +++++++++++++++++++--
 1 file changed, 19 insertions(+), 2 deletions(-)

diff --git a/lib/layerindexlib/__init__.py b/lib/layerindexlib/__init__.py
index d231cf6a..77196b40 100644
--- a/lib/layerindexlib/__init__.py
+++ b/lib/layerindexlib/__init__.py
@@ -376,7 +376,7 @@ layerBranches set.  If not, they are effectively blank.'''
                 invalid.append(name)
 
 
-        def _resolve_dependencies(layerbranches, ignores, dependencies, invalid):
+        def _resolve_dependencies(layerbranches, ignores, dependencies, invalid, processed=None):
             for layerbranch in layerbranches:
                 if ignores and layerbranch.layer.name in ignores:
                     continue
@@ -388,6 +388,13 @@ layerBranches set.  If not, they are effectively blank.'''
                     if ignores and deplayerbranch.layer.name in ignores:
                         continue
 
+                    # Since this is depth first, we need to know what we're currently processing
+                    # in order to avoid infinite recursion on a loop.
+                    if processed and deplayerbranch.layer.name in processed:
+                        # We have found a recursion...
+                        logger.warning('Circular layer dependency found: %s -> %s' % (processed, deplayerbranch.layer.name))
+                        continue
+
                     # This little block is why we can't re-use the LayerIndexObj version,
                     # we must be able to satisfy each dependencies across layer indexes and
                     # use the layer index order for priority.  (r stands for replacement below)
@@ -411,7 +418,17 @@ layerBranches set.  If not, they are effectively blank.'''
 
                     # New dependency, we need to resolve it now... depth-first
                     if deplayerbranch.layer.name not in dependencies:
-                        (dependencies, invalid) = _resolve_dependencies([deplayerbranch], ignores, dependencies, invalid)
+                        # Avoid recursion on this branch.
+                        # We copy so we don't end up polluting the depth-first branch with other
+                        # branches.  Duplication between individual branches IS expected and
+                        # handled by 'dependencies' processing.
+                        if not processed:
+                            local_processed = []
+                        else:
+                            local_processed = processed.copy()
+                        local_processed.append(deplayerbranch.layer.name)
+
+                        (dependencies, invalid) = _resolve_dependencies([deplayerbranch], ignores, dependencies, invalid, local_processed)
 
                     if deplayerbranch.layer.name not in dependencies:
                         dependencies[deplayerbranch.layer.name] = [deplayerbranch, layerdependency]
-- 
2.16.0.rc2