[OE-core] [PATCH v2 3/4] bitbake.conf: omit XZ threads and RAM from sstate signatures

[André Draszik]

The number of threads used, and the amount of memory allowed
to be used, should not affect sstate signatures, as they
don't affect the outcome of the compression if xz operates
in multi-threaded mode [1].

Otherwise, it becomes impossible to re-use sstate from
automated builders on developer's machines (as the former
might execute bitbake with certain constraints different
compared to developer's machines).

This is in particular a problem with the opkg package writing
backend, as the OPKGBUILDCMD depends on XZ_DEFAULTS. Without
the vardepexclude, there is no re-use possible of the
package_write_ipk sstate.

Whitelist the maximum number of threads and the memory limit
given assumptions outlined in [2] below.

Signed-off-by: André Draszik <git at andred.net>

[1] When starting out in multi-threaded mode, the output is always
deterministic, as even if xz scales down to single-threaded later,
the archives are still split into blocks and size information is
still added, thus keeping them compatible with multi-threaded mode.
Also, when starting out in multi-threaded mode, xz never scales
down the compression level to accomodate memory usage restrictions,
it just scales down the number of threads and errors out if it
can not accomodate the memory limit.

[2] Assumptions
* We only support multi-threaded mode (threads >= 2), builds
  should not try to use xz in single-threaded mode
* The thread limit should be set via XZ_THREADS, not via
  modifying XZ_DEFAULTS or XZ_OPTS, or any other way
* The thread limit should not be set to xz's magic value
  zero (0), as that will lead to single-threaded mode on
  single-core systems.
---
 meta/conf/bitbake.conf | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
index 131ba296d3..4b544a22cd 100644
--- a/meta/conf/bitbake.conf
+++ b/meta/conf/bitbake.conf
@@ -795,7 +795,10 @@ BB_NUMBER_THREADS ?= "${@oe.utils.cpu_count()}"
 PARALLEL_MAKE ?= "-j ${@oe.utils.cpu_count()}"
 
 # Default parallelism and resource usage for xz
-XZ_DEFAULTS ?= "--memlimit=50% --threads=${@oe.utils.cpu_count(at_least=2)}"
+XZ_MEMLIMIT ?= "50%"
+XZ_THREADS ?= "${@oe.utils.cpu_count(at_least=2)}"
+XZ_DEFAULTS ?= "--memlimit=${XZ_MEMLIMIT} --threads=${XZ_THREADS}"
+XZ_DEFAULTS[vardepsexclude] += "XZ_MEMLIMIT XZ_THREADS"
 
 ##################################################################
 # Magic Cookie for SANITY CHECK
-- 
2.23.0.rc1