[OE-core] [oe-commits] [openembedded-core] branch master-next updated: package: Rework PACKAGELOCK based upon sstate for do_packagedata

[Martin Jansa]

I haven't seen this one on oe-core ML, so I'll ask here.

Will this fix the PACKAGELOCK appearing pseudo.log quite often (as
discussed in https://patchwork.openembedded.org/patch/144312/) or do you
have any idea how to fix that one?



On Wed, Jul 18, 2018 at 2:56 PM <git at git.openembedded.org> wrote:

> This is an automated email from the git hooks/post-receive script.
>
> rpurdie pushed a commit to branch master-next
> in repository openembedded-core.
>
> The following commit(s) were added to refs/heads/master-next by this push:
>      new 966b8bf  package: Rework PACKAGELOCK based upon sstate for
> do_packagedata
> 966b8bf is described below
>
> commit 966b8bf2916f75c6dea642786b0e73c7adfc0b30
> Author: Richard Purdie <richard.purdie at linuxfoundation.org>
> AuthorDate: Wed Jul 18 13:54:05 2018 +0100
>
>     package: Rework PACKAGELOCK based upon sstate for do_packagedata
>
>     I think this lock dates from before we had sstate for do_packagedata.
>
>     Since WORKDIR is recipe specific and we write into WORKDIR, we no
> longer need
>     any write locks in the do_packagedata code itself, its handled by the
> sstate
>     task lock for the final copy in at the end. The final write lock can
> be simply
>     removed.
>
>     The only time we need read locking is when actually reading data from
> the
>     shared directory. We can therefore reduce the window the lock is held
>     significantly as well, hence improving the speed of packagedata tasks
> running
>     in parallel.
>
>     Signed-off-by: Richard Purdie <richard.purdie at linuxfoundation.org>
> ---
>  meta/classes/package.bbclass | 23 +++++++++--------------
>  1 file changed, 9 insertions(+), 14 deletions(-)
>
> diff --git a/meta/classes/package.bbclass b/meta/classes/package.bbclass
> index 07bc378..6f7015d 100644
> --- a/meta/classes/package.bbclass
> +++ b/meta/classes/package.bbclass
> @@ -1411,9 +1411,6 @@ fi
>      pkgdest = d.getVar('PKGDEST')
>      pkgdatadir = d.getVar('PKGDESTWORK')
>
> -    # Take shared lock since we're only reading, not writing
> -    lf = bb.utils.lockfile(d.expand("${PACKAGELOCK}"), True)
> -
>      data_file = pkgdatadir + d.expand("/${PN}" )
>      f = open(data_file, 'w')
>      f.write("PACKAGES: %s\n" % packages)
> @@ -1518,7 +1515,6 @@ fi
>      if bb.data.inherits_class('allarch', d) and not
> bb.data.inherits_class('packagegroup', d):
>          write_extra_runtime_pkgs(global_variants, packages, pkgdatadir)
>
> -    bb.utils.unlockfile(lf)
>  }
>  emit_pkgdata[dirs] = "${PKGDESTWORK}/runtime
> ${PKGDESTWORK}/runtime-reverse ${PKGDESTWORK}/runtime-rprovides"
>
> @@ -1629,9 +1625,6 @@ python package_do_shlibs() {
>
>      shlibswork_dir = d.getVar('SHLIBSWORKDIR')
>
> -    # Take shared lock since we're only reading, not writing
> -    lf = bb.utils.lockfile(d.expand("${PACKAGELOCK}"), True)
> -
>      def linux_so(file, needed, sonames, renames, pkgver):
>          needs_ldconfig = False
>          ldir = os.path.dirname(file).replace(pkgdest + "/" + pkg, '')
> @@ -1744,7 +1737,11 @@ python package_do_shlibs() {
>      use_ldconfig = bb.utils.contains('DISTRO_FEATURES', 'ldconfig', True,
> False, d)
>
>      needed = {}
> +
> +    # Take shared lock since we're only reading, not writing
> +    lf = bb.utils.lockfile(d.expand("${PACKAGELOCK}"), True)
>      shlib_provider = oe.package.read_shlib_providers(d)
> +    bb.utils.unlockfile(lf)
>
>      for pkg in shlib_pkgs:
>          private_libs = d.getVar('PRIVATE_LIBS_' + pkg) or
> d.getVar('PRIVATE_LIBS') or ""
> @@ -1800,8 +1797,6 @@ python package_do_shlibs() {
>              d.setVar('pkg_postinst_%s' % pkg, postinst)
>          bb.debug(1, 'LIBNAMES: pkg %s sonames %s' % (pkg, sonames))
>
> -    bb.utils.unlockfile(lf)
> -
>      assumed_libs = d.getVar('ASSUME_SHLIBS')
>      if assumed_libs:
>          libdir = d.getVar("libdir")
> @@ -1914,9 +1909,6 @@ python package_do_pkgconfig () {
>                              if hdr == 'Requires':
>                                  pkgconfig_needed[pkg] += exp.replace(',',
> ' ').split()
>
> -    # Take shared lock since we're only reading, not writing
> -    lf = bb.utils.lockfile(d.expand("${PACKAGELOCK}"), True)
> -
>      for pkg in packages.split():
>          pkgs_file = os.path.join(shlibswork_dir, pkg + ".pclist")
>          if pkgconfig_provided[pkg] != []:
> @@ -1925,6 +1917,9 @@ python package_do_pkgconfig () {
>                  f.write('%s\n' % p)
>              f.close()
>
> +    # Take shared lock since we're only reading, not writing
> +    lf = bb.utils.lockfile(d.expand("${PACKAGELOCK}"), True)
> +
>      # Go from least to most specific since the last one found wins
>      for dir in reversed(shlibs_dirs):
>          if not os.path.exists(dir):
> @@ -1940,6 +1935,8 @@ python package_do_pkgconfig () {
>                  for l in lines:
>                      pkgconfig_provided[pkg].append(l.rstrip())
>
> +    bb.utils.unlockfile(lf)
> +
>      for pkg in packages.split():
>          deps = []
>          for n in pkgconfig_needed[pkg]:
> @@ -1957,8 +1954,6 @@ python package_do_pkgconfig () {
>              for dep in deps:
>                  fd.write(dep + '\n')
>              fd.close()
> -
> -    bb.utils.unlockfile(lf)
>  }
>
>  def read_libdep_files(d):
>
> --
> To stop receiving notification emails like this one, please contact
> the administrator of this repository.
> --
> _______________________________________________
> Openembedded-commits mailing list
> Openembedded-commits at lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-commits
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openembedded.org/pipermail/openembedded-core/attachments/20180718/7bf76808/attachment-0002.html>