[OE-core] bug with dpkg-native and sstate-cache mirrors

[Anders Oleson]

Should I open a bug report for this?
Does this make sense and does it sound like a problem?
Are you interested in a patch or fixes? I see some activity with dpkg,
so I know there's a maintainer out there?

On Fri, Dec 16, 2016 at 11:31 AM, Anders Oleson <anders at openpuma.org> wrote:
> I originally posted this here:
> https://lists.yoctoproject.org/pipermail/yocto/2016-December/033542.html.
> Apologies, I did not know to report OE core issues here.
>
> Also, following Jussi's advice I started reading the submission
> guidelines and I posted the patch to dpkg itself to their list to see
> if it was something that could be upstreamed. Led to a good discussion
> here: https://lists.debian.org/debian-dpkg/2016/12/msg00013.html.
> While this was an expedient way to fix my problem, it probably isn't
> the best way forward as a real change to dpkg. They have offered to
> look at submissions to fix what I think is the true root cause - the
> non-override-able, hard-coded CONFIGDIR.
>
> Problem description:
> 1. user "joe" clones the build repo, ex. poky from Yocto and builds
> everything, ex. core-system-minimal completely clean build from
> scratch. The local.conf is set to use package_deb for our system.
> 2. "joe" is the build master and then publishes the resultant
> "sstate-cache" in a shared directory to be used as a mirror for the
> other users. Makes the sstate-cache-mirror directory read-only, etc.
> 3. "joe" deletes the build directory creates a new one and tests the
> build in a new directory which works fine and runs quickly using the
> sstate-cache-mirror.
> 3. user "bob" clones a similar revision and builds using the
> SSTATE_MIRROR pointing at the mirror.
> 4. During "do_rootfs" dpkg (dpkg-native) fails with the message:
> dpkg: error: error opening configuration directory
> '/home/net/joe/work/sysgen-mrp/build/tmp/sysroots/x86_64-linux/etc/dpkg/dpkg.cfg.d':
> Permission denied
> E: Sub-process dpkg returned an error code (2)
>
> What happened is that in dpkg-native, the CONFIGDIR is compiled in and
> hard-coded to the failing path. dpkg does not currently have a way to
> override this at runtime in the same way as --instdir and --admindir.
> So dpkg is still looking for config files user "joes" directory which
> may:
> - have wrong permissions
> - be missing or parent dirs missing
> - contain malicious garbage because "joe" wants to screw with "bob" :)
> - any/all of the above (we had a combination)
>
> Normally /etc/dpkg/dpkg.d is empty for the native sysroot, so our
> quick fix was to modify dpkg to just ignore ANY error reading that
> directory and pretend it was empty (which for Yocto builds it was
> anyway). This was preferable to removing the whole package from the
> SSTATE_MIRROR to force rebuilds in each work directory. See the patch
> I posted to the Yocto list linked above. Debian dpkg developers don't
> want to remove those checks and that seems advisable.
>
> So that leaves two options that I can see (is there an easier/better fix?):
> - we can carry a patch to dpkg-native similar to what I posted. For
> Yocto/OE it probably is good enough, at least if we limit it to
> dpkg-native
> - add something like a --configdir command line switch to dpkg so that
> we can point it toward the proper sysroot rather than use the compiled
> in default
>
> I'd actually prefer the second option because, for one thing, it would
> eliminate the baked in paths that contain user names, etc. I'd suggest
> that if we pass in --configdir we should configure/compile dpkg-native
> with the default paths pointing to neutral, constant, invalid paths to
> avoid leaking build specific information into sstate and to catch
> errors.
>
> Does this sound like I'm on the right track or like something that
> could be included? I'd like to fix this so that it doesn't sneak up on
> someone else.
>
> I'm willing to take a hack at it and test it in the scenario where
> this bit us. It would involve steps:
> 1. develop a patch to dpkg to add the option
> 2. develop a patch for OE to change the configure for dpkg-native
> 3. a patch for OE to pass --configdir to dpkg in all the right places.
> I could use help to insure I find them all.
>
> Thanks,
>
> Anders
>
> error log below:
> ----------------------
> ERROR: system-image-1.0-r0 do_rootfs: Unable to install packages.
> Command '/home/local/MrProductName/mrp-system/build/tmp/sysroots/x86_64-linux/usr/bin/apt-get
>  install --force-yes --allow-unauthenticated bash run-postinsts
> packagegroup-core-eclipse-debug mrp-ofp dosfstools apt e2fsprogs dpkg
> packagegroup-core-boot' returned 100:
> Reading package lists...
> Building dependency tree...
> The following extra packages will be installed:
>   base-files base-passwd busybox busybox-hwclock busybox-syslog busybox-udhcpc
>   ca-certificates debianutils debianutils-run-parts e2fsprogs-badblocks
>   e2fsprogs-e2fsck e2fsprogs-mke2fs eudev gdbserver init-ifupdown initscripts
>   initscripts-functions kernel-4.4.26-yocto-standard kernel-module-uvesafb
>   libblkid1 libbz2-1 libc6 libc6-thread-db libcom-err2 libcrypto1.0.0 libcurl4
>   libe2p2 libext2fs2 libgcc1 libgmp10 libgnutls30 libidn11 libkmod2 liblzma5
>   libperl5 libss2 libssl1.0.0 libstdc++6 libtinfo5 libuuid1 libz1
>   modutils-initscripts ncurses-terminfo-base netbase nettle
>   openssh-sftp-server openssl-conf perl sysvinit sysvinit-inittab
>   sysvinit-pidof tcf-agent udev-cache update-alternatives-opkg update-rc.d
>   v86d xz
> Suggested packages:
>   ncurses-terminfo
> The following NEW packages will be installed:
>   apt mrp-ofp base-files base-passwd bash busybox busybox-hwclock
>   busybox-syslog busybox-udhcpc ca-certificates debianutils
>   debianutils-run-parts dosfstools dpkg e2fsprogs e2fsprogs-badblocks
>   e2fsprogs-e2fsck e2fsprogs-mke2fs eudev gdbserver init-ifupdown initscripts
>   initscripts-functions kernel-4.4.26-yocto-standard kernel-module-uvesafb
>   libblkid1 libbz2-1 libc6 libc6-thread-db libcom-err2 libcrypto1.0.0 libcurl4
>   libe2p2 libext2fs2 libgcc1 libgmp10 libgnutls30 libidn11 libkmod2 liblzma5
>   libperl5 libss2 libssl1.0.0 libstdc++6 libtinfo5 libuuid1 libz1
>   modutils-initscripts ncurses-terminfo-base netbase nettle
>   openssh-sftp-server openssl-conf packagegroup-core-boot
>   packagegroup-core-eclipse-debug perl run-postinsts sysvinit sysvinit-inittab
>   sysvinit-pidof tcf-agent udev-cache update-alternatives-opkg update-rc.d
>   v86d xz
> 0 upgraded, 66 newly installed, 0 to remove and 0 not upgraded.
> Need to get 0 B/7850 kB of archives.
> After this operation, 0 B of additional disk space will be used.
> WARNING: The following packages cannot be authenticated!
>   libc6 libgcc1 libstdc++6 liblzma5 libz1 libgmp10 nettle libidn11 libgnutls30
>   libcurl4 update-alternatives-opkg libtinfo5 base-files bash run-postinsts
>   libperl5 perl xz libbz2-1 dpkg debianutils-run-parts debianutils apt mrp-ofp
>   base-passwd busybox busybox-hwclock busybox-syslog busybox-udhcpc
>   ca-certificates dosfstools libcom-err2 libss2 libuuid1 libblkid1 libe2p2
>   libext2fs2 e2fsprogs-badblocks e2fsprogs e2fsprogs-e2fsck e2fsprogs-mke2fs
>   libkmod2 eudev gdbserver netbase init-ifupdown initscripts-functions
>   initscripts kernel-4.4.26-yocto-standard kernel-module-uvesafb
>   libc6-thread-db libcrypto1.0.0 libssl1.0.0 modutils-initscripts
>   ncurses-terminfo-base openssh-sftp-server openssl-conf v86d sysvinit-pidof
>   sysvinit-inittab sysvinit packagegroup-core-boot tcf-agent
>   packagegroup-core-eclipse-debug udev-cache update-rc.d
> Authentication warning overridden.
> dpkg: error: error opening configuration directory
> '/home/net/joe/work/sysgen-mrp/build/tmp/sysroots/x86_64-linux/etc/dpkg/dpkg.cfg.d':
> Permission denied
> E: Sub-process dpkg returned an error code (2)
>
> ERROR: system-image-1.0-r0 do_rootfs: Function failed: do_rootfs
> ERROR: Logfile of failure stored in:
> /home/local/MrProductName/mrp-system/build/tmp/work/qemux86-hbdc-linux/system-image/1.0-r0/temp/log.do_rootfs.31848
> ERROR: Task 9 (/home/local/MrProductName/mrp-system/poky/../meta-system/recipes-core/images/system-image.bb,
> do_rootfs) failed with exit code '1'