[OE-core] [PATCH] image: Attempt to work around do_image_complete race

[Khem Raj]

On Mon, Feb 13, 2017 at 4:41 AM, Mike Crowe <mac at mcrowe.com> wrote:
> If two do_image_complete tasks run in parallel they risk both trying to put
> their image into ${DEPLOY_DIR_IMAGE} at the same time. Both will contain a
> README_-_DO_NOT_DELETE_FILES_IN_THIS_DIRECTORY.txt file. In theory this
> should be safe because "cp -alf" will just cause one to overwrite the
> other. Unfortunately, coreutils cp also has a race which means that if one
> copy creates the file at just the wrong point the other will fail with:
>
>  cp: cannot create hard link ‘..../tmp-glibc/deploy/images/pantera/README_-_DO_NOT_DELETE_FILES_IN_THIS_D.txt’ to ‘..../tmp-glibc/work/rage_against-oe-linux-gnueabi/my-own-image/1.0-r0/deploy-my-own-image-complete/README_-_DO_NOT_DELETE_FILES_IN_THIS_DIRECTORY.txt’: File exists
>
> The coreutils bug has been reported and fixed upstream as
> https://debbugs.gnu.org/cgi/bugreport.cgi?bug=25680 , but it will take
> a while for host distributions to catch up.
>
> In the meantime, let's try and avoid the race altogether by causing
> sstate.bbclass:sstate_install to take a lock file in
> ${DEPLOY_DIR_IMAGE} whilst writing. This lock is probably a bit broad,
> but it's better than the build failing.
>

will this lock serialize writing other files too ? then this will slow
down parallel builds. Perhaps, writing common files can be spun into a
task of its own which is then common 1 task across all images building
in parallel.

> Signed-off-by: Mike Crowe <mac at mcrowe.com>
> ---
>  meta/classes/image.bbclass | 4 ++++
>  1 file changed, 4 insertions(+)
>
> diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass
> index b5a4fb4a33..5c2f183446 100644
> --- a/meta/classes/image.bbclass
> +++ b/meta/classes/image.bbclass
> @@ -298,6 +298,10 @@ SSTATE_SKIP_CREATION_task-image-complete = '1'
>  do_image_complete[sstate-inputdirs] = "${IMGDEPLOYDIR}"
>  do_image_complete[sstate-outputdirs] = "${DEPLOY_DIR_IMAGE}"
>  do_image_complete[stamp-extra-info] = "${MACHINE}"
> +# Defend against copying two different images to ${DEPLOY_DIR_IMAGE}
> +# at the same time because they risk both trying to create
> +# README_-_DO_NOT_DELETE_FILES_IN_THIS_DIRECTORY.txt at the same time.
> +do_image_complete[sstate-lockfile-shared] = "${DEPLOY_DIR_IMAGE}/image-complete-output.lock"
>  addtask do_image_complete after do_image before do_build
>
>  # Add image-level QA/sanity checks to IMAGE_QA_COMMANDS
> --
> 2.12.0.rc0
>
> --
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core at lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core