[OE-core] [RFC PATCH 1/2] cross-localedef-native: Add hardlink resolver from util-linux

[Mark Hatle]

On 8/16/19 4:29 PM, Khem Raj wrote:
> On Fri, Aug 16, 2019 at 2:25 PM Richard Purdie
> <richard.purdie at linuxfoundation.org> wrote:
>>
>> On Fri, 2019-08-16 at 14:22 -0700, Khem Raj wrote:
>>> On Fri, Aug 16, 2019 at 2:06 PM Jason Wessel <
>>> jason.wessel at windriver.com> wrote:
>>>> The hard link resolver that is built into localedef cannot be run
>>>> in
>>>> parallel.  It will search sibling directories (which are be
>>>> processed
>>>> in parallel) and perform a creation of a .tmp file and remove the
>>>> original and move the .tmp file in.  The problem is that if a probe
>>>> occurs a hard link can be requested to the file that is being
>>>> removed.
>>>> This will lead to a stray copy or potentially, on a loaded system
>>>> cause race condition which pseudo cannot deal with, where it is
>>>> left
>>>> with a hard link request to a file that no longer exists.  In this
>>>> situation psuedo will inherit the permissions of what ever the
>>>> target
>>>> inode had to offer.
>>>>
>>>> In short, there are two problems:
>>>>
>>>>   1) You will be left with stray copies when using the hard link
>>>>      resolution that is built in while running in parallel with
>>>>      localedef.
>>>>
>>>>   2) When running under pseudo the possibility exists for uid/gid
>>>>      leakage when the source file is removed before the hard link
>>>> can
>>>>      be completed.
>>>>
>>>> The solution is to call localedef with --no-hard-links and
>>>> separately
>>>> process the hardlinks at a later point.  To do this requires the
>>>> inclusion of the hardlink utility found in modern versions of
>>>> util-linux.  Most host systems do not have this, so it will be
>>>> included with the cross-localedef binary.
>>>>
>>>
>>> can we depend on util-linux-native for this ?
>>
>> This was discussed on IRC. We really really want to avoid that as a
>> dependency if we care about build performance. Its really easy to just
>> add that in but the implications are significant :(
>>
> 
> A captured version of this utility would also mean that we have to
> keep this in sync
> for bug fixes and security fixes etc. which could also be considered
> as expense, so
> how much do we lose if util-linux-native is brought early into dependency chain?

security should not be a significant concern here as the utility is only local
(and used by) the localdef build (consider it an internal to the recipe) tool.
So I'm not concerned about that.

As for the bug fixes, this utility is quite stable and the last changes were
simply to move it from a standalone utility into util-linux.  So again, I find
it unlikely but again since it's an internal tool unless it impacted the cross
localedef functionality it's unlikely to be a problem.

Longer term, as hardlink becomes standard on distributions then the need for the
local tool should go away, and it can be removed at that time.

But currently it's only common on Red Hat lineage machines, while Debian and
Ubuntu either have their own versions, or it's "optional" and not part of the
regular distribution (even though it comes from util-linux).

--Mark

>> Cheers,
>>
>> Richard
>>
>>
>>