[OE-core] Contents of non-rootfs partitions

[Andreas Oberritter]

On 24.11.2016 15:51, Ed Bartosh wrote:
> On Thu, Nov 24, 2016 at 03:43:18PM +0100, Kristian Amlie wrote:
>> On 24/11/16 14:23, Ed Bartosh wrote:
>>> On Thu, Nov 24, 2016 at 08:38:46AM +0100, Kristian Amlie wrote:
>>>> On 24/11/16 07:15, Ulrich Ölmann wrote:
>>>>> Hi,
>>>>>
>>>>> On Wed, Nov 23, 2016 at 04:56:56PM +0100, Patrick Ohly wrote:
>>>>>> On Wed, 2016-11-23 at 15:22 +0200, Ed Bartosh wrote:
>>>>>>> On Wed, Nov 23, 2016 at 02:08:28PM +0100, Kristian Amlie wrote:
>>>>>>>> On 23/11/16 13:08, Ed Bartosh wrote:
>>>>>>>>> On Tue, Nov 22, 2016 at 12:54:52PM +0100, Kristian Amlie wrote:
>>>>>>>>> [...]
>>>>>>>>> This can be done by extending existing rootfs plugin. It should be able
>>>>>>>>> to do 2 things:
>>>>>>>>>
>>>>>>>>> - populate content of one rootfs directory to the partition. We can
>>>>>>>>>   extend syntax of --rootfs-dir parameter to specify optional directory path to use
>>>>>>>>>
>>>>>>>>> - exclude rootfs directories when populating partitions. I'd propose to
>>>>>>>>>   introduce --exclude-dirs wks parser option to handle this.
>>>>>>>>>
>>>>>>>>> Example of wks file with proposed new options:
>>>>>>>>> part /     --source rootfs --rootfs-dir=core-image-minimal       --ondisk sda --fstype=ext4 --label root --align 1024 --exclude-dirs data --exclude-dirs home
>>>>>>>>> part /data --source rootfs --rootfs-dir=core-image-minimal:/home --ondisk sda --fstype=ext4 --label data --align 1024
>>>>>>>>> part /home --source rootfs --rootfs-dir=core-image-minimal:/data --ondisk sda --fstype=ext4 --label data --align 1024
>>>>>>>>>
>>>>>>>>> Does this make sense?
>>>>>>>>
>>>>>>>> Looks good. The only thing I would question is that, in the interest of
>>>>>>>> reducing redundancy, maybe we should omit --exclude-dirs and have wic
>>>>>>>> figure this out by combining all the entries, since "--exclude-dirs
>>>>>>>> <dir>" and the corresponding "part <dir>" will almost always come in
>>>>>>>> pairs. Possibly we could mark the "/" partition with one single
>>>>>>>> --no-overlapping-dirs to force wic to make this consideration. Or do you
>>>>>>>> think that's too magical?
>>>>>>>>
>>>>>>> Tt's quite implicit from my point of view. However, if people like it we
>>>>>>> can implement it this way.
>>>>>>
>>>>>> I prefer the explicit --exclude-dirs. It's less surprising and perhaps
>>>>>> there are usages for having the same content in different partitions
>>>>>> (redundancy, factory reset, etc.).
>>>>>>
>>>>>> Excluding only the directory content but not the actual directory is
>>>>>> indeed a good point. I'm a bit undecided. When excluding only the
>>>>>> directory content, there's no way of building a rootfs without that
>>>>>> mount point, if that's desired. OTOH, when excluding also the directory,
>>>>>> the data would have to be staged under a different path in the rootfs
>>>>>> and the mount point would have to be a separate, empty directory.
>>>>>>
>>>>>> I'm leaning towards excluding the directory content and keeping the
>>>>>> directory.
>>>>>
>>>>> what about having both possibilities by leaning against the syntax that rsync
>>>>> uses to specify if a whole source directory or only it's contents shall be
>>>>> synced to some destination site (see [1])?
>>>>>
>>>>> In analogy to this to exclude only the contents of the directory named 'data'
>>>>> you would use
>>>>>
>>>>>   --exclude-dirs data/
>>>>>
>>>>> but to additionally exclude the dir itself as well it would read
>>>>>
>>>>>   --exclude-dirs data
>>>>
>>>> This is creative, but ultimately too unintuitive IMHO. Rsync is the only
>>>> tool which uses this syntax AFAIK, and it's a constant source of
>>>> confusion, especially when mixed with cp or similar commands.
>>>>
>>>
>>> Would this way be less intuitive?
>>> --exclude-path data/*
>>> --exclude-path data
>>>
>>> We can go even further with it allowing any level of directories:
>>> --exclude-path data/tmp/*
>>> --exclude-path data/db/tmp
>>> ...
>>
>> I agree, this is pretty unambiguous and easy to understand.
>>
>> But this raises the question: Should we go all the way and support
>> wildcards? Which might make it a bit complicated. Maybe support only
>> pure '*' for now?
> 
> As it shouldn't be hard to implement I'd go for it.

Note that in a shell, "data/*" doesn't include "data/.*". So this syntax
avoids the confusing rsync syntax in trade for another one. I'd prefer
the rsync flavour, because "behaves like rsync" is easier to remember
than "behaves like a shell glob, but differently". Confusion could be
prevented with good documentation, FWIW.

Regards,
Andreas