[OE-core] Contents of non-rootfs partitions

Wed Nov 23 10:12:35 UTC 2016

On 11/23/2016 10:24 AM, Maciej Borzęcki wrote:
> On Wed, Nov 23, 2016 at 9:40 AM, Enrico Joerns <ejo at pengutronix.de> wrote:
>> On 11/22/2016 12:54 PM, Kristian Amlie wrote:
>>>
>>> On 22/11/16 12:10, Patrick Ohly wrote:
>>>>>
>>>>> ...
>>>>
>>>>
>>>> All of these introduce some special mechanism. Let me propose something
>>>> that might integrate better with the existing tooling:
>>>>
>>>> The "rootfs" directory gets redefined as representing the entire virtual
>>>> file system. When creating a disk image, it gets split up into different
>>>> partitions based on the image configuration.
>>>>
>>>> For example, the /home or /data directories in the rootfs could hold the
>>>> content that in some image configurations goes into separate partitions.
>>>>
>>>> The advantage of this approach is that the tooling for staging content
>>>> for image creation does not need to be changed. The same staged content
>>>> then can be used to create different images, potentially even using
>>>> different partition layouts.
>>>
>>>
>>> That's a very good idea. I think it beats all of my suggestions!
>>
>>
>> I totally agree with this solution, because it is the same approach we are
>> already using for several years.
>>
>> Instead of wic we are using the genimage tool
>> (http://public.pengutronix.de/software/genimage/) together with a Yocto
>> class for easing image creation, but its pretty much the same concerning its
>> base purpose.
>>
>> We called the option to split up 'mountpoint' while assuming that the
>> splitted partition will be mounted to the same location in the file system
>> it was taken from.
>>
>> An example configuration would look like this (stripped down to the
>> essential parts):
>>
>>   image my-disk-image {
>>           [...]
>>           partition root {
>>                   image = "rootfs.ext4"
>>                   [...]
>>           }
>>
>>           partition home {
>>                   image = "home.ext4"
>>                   [...]
>>           }
>>   }
>>
>>   image rootfs.ext4 {
>>           [...]
>>           mountpoint = "/"
>>   }
>>
>>   image home.ext4 {
>>           [...]
>>           mountpoint = "/home"
>>   }
>>
>> Maybe a similar approach could be used in wic, too.
>
> I went through the README from genimage repo
> https://git.pengutronix.de/cgit/genimage/tree/README. Definitely looks
> interesting and covers storage medias that are not supported by wic
> right now. The split option does not seem to be documented though. If I
> understood you correctly, the partition is seeded with data from
> location specified by `mountpoint` inside IMAGE_ROOTFS. Is that correct?

Yes, what the genimage class (see below) does, is unpacking the final 
rootfs.tar.xz inside a fakeroot environment and call the genimage tool 
with the --rootpath argument set to this path.

The mountpoint="/home" tells genimage to pack all content in directory 
/home into a separate data ext4 image (temporary named 'home.ext4') and 
all other content in a rootfs ext4 image (temporary named 'rootfs.ext4').

The `image` section then specifies how to construct the final disk image 
using the referenced generated rootfs and datafs images (and optionally 
some bootloader, non-partition-data, etc.).

Note that this is only one way to use genimage. Another common way to 
use it is to simply construct complex disk images out of different 
filesystem images, e.g. with

   partition barebox {
           image = "barebox-any-platform.img"
           size = 1M
           in-partition-table = false
   }

> I see there's a large overlap in functionality with wic (at least for
> block devices), so perhaps the best way would be introduce genimage as
> another IMAGE_CMD.
>
> I've quickly checked at https://layers.openembedded.org and there are no
> recipes for genimage/libconfuse listed there. Do happen to have done
> integration with OE in some private repo already? Would it be possible
> for you to post the relevant patches?
>

We have a layer for this, which is potentially publicly available but 
not advertised as the general plan was to move some of its content to 
upstream (oe-core, meta-oe) and split up other content (e.g. the rauc 
update tool stuff) to separate layers that will be made publicly 
available. But, as always, there are more plans than time left ;)

Well, to make a long story short, the path to the git repository is:

   https://git.pengutronix.de/cgit/meta-ptx

The genimage class I mentioned above you can find in it, too:

   https://git.pengutronix.de/cgit/meta-ptx/tree/classes/genimage.bbclass

When starting with Yocto I thought about changing from genimage to wic 
for disk image creation as wic already seemed to become the standard 
solution for building disk images. But back in these days it was a very 
separate tool and there was no integration with the BSP itself (which is 
crucial when having to reproduce builds). But the main reason why I 
decided to stay with genimage was that wic simply did not cover most of 
the use cases I required for building complex disk images for different 
embedded platforms. I'm pretty sure that this has changed in some 
points, but had no time to get deeper into this topic again, yet.

Hope that helps, best regards

Enrico

-- 
Pengutronix e.K.                           | Enrico Jörns                |
Industrial Linux Solutions                 | http://www.pengutronix.de/  |
Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-5080 |
Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |