[OE-core] Source Archiver Class

[Saul Wold]

This is clearly coming to a head and we need to a better definition of 
what we are doing and direction we are going.  We recently had the 2 
different implementation suggested on the list to accomplish WORKDIR 
archiving along with the existing sourcepkg, distribute_src, 
copyleft_compliance and other possibly others.

My understanding of what we are trying to tackle include a variety of 
legal and policy requirements.

This is a progression list of what the source archives should include:

1 - Original Upstream Archive & Patches
     - 2 archives (tarballs)
2 - Original Source code & Patches
     - could include additional code
     - post unpack
     - 1 archive
3 - Original Source code & Patches & temp (scripts & logs)
     - Could possibly include the .bb and .inc files
     - 2 archives (from #2 & temp tarball)
4 - Patched source code
     - original source code could be removed
     - post do_patch
5 - Patched source code & temp
     - 2 archives (from #4 & temp tarball)
6 - Configured Source
     - post do_configure
7 - SRPM format of Original Source & Patches
   - rpm will apply the patches
   - internally contains #2 above
8 - SRPM various of #3 above
...
100 - Buildable SRPM
   - can actually build, this is way future!

(Patches = patch files & series list)

Each of these build on the previous in some way, the key being that we 
generate a tarball for the source from the existing state of the 
WORKDIR, a challenge maybe to create the source snapshot after the build 
has already occurred.

The SPDX License info could also be included in any of these

After talking with Richard, we think we have a novel approach to make 
this work. It would entail using the postfuncs feature similar to the 
way that Shared State does it's work. The existing copyleft_compliance 
class functionality can be folded into this as a filter. Additional 
flags could be passed from individual classes to a core set of methods 
in an archiver class defining the type of data (source, patches, temp, 
env, recipe info, ...) and format (tar, sprm, ...)

I noted that recipe type code might be better suited as generic code, as 
I believe there are other places that could benefit from that code.

The sourcepkg class seems to be a basic archiver and differ that 
includes the metadata/environment (as dumpdata), this could be replaced 
by the new approach.  While the src_distribute class copies the 
downloaded archive and then creates a link, into LICENSE directories 
along with the patches. The src_distribute by default seems to move 
files and create links (incorrectly btw!). This work can be done by the 
archiver class.

The Nugget: Create a new core "archiver" class that implements a general 
functions that can archive the original tarball or workdir at various 
states along task list with additional metadata (recipe info, temp dir, 
environment). This class would be inherited by a set of classes that use 
the postfunc (similar to sstate) that setup what level of archive is 
needed (based on the list above).

Thoughts, Comments?

Thanks



-- 
     Sau!

Saul Wold
Yocto Component Wrangler @ Intel
Yocto Project / Poky Build System