[OE-core] Status of the NPM refactoring

[Jean-Marie LEMETAYER]

Hi Andre,

On Dec 12, 2019, at 9:17 PM, André Draszik git at andred.net wrote:
> Hi,
> 
> On Thu, 2019-12-12 at 07:49 -0500, Jean-Marie LEMETAYER wrote:
>> Hi folks,
>> 
>> I am currently trying to update/refactor the handling of the NPM packages.
>> [...]
>> Is it OK ? Any thought ? Any advice ?
> 
> Three questions occurred to me on that whole subject... Mostly based on the
> understanding that there are no recipes for
> the dependencies, only for the
> main package.
> 
> Firstly:
> With your approach, can I force a different version of a dependency to be
> fetched? I.e. a versions different from what npm would do?
> 
> Use case:
> Let's say an npm package A somewhere down in the chain of my own (recursive)
> dependencies pulls in version 1.2 of package B, but version 1.2 is unsuitable
> for use, e.g. licensing issue. An updated, compatible, but yet unreleased on
> npmjs.org version of package B exists (e.g. in github), with the licensing
> fixed.
> 
> Can I pull in that alternative version using your approach, as
> I believe there is no recipe for package C if I understand right, i.e. no
> place to put the alternative URL.

Yes you can ! Just update manually the shrinkwrap file and use the npm
formatting [1].

1: https://docs.npmjs.com/files/package-lock.json#dependencies

For example you will be able to replace this entry:

    "array-flatten": {
      "version": "1.1.1",
      "resolved": "https://registry.npmjs.org/array-flatten/-/array-flatten-1.1.1.tgz",
      "integrity": "sha512-PCVAQswWemu6UdxsDFFX/+gVeYqKAod3D3UVm91jHwynguOwAvYPhx8nNlM++NqRcK6CxxpUafjmhIdKiHibqg=="
    }

By this one:

    "array-flatten": {
      "version": "git+https://github.com/blakeembrey/array-flatten.git#66299a02d2ce6a9b6998be333581a87affbb8631"
    }

Note that the first case will use the wget fether and the second case
will use the git fetcher.

> Secondly:
> Say I need to patch an npm package C somewhere down in the chain of my own
> (recursive) dependencies to fix an issue with package C. Can I do that, as
> again, I believe there is no recipe for package C where I could update
> SRC_URI to include my patch.

Yes you can ! After the fetch tasks the dependencies will be available in
the ${S} directory (this is mandatory for the license management). The
dependencies can be patched as usual. Note that you have to use the full
path of the dependency ("/node_modules/array-flatten/...").

For example your patch could look like this:

    diff --git a/node_modules/array-flatten/array-flatten.js b/node_modules/array-flatten/array-flatten.js
    index 089117b..1b83b82 100644
    --- a/node_modules/array-flatten/array-flatten.js
    +++ b/node_modules/array-flatten/array-flatten.js
    @@ -60,5 +60,6 @@ function arrayFlatten (array, depth) {
         return flattenForever(array, [])
       }
     
    +  console.log("Hello World");
       return flattenWithDepth(array, [], depth)
     }
    --

> Thirdly:
> Will it be possible to handle projects that use angular natively in yocto
> using your approach. E.g. have an angular-native recipe that will allow
> running ng build on the code I'm interested in?

Yes, the ultimate goal of this update / refactoring is to add an angular
support in Yocto. I have already done this work for my project.

The final recipe could look like this:

    SRC_URI = "npm://registry.npmjs.org;name=my-package;version=${PV} \
               npmsw://${THISDIR}/${BPN}/npm-shrinkwrap.json"

    S = "${WORKDIR}/npm"

    inherit angular


Kind regards,
Jean-Marie