[bitbake-devel] [bitbake-devel, 1.42, 1.40, v2] git.py: Handle space in git repo name for http/https protocol

[Khem Raj]

On 8/3/19 2:34 PM, Ankur Tyagi wrote:
> Hi Richard,
> 
> Any input on how it should be handled? I have identified the area like 
> you asked and some ways to fix it as well
> 
> Thanks
> Ankur
> 
> On Sun, 28 Jul 2019, 9:15 p.m. Ankur Tyagi, <ankur.tyagi85 at gmail.com 
> <mailto:ankur.tyagi85 at gmail.com>> wrote:
> 
>     How about something like this:
> 
>     diff --git a/lib/bb/fetch2/__init__.py b/lib/bb/fetch2/__init__.py
>     index f6b5529b..fa1f9e27 100644
>     --- a/lib/bb/fetch2/__init__.py
>     +++ b/lib/bb/fetch2/__init__.py
>     @@ -380,6 +380,9 @@ def decodeurl(url):
>               user = ''
>               pswd = ''
> 
>     +    if type not in ['git']:
>     +        path = urllib.parse.unquote(path)
>     +
>           p = collections.OrderedDict()
>           if parm:
>               for s in parm.split(';'):
>     @@ -389,7 +392,7 @@ def decodeurl(url):
>                       s1, s2 = s.split('=')
>                       p[s1] = s2
> 
>     -    return type, host, urllib.parse.unquote(path), user, pswd, p
>     +    return type, host, path, user, pswd, p

It would be good to add some self test for this case as well and patch 
needs to be formatted as per 
https://www.openembedded.org/wiki/Commit_Patch_Message_Guidelines

> 
>       def encodeurl(decoded):
>           """Encodes a URL from tokens (scheme, network location, path,
> 
>     regards
>     Ankur
> 
>     On Fri, Jul 26, 2019 at 3:13 PM Ankur Tyagi <ankur.tyagi85 at gmail.com
>     <mailto:ankur.tyagi85 at gmail.com>> wrote:
>      >
>      > Hi,
>      >
>      > Previous idea is stupid, please discard that.
>      > Just checked quote/unquote were added as part of commit
>      > b1dbc24ebcc4e5100c32568c2c41fd982fb4bcce
>      >
>      > So explaining the situation again --> when fecthing from repo url
>      > https://tfs.local/repos/foo%bar
>      >
>      > During FetchData initialization, decodeurl(d.expand(url)) returns
>     path
>      > with %20 stripped
>      >
>      > [log][__init__][decodeurl] Path returned /repos/foo bar
>      >
>      > And when git method urldata_init() is called, it sets gitsrcname with
>      > space instead of %20
>      >
>      > [log][git][urldata_init] gitsrcname is tfs.local.repos.foo bar
>      >
>      > And when git method _get_repo_url is called, it always returns Path
>      > with space instead of %20
>      >
>      > [log][git][_get_repo_url] Path returned /repos/foo bar
>      >
>      > So I am bit confused now on how to handle it properly as unquote is
>      > removing %20 and it cannot be added back in later stage.
>      >
>      > Regards
>      > Ankur
>      >
>      >
>      > On Thu, Jul 25, 2019 at 7:23 PM Ankur Tyagi
>     <ankur.tyagi85 at gmail.com <mailto:ankur.tyagi85 at gmail.com>> wrote:
>      > >
>      > > Hi,
>      > >
>      > > Sorry for the late reply but I have figured out where %20 is
>     being stripped out.
>      > >
>      > > File lib/bb/fetch2/__init__.py, method decodeurl(url) is returning
>      > > path as urllib.parse.unquote(path) which strips out %20
>      > > So if I return path as it is, then %20 is preserved and works
>     as expected.
>      > >
>      > > diff --git a/lib/bb/fetch2/__init__.py b/lib/bb/fetch2/__init__.py
>      > > index f6b5529b..97fd59e1 100644
>      > > --- a/lib/bb/fetch2/__init__.py
>      > > +++ b/lib/bb/fetch2/__init__.py
>      > > @@ -389,7 +389,7 @@ def decodeurl(url):
>      > >                  s1, s2 = s.split('=')
>      > >                  p[s1] = s2
>      > >
>      > > -    return type, host, urllib.parse.unquote(path), user, pswd, p
>      > > +    return type, host, path, user, pswd, p
>      > >
>      > >
>      > > Is above change acceptable?
>      > >
>      > > Regards
>      > > Ankur
>      > >
>      > > On Wed, Jul 10, 2019 at 4:33 AM Richard Purdie
>      > > <richard.purdie at linuxfoundation.org
>     <mailto:richard.purdie at linuxfoundation.org>> wrote:
>      > > >
>      > > > On Wed, 2019-07-10 at 21:26 +1200, Ankur Tyagi wrote:
>      > > > > Hi,
>      > > > >
>      > > > > Is there something else that needs to be done in this patch? Or
>      > > > > perhaps there is a better to handle this?
>      > > > > I am willing to learn and try it out if original patch is
>     not good.
>      > > >
>      > > > I think Mark is right, we need to figure out where the %20
>     characters
>      > > > are being stripped out and preserve them rather than
>     injecting them
>      > > > back in again.
>      > > >
>      > > > The question is therefore where they're being stripped out in
>     the code?
>      > > >
>      > > > Cheers,
>      > > >
>      > > > Richard
>      > > >
> 
>