[OE-core] [PATCH 1/1] rpm: make install with --nosignature and --nodigest work

[Robert Yang]

On 09/21/2016 10:10 PM, Mark Hatle wrote:
> On 9/20/16 10:37 PM, Robert Yang wrote:
>>
>>
>> On 09/21/2016 04:33 AM, Mark Hatle wrote:
>>> On 9/20/16 10:00 AM, Burton, Ross wrote:
>>>>
>>>> On 20 September 2016 at 09:15, Hongxu Jia <hongxu.jia at windriver.com
>>>> <mailto:hongxu.jia at windriver.com>> wrote:
>>>>
>>>>     -Upstream-Status: Submitted [Sent email to rpm-devel at rpm5.org
>>>>     <mailto:rpm-devel at rpm5.org>]
>>>>     +Upstream-Status: Rejected [Sent email to rpm-devel at rpm5.org
>>>>     <mailto:rpm-devel at rpm5.org>]
>>>>     +http://rpm5.org/community/rpm-devel/5655.html
>>>>     <http://rpm5.org/community/rpm-devel/5655.html>
>>>>
>>>>
>>>> Considering upstream has explicitly rejected this patch, why should we accept it?
>>>>
>>>> Ross
>>>>
>>>>
>>>
>>> I'm confused by what the patch is doing looking at it.
>>>
>>> It sounds like from the description there is a bug that without the change,
>>> packages with (intentionally) bad checksums and such are allowed to be installed.
>>>
>>> The bug is caused by a previous patch that enabled nosignature, etc -- because
>>> the comparisons turned out to be backwards.
>>>
>>> So really nosignature, etc is already in place -- it's just not working properly?
>>>
>>> What was rejected upstream is the use of nosignature in any context.  RPM5
>>> maintainer believes it is unwise and unsafe to permit uses to install packages
>>> that have failed basic validation.  (I tend to agree.)  Similarly, even being
>>> able to run queries and other operations against them may be dangerous as well.
>>
>> If command like "rpm5 -qp --nosignature --nodigest <file.rpm>" queries database,
>> then it would cause rpm5 hang when more than one "rpm5 -qp" is running, so I
>> fixed the *query* problem, and the *install* problem is not caused by the fix.
>> Btw., *rpm4* doesn't query database when "rpm -qp file.rpm", if we are going
>> to use dnf in next release, maybe we can consider using rpm4 as Fedora does.
>> I did a rough statistics for oe-core's local patches, the winner is rpm, it
>> has 77 local patches, that's not good for maintaining, and you can imagine that
>
> 30 patches are simply configuration changes.
>
> 42 patches are bug fixes that have been submitted and will likely go away in the
> next uprev.
>
>> how many times it surprised us. rpm4 should be more stable than rpm5, I don't
>> know how many distros use rpm5, I can't access rpm5.org atm, it seems that the
>> web is down (It was OK yesterday), but widely used distros like Redhat and Suse
>
> The website is working for me today.
>
>> uses rpm4, so maybe rpm4 will make our life much more easier, and also for
>> yocto's user. I think that why we need rpm5 before was because we need use it
>
> There is a major problem with rpm4.  It doesn't support cross compiling at all.
> We know in the past there have been significant problems with it and BerkleyDB
> with endian issues.  Also it was not possible to do cross-architecture installs
> (in the past, this one might be resolved.)
>
> The general functionality between RPM4 and RPM5 is nearly identical.  RPM 4 has
> a plug-in interface (which I personally don't believe should be used due to
> security issues.)  RPM 5 has a focus around specific security enhancements.
> Some are reasonable, some are a bit too experimental to be used.
>
> The other main difference is software license.  RPM 4 is 'GPLv2', while RPM5 is
> 'LGPLv2'.  This doesn't sound like a big deal at first -- but RPM 4 can't be
> linked to OpenSSL for it's crypto library.  RPM 4 is linked ONLY to NSS, while
> RPM5 can use beecrypt, nss, openssl and others.  This makes for a significantly
> more flexible embedded system.
>
>> to compute the package dependencies, but now we have smartpm, so we don't rely
>> on that any more.
>>
>> Here is the recipe list which has more than 10 local patches, and we have 1762
>> patches atm:
>>       77 rpm
>
> I'd think it's a better comparison to say '30', the configuration items.  So
> it's similar to perl or openssl in complexity.  I'd agree that is a fair comparison.
>
>>       59 gcc-5.4
>>       49 gcc-6.2
>>       36 ltp
>>       31 python3
>>       30 perl
>>       29 openssl
>>       28 man
>>       27 glibc
>>       24 python
>>       23 tcp-wrappers-7.6
>>       23 systemd
>>       18 busybox
>>       14 syslinux
>>       14 python-smartpm
>>       14 elfutils-0.166
>>       14 apt
>>       13 qemu
>>       13 libtool
>>       13 gstreamer1.0-plugins-bad
>>       13 glib-2.0
>>       13 binutils
>>       13 bind
>>       12 coreutils-6.9
>>       11 valgrind
>>       11 gdb
>>       11 dhcp
>>       11 autoconf
>>       10 unzip
>>       10 dpkg
>>       10 console-tools-0.3.2
>>
>>>
>>> If fixing the problem is as simple as reverting the other change -- and that
>>> doesn't cause other problems elsewhere...  I'd rather see that.
>>
>> We can't revert the patch, it would break packagefeed-stability.bbclass,
>> we will see the hang
>
> Something is wrong with the implementation then.
>
> You should not be doing the equivalent of:
>
> list = '<all packages>'
> for each in list; do rpm -qp $each ; done
>
> This is HORRIBLY inefficient, even with rpm4.
>
> The proper way to query a number of packages is in a single transaction.  This
> permits RPM to load the data and validation structures once.
>
> This is equivalent to:
>   rpm -qp `cat list`
>
> or
>
>   echo list > /tmp/tmpfile
>   rpm -qp /tmp/tmpfile

pkg-diff.sh from build-compare can only use two rpm pkgs as arguments:

pkg-diff.sh rpm1 rpm2

So we use "for each in list; do pkg-diff.sh each1 each2; done". We need
change pkg-diff.sh to make it accept files as arguments. And it works well
with rpm4, so I'm not sure whether the upstream can accept such a change
or not. The hang reason for rpm5 is that it reads database (/var/lib/rpm)
when not needed, so I think that fix rpm5 is the correct way.

// Robert



>
> (The later solving the problem of 'too many arguments')
>
> This may seem counter intuitive, as the first way looks to be better on a
> multi-core machine -- but in my experience the overhead of starting rpm, setting
> up the data structures is very expensive, much more then the
> load/validate/display of the package information.
>
> --Mark
>
>>
>>>
>>> --Mark
>>>
>
>