Re: [Ltib] PKG_KERNEL_LEAVSRC issues...

[Stuart Hughes]

Hi Peter,

Sorry for the long delay getting back to you.

I re-read your original email:
http://lists.nongnu.org/archive/html/ltib/2010-12/msg00115.html
and properly comprehended what you were saying (I must have been tired).
Anyhow, I've checked-in a change pretty much as you described in that
email.  I've done some testing and it seems to work okay.  Please check
out the latest and let me know if this is what you intended.

Regards, Stuart

BTW: whoever maintains: dist/lfs-5.1/apex/apex-common.tmpl
I don't think you need the conditional if [ -n "$PKG_APEX_LEAVESRC" ] in
the '%clean' section.  Just the normal rm -rf $RPM_BUILD_ROOT.

On 21/12/10 21:16, Peter Barada wrote:
> On 12/21/2010 03:56 PM, Stuart Hughes wrote:
>> Hi Peter,
>>
>> Peter Barada wrote:
>>> On 12/20/2010 03:24 PM, Stuart Hughes wrote:
>>>> Hi Peter,
>>>>
>>>> See inline:
>>>>
>>>> Peter Barada wrote:
>>>>> On 12/20/2010 04:35 AM, address@hidden wrote:
>>>>>> Hi Peter,
>>>>>>
>>>>>> I'm going to need to re-read this email as at first read I'm not
>>>>>> exactly
>>>>>> sure what you are running into.  If possible can you reduce it to 2
>>>>>> sentences of; "I expected this... but observe this...."
>>>>> Ok, to clear things up, I expect if I remove rpm/RPMS/arm/kernel*.rpm
>>>>> and everything in rpm/BUILD that "./ltib" will "do the right thing",
>>>>> namely %prep, %build but not %clean the kernel if I have a package
>>>>> enabled that selects PKG_KERNEL_LEAVESRC.  Since PKG_*_LEAVESRC is a
>>>>> transient config it fails when I try to build helloworld_mod.
>>>> Okay I can see what you mean, that's a bug/design flaw.
>>>>
>>>> I think a simple fix maybe to just make LEAVESRC non-transient, but I
>>>> worry about some unwanted side-effect.  I have a vague recollection
>>>> that
>>>> I was going to do this at one point and backed out.
>>>>
>>>> Maybe for now you could try for your use cases by deleting line 3149:
>>>>
>>>>           s,^(\w+LEAVESRC)=y,# $1 is not set,;
>>>>
>>> That would change the transient case - have to try to know for sure if
>>> that alone will fix what I'm seeing.
>> Try it and see if it does.
>>
>> On another note, I was trying to recall and I think the reason I made
>> LEAVESRC transient was so that if someone checked in a defconfig it
>> didn't cause a kernel rebuild on first install.  It's a long time ago
>> now, so I'm a bit vague.  If this is right, another alternative would be
>> to removed these when loading any defconfig.
> But if that is done, then unless a configuration step is done to see
> that PGK_HELLOWORLD_MOD is enabled which selects PKG_KERNEL_LEAVESRC,
> then ltib won't know that the kernel source needs to not be cleaned, right?
> 
>>
>>>>>> Just to shed some light on the design, normally .config will override
>>>>>> defconfig, however if you touch defconfig ltib will notice this and
>>>>>> use
>>>>>> the updated defconfig.  The purpose of this is so that if you have an
>>>>>> autobuilder and check-in an update to defconfig, the autobuilder
>>>>>> will use
>>>>>> the updated defconfig.
>>>>> In this case I have not touched any of the config files...
>>>> No, but I wanted you to understand that touching the defconfig file
>>>> will
>>>> have this effect in case you run into it or want to know how to trigger
>>>> this behaviour.
>>> Thanks - possibly useful but not sure, kinda like a hidden switch.
>> Overall practice showed this to be the best compromise.  So this is a
>> case where opinions may differ, but this behaviour works best IMHO.
> Pardon me, on rereading you're right, touching defconfig should make it
> take precedence over defconfig.dev and .config.  I must have been
> confused with something else.
> 
>>>>>> Regarding "LEAVESRC" the idea is that this only needs to be
>>>>>> transient as
>>>>>> once a package is left unpacked, it will never be removed and so on
>>>>>> subsequence builds the same sources remain there and just get
>>>>>> re-built.
>>>>> True, but being "paranoid" especially if I'm mucking with the
>>>>> config(s)
>>>>> and package spec files, its easiest to force multiple package rebuilds
>>>>> by deleting their resultant .rpm and rpm/BUILD directories and
>>>>> executing
>>>>> "./ltib".  Also my buildbot leaves its universe alone (i.e. all it
>>>>> does
>>>>> is "svn update; ./ltib -m clean; ./ltib -b --preconfig
>>>>> config/platform/<platform>/defconfig").
>>>>>
>>>>> I'm sure I'm missing something - how can I tell LTIB to remove a
>>>>> package's source (that's left around due to LEAVESRC) - if its
>>>>> exists in
>>>>> rpm/BUILD, ltib cowardly refuses to remove it even if I update its
>>>>> .spec
>>>>> file. So I added a "rm -rf rpm/BUILD/* to the buildbot steps and ended
>>>>> up w/o kernel source and helloworld_mod failing.
>>>>>
>>>> The idea is once a package source is unpacked the only way it will get
>>>> removed is manually (or by some external autobuild script).  The reason
>>>> is that you may be working on that source (e.g. developing a kernel
>>>> driver), the last thing you want is to lose those changes.  Touching
>>>> the
>>>>    .spec file just means that you want a rebuild, not that any sources
>>>> need unpacking (unless they are not there already).
>>> In my case where I have buildbot running on a slave that I can't log
>>> into, if the kernel was left unpacked (because helloworld_mod needed
>>> it), if I add a patch to the kernel, in the next build after an update
>>> from my SVN, LTIB realizes *something* needs to be done but doesn't
>>> remove the offending source in rpm/BUILD.  So I'm caught in a
>>> problem.  If I blow away all the source in rpm/BUILD, LTIB doesn't
>>> repopulate(or enough to build helloworld_mod).  If I leave the sources
>>> in rpm/BUILD as is, then LTIB can't pick up new patches to any package
>>> %prep'd in rpm/BUILD.  NEEDSRC is a way to fix this case.  I agree
>>> that LEAVESRC is useful for transient "hey, leave this around so I can
>>> look", but because its transient, .config doesn't have it left on
>>> (i.e. its cleared out of .config, left in defconfig.dev) and unless
>>> defconfig is explicitly used the behavior is inconsistent.
>>>
>> See earlier; I think the transient nature can be removed and put into
>> the section that parses the defconfig.  I think this would fix your
>> problem as I believe on a fresh install, the static LEAVESRC=y would get
>> removed, however when run through mconf (conf) the select in the kernel
>> module would turn it on for you and unpack the sources.
> Is mconf invoked in the -b case?  If not then I can see things being out
> of kilter.
>> BTW: if you're actually doing kernel development, it's probably better
>> to use a directory build and point at your kernel sources.  This way
>> your external autobuild scripts can do a "git pull" before running LTIB
>> to get the kernel source tree updated.
> In my world I do kernel (and some package development) using SVN trees
> and appropriate .spec files, but then diff the kernel tree an create a
> patch for SVN using a .spec.in file to build it.  Its a bit messy but is
> needed since I use the buildslave to check that the LTIB tree I pull out
> has everything necessary to duplicate my internal SVN kernel/package
> trees (and this LTIB tree becomes what gets distributed to customers). 
> Since the build can take quite from scratch I want to limit the building
> on the slave to the minimum to allow it to keep up with checkins as part
> of continuous integration testing.
>> Can you think this through/try it out (by removing that line) and let me
>> know how it goes.  Depending how that goes, and time permitting I'll try
>> to get a fix in for this.
> Sure, will do, I'll get back to you.  Thanks!
>> Regards, Stuart
>>
> 
>