Re: [rdiff-backup-users] OverflowError: signed integer is greater than maximum

[Sai kee Wong]

Thanks for the reply.

Yes I did check the related pages.  And I tried to use the Python version
2.6 which I assume should not suffer from the problem.  But it does suffer.
At that time, I was using Mac OS 10.6 which is assumed to be 64 bits.
Until recently, test was done with Mac OS 10.8 with Python 2.7, still the
same problem.  So I made the fix.

B.Rgds

SK

On 21 Oct 2012, at 6:28 AM, Joe Steele wrote:

> It looks like you were encountering the python bug identified here:
> 
> http://bugs.python.org/issue1747858
> 
> Before their fix, the uid was being converted to a (32 bit) signed int, 
> which, as you have noted, isn't big enough.  After the fix, the uid is 
> converted to a long.  I don't know anything about Mac OS X, but on linux, a 
> long is either 32 bits or 64 bits, depending on the architecture.  Their fix 
> should obviously work on 64 bit systems.  (And on 32 bit systems, it's 
> probably unlikely that you will encounter 32 bit uids anyway.)
> 
> It also looks like their fix was implemented in python v2.5.3+, so I am a 
> little puzzled as to why you were experiencing problems with the versions of 
> python that you mention, if indeed you are using a 64 bit version.
> 
> In any case, it looks like you found a fix that works for you.
> 
> --Joe
> 
> 
> On 10/17/2012 9:43 AM, Sai kee Wong wrote:
>> After the orignal post, there is no feedback.  Recently, I tried to fix
>> the problem.  I didn't know any about Python, started debugging by inserting
>> print statement.  Figured out in the Mac, some times it has file with uid
>> and gid set to 4294967294 (nobody).  It seems Python doesn't accept unsigned
>> 32-bits.  And it crashed when calling the os.chown() with this high value.
>> 
>> I don't know how to make the os.chown() to accept unsigned int instead of
>> signed 32-bits int.  So I searched the web and add a function to change
>> too big value to -ve, in this case is -2.  Then it works.  As in previous
>> posts, somebodies asked similar questions and got no helpful response;
>> I published the solution here such that if anyone is suffering from the 
>> problem
>> can adopt the fix.
>> 
>> Also, please remind me if I am doing something wrong.
>> 
>> Thanks !
>> 
>> SK
>> 
>> Fix:
>> 
>> After installed the rdiff-backup 1.3.3, find the file rpath.py
>> In Mac, it is in
>> 
>> /Library/Python/2.7/site-packages/rdiff_backup/rpath.py
>> 
>> At line 977, the line was
>> 
>>                 else: os.chown(self.path, uid, gid)
>> 
>> change it to
>> 
>>                 else: os.chown(self.path, int32(uid), int32(gid))
>> 
>> At line 46, add following
>> 
>> def int32(x):
>>     if x>0xFFFFFFFF:
>>         raise OverflowError
>>     if x>0x7FFFFFFF:
>>         x=int(0x100000000-x)
>>         if x<2147483648:
>>             return -x
>>         else:
>>             return -2147483648
>>     return x
>> 
>> On 11 Apr 2011, at 9:04 AM, Sai kee Wong wrote:
>> 
>>> I'm new to the rdiff-backup
>>> 
>>> I'm running Mac OS 10.6.6, installed the rdiff-backup 1.3.3 and tried
>>> to backup 110GB of data, at around 60GB, it stopped with following error:
>>> 
>>> Exception 'signed integer is greater than maximum' raised of class '<type 
>>> 'exceptions.OverflowError'>':
>>>  File "/Library/Python/2.6/site-packages/rdiff_backup/Main.py", line 306, 
>>> in error_check_Main
>>>    try: Main(arglist)
>>>  File "/Library/Python/2.6/site-packages/rdiff_backup/Main.py", line 326, 
>>> in Main
>>>    take_action(rps)
>>>  File "/Library/Python/2.6/site-packages/rdiff_backup/Main.py", line 282, 
>>> in take_action
>>>    elif action == "backup": Backup(rps[0], rps[1])
>>>  File "/Library/Python/2.6/site-packages/rdiff_backup/Main.py", line 348, 
>>> in Backup
>>>    backup.Mirror(rpin, rpout)
>>>  File "/Library/Python/2.6/site-packages/rdiff_backup/backup.py", line 38, 
>>> in Mirror
>>>    DestS.patch(dest_rpath, source_diffiter)
>>>  File "/Library/Python/2.6/site-packages/rdiff_backup/backup.py", line 240, 
>>> in patch
>>>    ITR(diff.index, diff)
>>>  File "/Library/Python/2.6/site-packages/rdiff_backup/rorpiter.py", line 
>>> 281, in __call__
>>>    last_branch.fast_process(*args)
>>>  File "/Library/Python/2.6/site-packages/rdiff_backup/backup.py", line 533, 
>>> in fast_process
>>>    if self.patch_to_temp(mirror_rp, diff_rorp, tf):
>>>  File "/Library/Python/2.6/site-packages/rdiff_backup/backup.py", line 563, 
>>> in patch_to_temp
>>>    rpath.copy_attribs(diff_rorp, new)
>>>  File "/Library/Python/2.6/site-packages/rdiff_backup/rpath.py", line 180, 
>>> in copy_attribs
>>>    rpout.chown(*rpout.conn.user_group.map_rpath(rpin))
>>>  File "/Library/Python/2.6/site-packages/rdiff_backup/rpath.py", line 977, 
>>> in chown
>>>    else: os.chown(self.path, uid, gid)
>>> 
>>> Traceback (most recent call last):
>>>  File "/usr/local/bin/rdiff-backup", line 30, in <module>
>>>    rdiff_backup.Main.error_check_Main(sys.argv[1:])
>>>  File "/Library/Python/2.6/site-packages/rdiff_backup/Main.py", line 306, 
>>> in error_check_Main
>>>    try: Main(arglist)
>>>  File "/Library/Python/2.6/site-packages/rdiff_backup/Main.py", line 326, 
>>> in Main
>>>    take_action(rps)
>>>  File "/Library/Python/2.6/site-packages/rdiff_backup/Main.py", line 282, 
>>> in take_action
>>>    elif action == "backup": Backup(rps[0], rps[1])
>>>  File "/Library/Python/2.6/site-packages/rdiff_backup/Main.py", line 348, 
>>> in Backup
>>>    backup.Mirror(rpin, rpout)
>>>  File "/Library/Python/2.6/site-packages/rdiff_backup/backup.py", line 38, 
>>> in Mirror
>>>    DestS.patch(dest_rpath, source_diffiter)
>>>  File "/Library/Python/2.6/site-packages/rdiff_backup/backup.py", line 240, 
>>> in patch
>>>    ITR(diff.index, diff)
>>>  File "/Library/Python/2.6/site-packages/rdiff_backup/rorpiter.py", line 
>>> 281, in __call__
>>>    last_branch.fast_process(*args)
>>>  File "/Library/Python/2.6/site-packages/rdiff_backup/backup.py", line 533, 
>>> in fast_process
>>>    if self.patch_to_temp(mirror_rp, diff_rorp, tf):
>>>  File "/Library/Python/2.6/site-packages/rdiff_backup/backup.py", line 563, 
>>> in patch_to_temp
>>>    rpath.copy_attribs(diff_rorp, new)
>>>  File "/Library/Python/2.6/site-packages/rdiff_backup/rpath.py", line 180, 
>>> in copy_attribs
>>>    rpout.chown(*rpout.conn.user_group.map_rpath(rpin))
>>>  File "/Library/Python/2.6/site-packages/rdiff_backup/rpath.py", line 977, 
>>> in chown
>>>    else: os.chown(self.path, uid, gid)
>>> OverflowError: signed integer is greater than maximum
>>> 
>>> Believe it should be problem with the 64 bits OS. Tried python 2.6.1
>>> and 2.6.6 both give above error. Tried also 2.7.1 but gives
>>> Segmentation fault
>>> 
>>> Search through the rdiff-backup-users Archives but couldn't find
>>> any cue to solve the problem.
>>> 
>>> Thanks in advance for help.
>>> B.rgds
>>> 
>>> SK
>> 
>> 
>> _______________________________________________
>> rdiff-backup-users mailing list at address@hidden
>> https://lists.nongnu.org/mailman/listinfo/rdiff-backup-users
>> Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki
>> 
>>