bug-wget
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Bug-wget] [PATCH 1/3] Added tests for HTTP authentication using cre


From: Tim Rühsen
Subject: Re: [Bug-wget] [PATCH 1/3] Added tests for HTTP authentication using credentials from .netrc
Date: Mon, 15 May 2017 17:04:07 +0200
User-agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.8.0

Hi Tomas,

my locale looks similar:

$ locale
LANG=en_US.UTF-8
LANGUAGE=en_US.UTF-8
LC_CTYPE="en_US.UTF-8"
LC_NUMERIC="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_COLLATE="en_US.UTF-8"
LC_MONETARY="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_PAPER="en_US.UTF-8"
LC_NAME="en_US.UTF-8"
LC_ADDRESS="en_US.UTF-8"
LC_TELEPHONE="en_US.UTF-8"
LC_MEASUREMENT="en_US.UTF-8"
LC_IDENTIFICATION="en_US.UTF-8"
LC_ALL=en_US.UTF-8


Reproducable here *without* your patches:

$ TESTS_ENVIRONMENT="env -i; " make check


The system is Debian unstable amd64.
I asked Darshit, the author of the python script, if he could take a
look... maybe this is something trivial for a Pythonista.

The traceback is:

Unhandled exception caught.
'ascii' codec can't decode byte 0xc1 in position 18: ordinal not in
range(128)
Traceback (most recent call last):
  File "./Test-metalink-http.py", line 124, in <module>
    err = http_test.begin ()
  File "/usr/oms/src/wget1.x/testenv/test/http_test.py", line 41, in begin
    self.do_test()
  File "/usr/oms/src/wget1.x/testenv/test/base_test.py", line 187, in
do_test
    self.post_hook_call()
  File "/usr/oms/src/wget1.x/testenv/test/base_test.py", line 206, in
post_hook_call
    self.hook_call(self.post_configs, 'Post Test Function')
  File "/usr/oms/src/wget1.x/testenv/test/base_test.py", line 196, in
hook_call
    conf.find_conf(conf_name)(conf_arg)(self)
  File "/usr/oms/src/wget1.x/testenv/conf/expected_files.py", line 36,
in __call__
    local_fs = self.gen_local_fs_snapshot()
  File "/usr/oms/src/wget1.x/testenv/conf/expected_files.py", line 30,
in gen_local_fs_snapshot
    f['content'] = fp.read()
  File "/usr/lib/python3.5/encodings/ascii.py", line 26, in decode
    return codecs.ascii_decode(input, self.errors)[0]
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc1 in position 18:
ordinal not in range(128)


With Best Regards, Tim



On 05/15/2017 04:53 PM, Tomas Hozza wrote:
> Hello Tim.
> 
> I run all the tests before sending patches and they all passed. If you can 
> send me more info about the failures, I could have a look at it.
> 
> My locale is:
> $ locale
> LANG=en_US.UTF-8
> LC_CTYPE="en_US.UTF-8"
> LC_NUMERIC=sk_SK.UTF-8
> LC_TIME=sk_SK.UTF-8
> LC_COLLATE="en_US.UTF-8"
> LC_MONETARY=sk_SK.UTF-8
> LC_MESSAGES="en_US.UTF-8"
> LC_PAPER=sk_SK.UTF-8
> LC_NAME="en_US.UTF-8"
> LC_ADDRESS="en_US.UTF-8"
> LC_TELEPHONE="en_US.UTF-8"
> LC_MEASUREMENT=sk_SK.UTF-8
> LC_IDENTIFICATION="en_US.UTF-8"
> LC_ALL=
> 
> Thanks.
> 
> Regards,
> Tomas
> 
> On 15.05.2017 15:52, Tim Rühsen wrote:
>> Hi Tomas,
>>
>> thank you very much for the fixes and the new tests !
>>
>> Currently, patch 1/3 breaks two Metalink tests:
>>
>> FAIL: Test-metalink-http.py
>> FAIL: Test-metalink-http-quoted.py
>>
>> I am still investigating, but it has something to do with the
>> environment that you reset in testenv/test/base_test.py:
>>
>>     ret_code = call(params, env={"HOME": os.getcwd()})
>>
>> Maybe a bug that just comes out with a different locale/charset encoding.
>>
>>
>> With Best Regards, Tim
>>
>>
>>
>> On 05/12/2017 07:17 PM, Tomas Hozza wrote:
>>> Getting credentials from .netrc has been broken from time to time, thus
>>> adding a test coverage to prevent regressions.
>>>
>>> Also added setting of "HOME" environment variable when executing wget,
>>> to make sure LocalFiles like .netrc, which are created just for the
>>> test, are actually used.
>>>
>>> Signed-off-by: Tomas Hozza <address@hidden>
>>> ---
>>>  testenv/Makefile.am                         |  3 ++
>>>  testenv/Test-auth-basic-netrc-pass-given.py | 68 
>>> +++++++++++++++++++++++++++++
>>>  testenv/Test-auth-basic-netrc-user-given.py | 68 
>>> +++++++++++++++++++++++++++++
>>>  testenv/Test-auth-basic-netrc.py            | 66 
>>> ++++++++++++++++++++++++++++
>>>  testenv/test/base_test.py                   |  2 +-
>>>  5 files changed, 206 insertions(+), 1 deletion(-)
>>>  create mode 100755 testenv/Test-auth-basic-netrc-pass-given.py
>>>  create mode 100755 testenv/Test-auth-basic-netrc-user-given.py
>>>  create mode 100755 testenv/Test-auth-basic-netrc.py
>>>
>>> diff --git a/testenv/Makefile.am b/testenv/Makefile.am
>>> index 3febec7..7104314 100644
>>> --- a/testenv/Makefile.am
>>> +++ b/testenv/Makefile.am
>>> @@ -75,6 +75,9 @@ if HAVE_PYTHON3
>>>    TESTS = Test-504.py                               \
>>>      Test-auth-basic-fail.py                         \
>>>      Test-auth-basic.py                              \
>>> +    Test-auth-basic-netrc.py                        \
>>> +    Test-auth-basic-netrc-user-given.py             \
>>> +    Test-auth-basic-netrc-pass-given.py             \
>>>      Test-auth-both.py                               \
>>>      Test-auth-digest.py                             \
>>>      Test-auth-no-challenge.py                       \
>>> diff --git a/testenv/Test-auth-basic-netrc-pass-given.py 
>>> b/testenv/Test-auth-basic-netrc-pass-given.py
>>> new file mode 100755
>>> index 0000000..43dfe34
>>> --- /dev/null
>>> +++ b/testenv/Test-auth-basic-netrc-pass-given.py
>>> @@ -0,0 +1,68 @@
>>> +#!/usr/bin/env python3
>>> +from sys import exit
>>> +from test.http_test import HTTPTest
>>> +from misc.wget_file import WgetFile
>>> +
>>> +"""
>>> +    This test ensures Wget uses credentials from .netrc for Basic 
>>> Authorization Negotiation.
>>> +    In this case we test that .netrc credentials are used in case only
>>> +    password is given on the command line.
>>> +    Also, we ensure that Wget saves the host after a successful auth and
>>> +    doesn't wait for a challenge the second time.
>>> +"""
>>> +############# File Definitions 
>>> ###############################################
>>> +File1 = "I am an invisble man."
>>> +File2 = "I too am an invisible man."
>>> +
>>> +User = "Sauron"
>>> +Password = "TheEye"
>>> +
>>> +File1_rules = {
>>> +    "Authentication"    : {
>>> +        "Type"          : "Basic",
>>> +        "User"          : User,
>>> +        "Pass"          : Password
>>> +    }
>>> +}
>>> +File2_rules = {
>>> +    "ExpectHeader"      : {
>>> +        "Authorization" : "Basic U2F1cm9uOlRoZUV5ZQ=="
>>> +    }
>>> +}
>>> +
>>> +Netrc = "machine 127.0.0.1\n\tlogin {0}".format(User)
>>> +
>>> +A_File = WgetFile ("File1", File1, rules=File1_rules)
>>> +B_File = WgetFile ("File2", File2, rules=File2_rules)
>>> +Netrc_File = WgetFile (".netrc", Netrc)
>>> +
>>> +WGET_OPTIONS = "--password={0}".format(Password)
>>> +WGET_URLS = [["File1", "File2"]]
>>> +
>>> +Files = [[A_File, B_File]]
>>> +LocalFiles = [Netrc_File]
>>> +
>>> +ExpectedReturnCode = 0
>>> +ExpectedDownloadedFiles = [A_File, B_File, Netrc_File]
>>> +
>>> +################ Pre and Post Test Hooks 
>>> #####################################
>>> +pre_test = {
>>> +    "ServerFiles"       : Files,
>>> +    "LocalFiles"        : LocalFiles
>>> +}
>>> +test_options = {
>>> +    "WgetCommands"      : WGET_OPTIONS,
>>> +    "Urls"              : WGET_URLS
>>> +}
>>> +post_test = {
>>> +    "ExpectedFiles"     : ExpectedDownloadedFiles,
>>> +    "ExpectedRetcode"   : ExpectedReturnCode
>>> +}
>>> +
>>> +err = HTTPTest (
>>> +                pre_hook=pre_test,
>>> +                test_params=test_options,
>>> +                post_hook=post_test
>>> +).begin ()
>>> +
>>> +exit (err)
>>> diff --git a/testenv/Test-auth-basic-netrc-user-given.py 
>>> b/testenv/Test-auth-basic-netrc-user-given.py
>>> new file mode 100755
>>> index 0000000..57b6148
>>> --- /dev/null
>>> +++ b/testenv/Test-auth-basic-netrc-user-given.py
>>> @@ -0,0 +1,68 @@
>>> +#!/usr/bin/env python3
>>> +from sys import exit
>>> +from test.http_test import HTTPTest
>>> +from misc.wget_file import WgetFile
>>> +
>>> +"""
>>> +    This test ensures Wget uses credentials from .netrc for Basic 
>>> Authorization Negotiation.
>>> +    In this case we test that .netrc credentials are used in case only
>>> +    user login is given on the command line.
>>> +    Also, we ensure that Wget saves the host after a successful auth and
>>> +    doesn't wait for a challenge the second time.
>>> +"""
>>> +############# File Definitions 
>>> ###############################################
>>> +File1 = "I am an invisble man."
>>> +File2 = "I too am an invisible man."
>>> +
>>> +User = "Sauron"
>>> +Password = "TheEye"
>>> +
>>> +File1_rules = {
>>> +    "Authentication"    : {
>>> +        "Type"          : "Basic",
>>> +        "User"          : User,
>>> +        "Pass"          : Password
>>> +    }
>>> +}
>>> +File2_rules = {
>>> +    "ExpectHeader"      : {
>>> +        "Authorization" : "Basic U2F1cm9uOlRoZUV5ZQ=="
>>> +    }
>>> +}
>>> +
>>> +Netrc = "machine 127.0.0.1\n\tlogin {0}\n\tpassword {1}".format(User, 
>>> Password)
>>> +
>>> +A_File = WgetFile ("File1", File1, rules=File1_rules)
>>> +B_File = WgetFile ("File2", File2, rules=File2_rules)
>>> +Netrc_File = WgetFile (".netrc", Netrc)
>>> +
>>> +WGET_OPTIONS = "--user={0}".format(User)
>>> +WGET_URLS = [["File1", "File2"]]
>>> +
>>> +Files = [[A_File, B_File]]
>>> +LocalFiles = [Netrc_File]
>>> +
>>> +ExpectedReturnCode = 0
>>> +ExpectedDownloadedFiles = [A_File, B_File, Netrc_File]
>>> +
>>> +################ Pre and Post Test Hooks 
>>> #####################################
>>> +pre_test = {
>>> +    "ServerFiles"       : Files,
>>> +    "LocalFiles"        : LocalFiles
>>> +}
>>> +test_options = {
>>> +    "WgetCommands"      : WGET_OPTIONS,
>>> +    "Urls"              : WGET_URLS
>>> +}
>>> +post_test = {
>>> +    "ExpectedFiles"     : ExpectedDownloadedFiles,
>>> +    "ExpectedRetcode"   : ExpectedReturnCode
>>> +}
>>> +
>>> +err = HTTPTest (
>>> +                pre_hook=pre_test,
>>> +                test_params=test_options,
>>> +                post_hook=post_test
>>> +).begin ()
>>> +
>>> +exit (err)
>>> diff --git a/testenv/Test-auth-basic-netrc.py 
>>> b/testenv/Test-auth-basic-netrc.py
>>> new file mode 100755
>>> index 0000000..5710fe7
>>> --- /dev/null
>>> +++ b/testenv/Test-auth-basic-netrc.py
>>> @@ -0,0 +1,66 @@
>>> +#!/usr/bin/env python3
>>> +from sys import exit
>>> +from test.http_test import HTTPTest
>>> +from misc.wget_file import WgetFile
>>> +
>>> +"""
>>> +    This test ensures Wget uses credentials from .netrc for Basic 
>>> Authorization Negotiation.
>>> +    In this case we test that .netrc credentials are used in case no user
>>> +    login and no password is given on the command line.
>>> +    Also, we ensure that Wget saves the host after a successful auth and
>>> +    doesn't wait for a challenge the second time.
>>> +"""
>>> +############# File Definitions 
>>> ###############################################
>>> +File1 = "I am an invisble man."
>>> +File2 = "I too am an invisible man."
>>> +
>>> +User = "Sauron"
>>> +Password = "TheEye"
>>> +
>>> +File1_rules = {
>>> +    "Authentication"    : {
>>> +        "Type"          : "Basic",
>>> +        "User"          : User,
>>> +        "Pass"          : Password
>>> +    }
>>> +}
>>> +File2_rules = {
>>> +    "ExpectHeader"      : {
>>> +        "Authorization" : "Basic U2F1cm9uOlRoZUV5ZQ=="
>>> +    }
>>> +}
>>> +
>>> +Netrc = "machine 127.0.0.1\n\tlogin {0}\n\tpassword {1}".format(User, 
>>> Password)
>>> +
>>> +A_File = WgetFile ("File1", File1, rules=File1_rules)
>>> +B_File = WgetFile ("File2", File2, rules=File2_rules)
>>> +Netrc_File = WgetFile (".netrc", Netrc)
>>> +
>>> +WGET_URLS = [["File1", "File2"]]
>>> +
>>> +Files = [[A_File, B_File]]
>>> +LocalFiles = [Netrc_File]
>>> +
>>> +ExpectedReturnCode = 0
>>> +ExpectedDownloadedFiles = [A_File, B_File, Netrc_File]
>>> +
>>> +################ Pre and Post Test Hooks 
>>> #####################################
>>> +pre_test = {
>>> +    "ServerFiles"       : Files,
>>> +    "LocalFiles"        : LocalFiles
>>> +}
>>> +test_options = {
>>> +    "Urls"              : WGET_URLS
>>> +}
>>> +post_test = {
>>> +    "ExpectedFiles"     : ExpectedDownloadedFiles,
>>> +    "ExpectedRetcode"   : ExpectedReturnCode
>>> +}
>>> +
>>> +err = HTTPTest (
>>> +                pre_hook=pre_test,
>>> +                test_params=test_options,
>>> +                post_hook=post_test
>>> +).begin ()
>>> +
>>> +exit (err)
>>> diff --git a/testenv/test/base_test.py b/testenv/test/base_test.py
>>> index b0087e9..bb706d8 100644
>>> --- a/testenv/test/base_test.py
>>> +++ b/testenv/test/base_test.py
>>> @@ -102,7 +102,7 @@ class BaseTest:
>>>              time.sleep(float(os.getenv("SERVER_WAIT")))
>>>  
>>>          try:
>>> -            ret_code = call(params)
>>> +            ret_code = call(params, env={"HOME": os.getcwd()})
>>>          except FileNotFoundError:
>>>              raise TestFailed("The Wget Executable does not exist at the "
>>>                               "expected path.")
>>>
>>
> 
> 
> 

Attachment: signature.asc
Description: OpenPGP digital signature


reply via email to

[Prev in Thread] Current Thread [Next in Thread]