Re: [Bug-wget] [PATCH 1/3] Added tests for HTTP authentication using credentials from .netrc

[Tomas Hozza]

Hello Tim.

I run all the tests before sending patches and they all passed. If you can send 
me more info about the failures, I could have a look at it.

My locale is:
$ locale
LANG=en_US.UTF-8
LC_CTYPE="en_US.UTF-8"
LC_NUMERIC=sk_SK.UTF-8
LC_TIME=sk_SK.UTF-8
LC_COLLATE="en_US.UTF-8"
LC_MONETARY=sk_SK.UTF-8
LC_MESSAGES="en_US.UTF-8"
LC_PAPER=sk_SK.UTF-8
LC_NAME="en_US.UTF-8"
LC_ADDRESS="en_US.UTF-8"
LC_TELEPHONE="en_US.UTF-8"
LC_MEASUREMENT=sk_SK.UTF-8
LC_IDENTIFICATION="en_US.UTF-8"
LC_ALL=

Thanks.

Regards,
Tomas

On 15.05.2017 15:52, Tim Rühsen wrote:
> Hi Tomas,
>
> thank you very much for the fixes and the new tests !
>
> Currently, patch 1/3 breaks two Metalink tests:
>
> FAIL: Test-metalink-http.py
> FAIL: Test-metalink-http-quoted.py
>
> I am still investigating, but it has something to do with the
> environment that you reset in testenv/test/base_test.py:
>
>     ret_code = call(params, env={"HOME": os.getcwd()})
>
> Maybe a bug that just comes out with a different locale/charset encoding.
>
>
> With Best Regards, Tim
>
>
>
> On 05/12/2017 07:17 PM, Tomas Hozza wrote:
> > Getting credentials from .netrc has been broken from time to time, thus
> > adding a test coverage to prevent regressions.
> >
> > Also added setting of "HOME" environment variable when executing wget,
> > to make sure LocalFiles like .netrc, which are created just for the
> > test, are actually used.
> >
> > Signed-off-by: Tomas Hozza <address@hidden>
> > ---
> >  testenv/Makefile.am                         |  3 ++
> >  testenv/Test-auth-basic-netrc-pass-given.py | 68 
> > +++++++++++++++++++++++++++++
> >  testenv/Test-auth-basic-netrc-user-given.py | 68 
> > +++++++++++++++++++++++++++++
> >  testenv/Test-auth-basic-netrc.py            | 66 
> > ++++++++++++++++++++++++++++
> >  testenv/test/base_test.py                   |  2 +-
> >  5 files changed, 206 insertions(+), 1 deletion(-)
> >  create mode 100755 testenv/Test-auth-basic-netrc-pass-given.py
> >  create mode 100755 testenv/Test-auth-basic-netrc-user-given.py
> >  create mode 100755 testenv/Test-auth-basic-netrc.py
> >
> > diff --git a/testenv/Makefile.am b/testenv/Makefile.am
> > index 3febec7..7104314 100644
> > --- a/testenv/Makefile.am
> > +++ b/testenv/Makefile.am
> > @@ -75,6 +75,9 @@ if HAVE_PYTHON3
> >    TESTS = Test-504.py                               \
> >      Test-auth-basic-fail.py                         \
> >      Test-auth-basic.py                              \
> > +    Test-auth-basic-netrc.py                        \
> > +    Test-auth-basic-netrc-user-given.py             \
> > +    Test-auth-basic-netrc-pass-given.py             \
> >      Test-auth-both.py                               \
> >      Test-auth-digest.py                             \
> >      Test-auth-no-challenge.py                       \
> > diff --git a/testenv/Test-auth-basic-netrc-pass-given.py 
> > b/testenv/Test-auth-basic-netrc-pass-given.py
> > new file mode 100755
> > index 0000000..43dfe34
> > --- /dev/null
> > +++ b/testenv/Test-auth-basic-netrc-pass-given.py
> > @@ -0,0 +1,68 @@
> > +#!/usr/bin/env python3
> > +from sys import exit
> > +from test.http_test import HTTPTest
> > +from misc.wget_file import WgetFile
> > +
> > +"""
> > +    This test ensures Wget uses credentials from .netrc for Basic 
> > Authorization Negotiation.
> > +    In this case we test that .netrc credentials are used in case only
> > +    password is given on the command line.
> > +    Also, we ensure that Wget saves the host after a successful auth and
> > +    doesn't wait for a challenge the second time.
> > +"""
> > +############# File Definitions 
> > ###############################################
> > +File1 = "I am an invisble man."
> > +File2 = "I too am an invisible man."
> > +
> > +User = "Sauron"
> > +Password = "TheEye"
> > +
> > +File1_rules = {
> > +    "Authentication"    : {
> > +        "Type"          : "Basic",
> > +        "User"          : User,
> > +        "Pass"          : Password
> > +    }
> > +}
> > +File2_rules = {
> > +    "ExpectHeader"      : {
> > +        "Authorization" : "Basic U2F1cm9uOlRoZUV5ZQ=="
> > +    }
> > +}
> > +
> > +Netrc = "machine 127.0.0.1\n\tlogin {0}".format(User)
> > +
> > +A_File = WgetFile ("File1", File1, rules=File1_rules)
> > +B_File = WgetFile ("File2", File2, rules=File2_rules)
> > +Netrc_File = WgetFile (".netrc", Netrc)
> > +
> > +WGET_OPTIONS = "--password={0}".format(Password)
> > +WGET_URLS = [["File1", "File2"]]
> > +
> > +Files = [[A_File, B_File]]
> > +LocalFiles = [Netrc_File]
> > +
> > +ExpectedReturnCode = 0
> > +ExpectedDownloadedFiles = [A_File, B_File, Netrc_File]
> > +
> > +################ Pre and Post Test Hooks 
> > #####################################
> > +pre_test = {
> > +    "ServerFiles"       : Files,
> > +    "LocalFiles"        : LocalFiles
> > +}
> > +test_options = {
> > +    "WgetCommands"      : WGET_OPTIONS,
> > +    "Urls"              : WGET_URLS
> > +}
> > +post_test = {
> > +    "ExpectedFiles"     : ExpectedDownloadedFiles,
> > +    "ExpectedRetcode"   : ExpectedReturnCode
> > +}
> > +
> > +err = HTTPTest (
> > +                pre_hook=pre_test,
> > +                test_params=test_options,
> > +                post_hook=post_test
> > +).begin ()
> > +
> > +exit (err)
> > diff --git a/testenv/Test-auth-basic-netrc-user-given.py 
> > b/testenv/Test-auth-basic-netrc-user-given.py
> > new file mode 100755
> > index 0000000..57b6148
> > --- /dev/null
> > +++ b/testenv/Test-auth-basic-netrc-user-given.py
> > @@ -0,0 +1,68 @@
> > +#!/usr/bin/env python3
> > +from sys import exit
> > +from test.http_test import HTTPTest
> > +from misc.wget_file import WgetFile
> > +
> > +"""
> > +    This test ensures Wget uses credentials from .netrc for Basic 
> > Authorization Negotiation.
> > +    In this case we test that .netrc credentials are used in case only
> > +    user login is given on the command line.
> > +    Also, we ensure that Wget saves the host after a successful auth and
> > +    doesn't wait for a challenge the second time.
> > +"""
> > +############# File Definitions 
> > ###############################################
> > +File1 = "I am an invisble man."
> > +File2 = "I too am an invisible man."
> > +
> > +User = "Sauron"
> > +Password = "TheEye"
> > +
> > +File1_rules = {
> > +    "Authentication"    : {
> > +        "Type"          : "Basic",
> > +        "User"          : User,
> > +        "Pass"          : Password
> > +    }
> > +}
> > +File2_rules = {
> > +    "ExpectHeader"      : {
> > +        "Authorization" : "Basic U2F1cm9uOlRoZUV5ZQ=="
> > +    }
> > +}
> > +
> > +Netrc = "machine 127.0.0.1\n\tlogin {0}\n\tpassword {1}".format(User, 
> > Password)
> > +
> > +A_File = WgetFile ("File1", File1, rules=File1_rules)
> > +B_File = WgetFile ("File2", File2, rules=File2_rules)
> > +Netrc_File = WgetFile (".netrc", Netrc)
> > +
> > +WGET_OPTIONS = "--user={0}".format(User)
> > +WGET_URLS = [["File1", "File2"]]
> > +
> > +Files = [[A_File, B_File]]
> > +LocalFiles = [Netrc_File]
> > +
> > +ExpectedReturnCode = 0
> > +ExpectedDownloadedFiles = [A_File, B_File, Netrc_File]
> > +
> > +################ Pre and Post Test Hooks 
> > #####################################
> > +pre_test = {
> > +    "ServerFiles"       : Files,
> > +    "LocalFiles"        : LocalFiles
> > +}
> > +test_options = {
> > +    "WgetCommands"      : WGET_OPTIONS,
> > +    "Urls"              : WGET_URLS
> > +}
> > +post_test = {
> > +    "ExpectedFiles"     : ExpectedDownloadedFiles,
> > +    "ExpectedRetcode"   : ExpectedReturnCode
> > +}
> > +
> > +err = HTTPTest (
> > +                pre_hook=pre_test,
> > +                test_params=test_options,
> > +                post_hook=post_test
> > +).begin ()
> > +
> > +exit (err)
> > diff --git a/testenv/Test-auth-basic-netrc.py 
> > b/testenv/Test-auth-basic-netrc.py
> > new file mode 100755
> > index 0000000..5710fe7
> > --- /dev/null
> > +++ b/testenv/Test-auth-basic-netrc.py
> > @@ -0,0 +1,66 @@
> > +#!/usr/bin/env python3
> > +from sys import exit
> > +from test.http_test import HTTPTest
> > +from misc.wget_file import WgetFile
> > +
> > +"""
> > +    This test ensures Wget uses credentials from .netrc for Basic 
> > Authorization Negotiation.
> > +    In this case we test that .netrc credentials are used in case no user
> > +    login and no password is given on the command line.
> > +    Also, we ensure that Wget saves the host after a successful auth and
> > +    doesn't wait for a challenge the second time.
> > +"""
> > +############# File Definitions 
> > ###############################################
> > +File1 = "I am an invisble man."
> > +File2 = "I too am an invisible man."
> > +
> > +User = "Sauron"
> > +Password = "TheEye"
> > +
> > +File1_rules = {
> > +    "Authentication"    : {
> > +        "Type"          : "Basic",
> > +        "User"          : User,
> > +        "Pass"          : Password
> > +    }
> > +}
> > +File2_rules = {
> > +    "ExpectHeader"      : {
> > +        "Authorization" : "Basic U2F1cm9uOlRoZUV5ZQ=="
> > +    }
> > +}
> > +
> > +Netrc = "machine 127.0.0.1\n\tlogin {0}\n\tpassword {1}".format(User, 
> > Password)
> > +
> > +A_File = WgetFile ("File1", File1, rules=File1_rules)
> > +B_File = WgetFile ("File2", File2, rules=File2_rules)
> > +Netrc_File = WgetFile (".netrc", Netrc)
> > +
> > +WGET_URLS = [["File1", "File2"]]
> > +
> > +Files = [[A_File, B_File]]
> > +LocalFiles = [Netrc_File]
> > +
> > +ExpectedReturnCode = 0
> > +ExpectedDownloadedFiles = [A_File, B_File, Netrc_File]
> > +
> > +################ Pre and Post Test Hooks 
> > #####################################
> > +pre_test = {
> > +    "ServerFiles"       : Files,
> > +    "LocalFiles"        : LocalFiles
> > +}
> > +test_options = {
> > +    "Urls"              : WGET_URLS
> > +}
> > +post_test = {
> > +    "ExpectedFiles"     : ExpectedDownloadedFiles,
> > +    "ExpectedRetcode"   : ExpectedReturnCode
> > +}
> > +
> > +err = HTTPTest (
> > +                pre_hook=pre_test,
> > +                test_params=test_options,
> > +                post_hook=post_test
> > +).begin ()
> > +
> > +exit (err)
> > diff --git a/testenv/test/base_test.py b/testenv/test/base_test.py
> > index b0087e9..bb706d8 100644
> > --- a/testenv/test/base_test.py
> > +++ b/testenv/test/base_test.py
> > @@ -102,7 +102,7 @@ class BaseTest:
> >              time.sleep(float(os.getenv("SERVER_WAIT")))
> >  
> >          try:
> > -            ret_code = call(params)
> > +            ret_code = call(params, env={"HOME": os.getcwd()})
> >          except FileNotFoundError:
> >              raise TestFailed("The Wget Executable does not exist at the "
> >                               "expected path.")
> >
>



-- 
Tomas Hozza
Associate Manager, Software Engineering - EMEA ENG Mainstream RHEL

PGP: 1D9F3C2D
UTC+2 (CEST)
Red Hat Inc.                 http://cz.redhat.com