Re: [Bug-wget] [GSoC Proposal] Python3-based FTP Server for Test Suit and Test Suit Refactoring

[Zihang Chen]

Sure.


2014-03-07 22:42 GMT+08:00 Darshit Shah <address@hidden>:

>
>
>
> On Fri, Mar 7, 2014 at 3:40 PM, 陈子杭 (Zihang Chen) <address@hidden>wrote:
>
>> Oh, right. Didn't come up with the second way. I'll take that into
>> consideration. Thanks.
>> BTW, how should the final proposal look like? Should I make a pdf file
>> out of latex or just plain text? I'm a first-time GSoCer.
>>
>> You'd have to use HTML. Since the proposals need to be sent through the
> melange website which has a WYSWYG editor I guess
> There's time for that though. Proposals open on 10th March. Check out the
> format then and send in one.
>
>>
>>
>> 2014-03-07 22:35 GMT+08:00 Darshit Shah <address@hidden>:
>>
>>
>>>
>>>
>>> On Fri, Mar 7, 2014 at 3:32 PM, 陈子杭 (Zihang Chen) <address@hidden>wrote:
>>>
>>>> Thanks for your comments! :D
>>>>
>>>>
>>>> 2014-03-07 22:17 GMT+08:00 Darshit Shah <address@hidden>:
>>>>
>>>> Hi Zihang,
>>>>>
>>>>> The proposal seems good.
>>>>>
>>>>>
>>>>> On Fri, Mar 7, 2014 at 11:39 AM, 陈子杭 (Zihang Chen) <address@hidden
>>>>> > wrote:
>>>>>
>>>>>> * this is only a draft, so it's not written according to the GNU GSoC
>>>>>> template. When discussion is done, I'll get a formal one done (with
>>>>>> schedule etc.) *
>>>>>>
>>>>>> Briefing
>>>>>> ====
>>>>>>
>>>>>> As is known, a new Python3-based test suite was implemented to
>>>>>> replace the
>>>>>> old one. And a configurable HTTP server utilizing `http.server` and
>>>>>> `http.BaseHTTPRequestHandler` was also implemented. Thus the HTTP
>>>>>> server
>>>>>> can be configured to respond certain headers according to the
>>>>>> configuration
>>>>>> in separate test cases.
>>>>>>
>>>>>> However, a configurable FTP server was not required to be implemented,
>>>>>> which breaks the consistency of the test suite of wget. So it's
>>>>>> necessary
>>>>>> that a configurable FTP server be built.
>>>>>>
>>>>>> Initial Design
>>>>>> ====
>>>>>>
>>>>>> The test suite consists of two parts: `WgetTest.py` (module that
>>>>>> actually
>>>>>> runs the tests), servers (configurable HTTP and FTP server). When
>>>>>> running
>>>>>> test cases, `HTTPTest` instances collect configurations from test
>>>>>> cases and
>>>>>> start the servers with them so that the HTTP handler will handle the
>>>>>> incoming requests according to the configurations (also known as
>>>>>> rules).
>>>>>>
>>>>>> About the FTP server, I think the handler architecture can be applied
>>>>>> here:
>>>>>> implement multiple handlers corresponding to the FTP commands and
>>>>>> register
>>>>>> them in server instances automatically. When registering handlers, the
>>>>>> server should also inform the handlers about the configurations. When
>>>>>> new
>>>>>> FTP command arrives, the server invoke the corresponding handler, and
>>>>>> then
>>>>>> the handler look up the rules itself and do the expected actions. When
>>>>>> doing real FTP actions, I intend to use third-party FTP library like
>>>>>> the
>>>>>> current FTP server (I fear sockets so best not directly touch them).
>>>>>>
>>>>>> There's two things that are unclear:
>>>>>> * the necessity of implementing all the FTP command. I presume that
>>>>>> wget
>>>>>> wouldn't utilize all of them.
>>>>>>
>>>>> If you're using a library, e.g. pyftpdlib, I don't see why you'd have
>>>>> to implement
>>>>> any commands on your own.
>>>>>
>>>> If the server is configurable, I think I'll have to intercept the
>>>> commands so I can apply rules on them. When I say implement FTP command, I
>>>> mean creating handle for the command, and invoke library in the handle. I
>>>> think that's what mislead you probably.
>>>>
>>> Oh. Okay then. In that scenario, you could probably add the command
>>> handlers
>>> iteratively as and when they are required by a test case.
>>>
>>> Another way would be to look at Wget's FTP client implementation and see
>>> what is the
>>> superset of commands that it may send. If you're not comfortable with C,
>>> I would help
>>> you out with that.
>>>
>>>>    Simply use the library for it.
>>>>>
>>>>>> * what will the configurations be, do I need to design the tests
>>>>>> myself? Or
>>>>>> should I also port the Perl scripts to Python3?
>>>>>>
>>>>> Porting the existing Perl tests should give you a good idea of the
>>>>> various
>>>>> configurations that will be needed. Looking up the Wget source is also
>>>>> often
>>>>> a good idea. Either ways, some of us can help out with identifying and
>>>>> writing
>>>>> test cases when the time comes.
>>>>>
>>>>>
>>>>>> Test Suite Refactoring
>>>>>> ====
>>>>>>
>>>>>> While reading source, I don't find the test suite pythonic. Multiple
>>>>>> parts
>>>>>> of it seems unmaintainable. If the test suite can be refactored into a
>>>>>> better coded one,  maintainability can further be achieved.
>>>>>>
>>>>>> Yes. I agree the current python based test suite might advantage from
>>>>> a
>>>>> few refactorings. There are a few places where the test suite is quite
>>>>> un-pythonic
>>>>> and fixing those would be a nice idea.
>>>>>
>>>>> Examples of unpythonicness:
>>>>>> * docstrings out of classes or methods
>>>>>> * mutable objects as default argument values
>>>>>> * multiple use of operator is (controversial)
>>>>>> * writing `_replace_substring (WgetTest.py:119)` instead of utilizing
>>>>>> the
>>>>>> format specification mini-language
>>>>>>
>>>>>> Examples of unmaintainability:
>>>>>> * referencing attributes which are defined in subclasses
>>>>>> (`WgetTest.py:61`
>>>>>> references `self.servers` which is defined in its subclass
>>>>>> `WgetTest.py:254`)
>>>>>> * inappropriate variable names (in `WgetTest.py:254` `self.servers`
>>>>>> actually becomes an integer while it regarded normally as a list of
>>>>>> servers)
>>>>>>
>>>>> Yes, the variable names are horrible aren't they? That should be
>>>>> refactored. (Shouldn't
>>>>> take much time ).
>>>>>
>>>>>> * no intuitive, self-declarative module hierarchy
>>>>>>
>>>>> The module hierarchy is slightly complicated. Making it more intuitive
>>>>> would help.
>>>>>
>>>>>>
>>>>>> About me
>>>>>> ====
>>>>>> My name's Zihang Chen. I'm a 3rd undergraduate student from China
>>>>>> (timezone: UTC+8). Please forgive me for any mistaken uses of the
>>>>>> English
>>>>>> language. I have been using Python for 3 years. I mostly code for
>>>>>> myself
>>>>>> (assignment, interest etc.) Never worked for any open source
>>>>>> organizations
>>>>>> before, so I'm thrilled to apply for GSoC. You can visit my GitHub
>>>>>> page here<https://github.com/mad4alcohol>
>>>>>>
>>>>>> .
>>>>>>
>>>>>> Looking forward to your comments. Thanks very much.
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Thanking You,
>>>>> Darshit Shah
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> Regards,
>>>> Chen Zihang,
>>>> Computer School of Wuhan University
>>>> ---
>>>> 此致
>>>> 陈子杭
>>>> 武汉大学计算机学院
>>>>
>>>>
>>>
>>>
>>> --
>>> Thanking You,
>>> Darshit Shah
>>>
>>>
>>
>>
>> --
>> Regards,
>> Chen Zihang,
>> Computer School of Wuhan University
>> ---
>> 此致
>> 陈子杭
>> 武汉大学计算机学院
>>
>>
>
>
> --
> Thanking You,
> Darshit Shah
>
>


-- 
Regards,
Chen Zihang,
Computer School of Wuhan University
---
此致
陈子杭
武汉大学计算机学院