Re: Interested in TISEAN package for GSoC

[Juan Pablo Carbajal]

On Thu, Apr 2, 2015 at 8:35 AM, Juan Pablo Carbajal
<address@hidden> wrote:
> On Thu, Apr 2, 2015 at 7:28 AM, Piotr Held <address@hidden> wrote:
>>
>>
>> 2015-04-01 9:48 GMT-06:00 Juan Pablo Carbajal <address@hidden>:
>>
>>> On Wed, Apr 1, 2015 at 4:40 AM, Piotr Held <address@hidden> wrote:
>>> >
>>> >
>>> > 2015-03-28 6:28 GMT-06:00 Juan Pablo Carbajal <address@hidden>:
>>> >
>>> >> On Sat, Mar 28, 2015 at 2:29 AM, Piotr Held <address@hidden> wrote:
>>> >> >
>>> >> >
>>> >> > 2015-03-27 0:46 GMT-06:00 Juan Pablo Carbajal
>>> >> > <address@hidden>:
>>> >> >
>>> >> >> On Fri, Mar 27, 2015 at 7:05 AM, Juan Pablo Carbajal
>>> >> >> <address@hidden> wrote:
>>> >> >> > On Fri, Mar 27, 2015 at 7:00 AM, Juan Pablo Carbajal
>>> >> >> > <address@hidden> wrote:
>>> >> >> >> On Fri, Mar 27, 2015 at 3:24 AM, Piotr Held <address@hidden>
>>> >> >> >> wrote:
>>> >> >> >>> I am sorry, I didn't realize I could edit the proposal.
>>> >> >> >>> Nevermind
>>> >> >> >>> then
>>> >> >> >>> on
>>> >> >> >>> the first question.
>>> >> >> >>>
>>> >> >> >>> 2015-03-26 20:19 GMT-06:00 Piotr Held <address@hidden>:
>>> >> >> >>>
>>> >> >> >>>> Juan,
>>> >> >> >>>>
>>> >> >> >>>> Thanks for the tips. I think I now have an idea how to complete
>>> >> >> >>>> this
>>> >> >> >>>> project.
>>> >> >> >>>>
>>> >> >> >>>> I am aware that the deadline is tomorrow. That is why I
>>> >> >> >>>> submitted
>>> >> >> >>>> a
>>> >> >> >>>> proposal on melange, that does not really have a timeline (as I
>>> >> >> >>>> did
>>> >> >> >>>> not
>>> >> >> >>>> fully understand how to tackle the problem). My wiki page is
>>> >> >> >>>> better
>>> >> >> >>>> in that
>>> >> >> >>>> matter. Should I submit another application on melange that is
>>> >> >> >>>> better, or is
>>> >> >> >>>> it enough that my wiki page is complete?
>>> >> >> >>>>
>>> >> >> >>>> Do you know of any other libraries that link fortran to oct
>>> >> >> >>>> files?
>>> >> >> >>>> I
>>> >> >> >>>> would
>>> >> >> >>>> appreciate the extra examples, if I could get a hold of some.
>>> >> >> >>>>
>>> >> >> >>>> Also could you give me any tips concerning my wiki page? Should
>>> >> >> >>>> some
>>> >> >> >>>> topic
>>> >> >> >>>> be discussed more thoroughly?
>>> >> >> >>>>
>>> >> >> >>>> :)
>>> >> >> >>>>
>>> >> >> >>>> 2015-03-26 6:12 GMT-06:00 Juan Pablo Carbajal
>>> >> >> >>>> <address@hidden>:
>>> >> >> >>>>
>>> >> >> >>>>> On Thu, Mar 26, 2015 at 12:38 PM, Juan Pablo Carbajal
>>> >> >> >>>>> <address@hidden> wrote:
>>> >> >> >>>>> > Please keep the mailing list always in copy.
>>> >> >> >>>>> >
>>> >> >> >>>>> > On Thu, Mar 26, 2015 at 2:15 AM, Piotr Held
>>> >> >> >>>>> > <address@hidden>
>>> >> >> >>>>> > wrote:
>>> >> >> >>>>> >> Hi,
>>> >> >> >>>>> >>
>>> >> >> >>>>> >> I implemented the 'henon' function like you suggested, that
>>> >> >> >>>>> >> is
>>> >> >> >>>>> >> as
>>> >> >> >>>>> >> a
>>> >> >> >>>>> >> m-file.
>>> >> >> >>>>> >> What should I do with it? Should I send it to you? Should I
>>> >> >> >>>>> >> upload it
>>> >> >> >>>>> >> somewhere?
>>> >> >> >>>>> >>
>>> >> >> >>>>> > You can create a repository in Bitbucket (using mercurial).
>>> >> >> >>>>> >
>>> >> >> >>>>> >> The way TISEAN was interfaced with Octave in that tutorial
>>> >> >> >>>>> >> is
>>> >> >> >>>>> >> really
>>> >> >> >>>>> >> just
>>> >> >> >>>>> >> calling the library from the system and reading the output.
>>> >> >> >>>>> >> Is
>>> >> >> >>>>> >> that
>>> >> >> >>>>> >> how this
>>> >> >> >>>>> >> project is suppose to be implemented?
>>> >> >> >>>>> > Absolutely not!
>>> >> >> >>>>> > Either link to the existing FORTRAN code and/or re-implement
>>> >> >> >>>>> > in
>>> >> >> >>>>> > C++.
>>> >> >> >>>>> >
>>> >> >> >>>>> > If not, where can I read about other
>>> >> >> >>>>> >> ways of connecting separate libraries to Octave. Would
>>> >> >> >>>>> >> looking
>>> >> >> >>>>> >> at
>>> >> >> >>>>> >> how
>>> >> >> >>>>> >> the
>>> >> >> >>>>> >> control package was incorporated into Octave help in
>>> >> >> >>>>> >> understanding
>>> >> >> >>>>> >> what
>>> >> >> >>>>> >> options there are out there?
>>> >> >> >>>>> > Yes, looking at the control package will definitely help.
>>> >> >> >>>>> > You can look at the Octave manual
>>> >> >> >>>>> >
>>> >> >> >>>>> >
>>> >> >> >>>>> >
>>> >> >> >>>>> > https://www.gnu.org/software/octave/doc/interpreter/Oct_002dFiles.html
>>> >> >> >>>>> >
>>> >> >> >>>>> >>
>>> >> >> >>>>> >> Should I try to create a list of which functions I would
>>> >> >> >>>>> >> like
>>> >> >> >>>>> >> to
>>> >> >> >>>>> >> incorporate
>>> >> >> >>>>> >> and in what order? And should I post that list on my octave
>>> >> >> >>>>> >> wiki?
>>> >> >> >>>>> > You will have to write an application to be selected for
>>> >> >> >>>>> > SoC,
>>> >> >> >>>>> > add
>>> >> >> >>>>> > this
>>> >> >> >>>>> > list and a description of your plan to that application
>>> >> >> >>>>> > Check the intro of this page
>>> >> >> >>>>> > http://wiki.octave.org/Summer_of_Code_Project_Ideas
>>> >> >> >>>>> >
>>> >> >> >>>>> >>
>>> >> >> >>>>> >> :)
>>> >> >> >>>>> >>
>>> >> >> >>>>> >>
>>> >> >> >>>>> >> 2015-03-25 7:09 GMT-06:00 Juan Pablo Carbajal
>>> >> >> >>>>> >> <address@hidden>:
>>> >> >> >>>>> >>
>>> >> >> >>>>> >>>
>>> >> >> >>>>> >>>
>>> >> >> >>>>> >>> On Tue, Mar 24, 2015 at 11:33 PM, Piotr Held
>>> >> >> >>>>> >>> <address@hidden>
>>> >> >> >>>>> >>> wrote:
>>> >> >> >>>>> >>>>
>>> >> >> >>>>> >>>> Hello,
>>> >> >> >>>>> >>>>
>>> >> >> >>>>> >>>> I am interested in doing this project for GSoC. I have
>>> >> >> >>>>> >>>> some
>>> >> >> >>>>> >>>> experience in
>>> >> >> >>>>> >>>> programming in C++ but have never done anything with
>>> >> >> >>>>> >>>> Fortran. I
>>> >> >> >>>>> >>>> thought this
>>> >> >> >>>>> >>>> would be interesting to contribute because I can learn
>>> >> >> >>>>> >>>> about
>>> >> >> >>>>> >>>> how to
>>> >> >> >>>>> >>>> make a
>>> >> >> >>>>> >>>> package for Octave.
>>> >> >> >>>>> >>>>
>>> >> >> >>>>> >>>> So far I have compiled Octave from the sources
>>> >> >> >>>>> >>>> successfully
>>> >> >> >>>>> >>>> and
>>> >> >> >>>>> >>>> have
>>> >> >> >>>>> >>>> been
>>> >> >> >>>>> >>>> playing around with it a bit.
>>> >> >> >>>>> >>>>
>>> >> >> >>>>> >>>> I was wondering what should be my next step?
>>> >> >> >>>>> >>>
>>> >> >> >>>>> >>>
>>> >> >> >>>>> >>> Hi,
>>> >> >> >>>>> >>>
>>> >> >> >>>>> >>> Thanks for your interest.
>>> >> >> >>>>> >>> The next step would be to download TISEAN
>>> >> >> >>>>> >>>
>>> >> >> >>>>> >>> http://www.mpipks-dresden.mpg.de/~tisean/archive_3.0.0.html
>>> >> >> >>>>> >>>
>>> >> >> >>>>> >>> Compile it and get acquainted with the tutorials and
>>> >> >> >>>>> >>> functions
>>> >> >> >>>>> >>> interfaces
>>> >> >> >>>>> >>> http://www.mpipks-dresden.mpg.de/~tisean/Tisean_3.0.0/
>>> >> >> >>>>> >>>
>>> >> >> >>>>> >>> This step is requiered before you make any implementation
>>> >> >> >>>>> >>> decision.
>>> >> >> >>>>> >>>
>>> >> >> >>>>> >>> You could also understand how TISEAN was interfaced with
>>> >> >> >>>>> >>> Octave/Matlab in
>>> >> >> >>>>> >>> the past to decide which functions you want to port first.
>>> >> >> >>>>> >>>
>>> >> >> >>>>> >>>
>>> >> >> >>>>> >>>
>>> >> >> >>>>> >>>
>>> >> >> >>>>> >>> http://www.agnld.uni-potsdam.de/~marwan/matlab-tutorials/html/tisean.html
>>> >> >> >>>>> >>>
>>> >> >> >>>>> >>> Note that there are many functions in the package (e.g.
>>> >> >> >>>>> >>> dynamical
>>> >> >> >>>>> >>> functions like henon, etc...) that do not need to be
>>> >> >> >>>>> >>> ported
>>> >> >> >>>>> >>> to
>>> >> >> >>>>> >>> C++
>>> >> >> >>>>> >>> since
>>> >> >> >>>>> >>> they can be easily implemented in octave. That is this
>>> >> >> >>>>> >>> functions
>>> >> >> >>>>> >>> can
>>> >> >> >>>>> >>> be come
>>> >> >> >>>>> >>> .m files, so maybe you want to start there once you are
>>> >> >> >>>>> >>> acquainted
>>> >> >> >>>>> >>> with the
>>> >> >> >>>>> >>> package.
>>> >> >> >>>>> >>>
>>> >> >> >>>>> >>> Cheers
>>> >> >> >>>>> >>>
>>> >> >> >>>>> >>
>>> >> >> >>>>> >
>>> >> >> >>>>> > Good luck!
>>> >> >> >>>>>
>>> >> >> >>>>> Piotr I am copy information form the other discussion
>>> >> >> >>>>> regarding
>>> >> >> >>>>> GSoC.
>>> >> >> >>>>>
>>> >> >> >>>>> The application deadline is tomorrow.  We then have until 13
>>> >> >> >>>>> of
>>> >> >> >>>>> April to
>>> >> >> >>>>> rank
>>> >> >> >>>>> and select students.
>>> >> >> >>>>> As far as I am aware, no one else is applying for this
>>> >> >> >>>>> project.
>>> >> >> >>>>> But
>>> >> >> >>>>> please, do not think that there's a slot for every project, or
>>> >> >> >>>>> that
>>> >> >> >>>>> we
>>> >> >> >>>>> will select a student just because there's no one else.
>>> >> >> >>>>
>>> >> >> >>>>
>>> >> >> >>>
>>> >> >> >>
>>> >> >> >> Piotr,
>>> >> >> >>
>>> >> >> >> Please answer at the bottom of the message or interleave your
>>> >> >> >> answer
>>> >> >> >> with previous mails. This is to make easier the reading of the
>>> >> >> >> archives.
>>> >> >> >>
>>> >> >> >> What is your username in the wiki?
>>> >> >> >> Did you send your proposal to GNU Project?
>>> >> >> >>
>>> >> >> >> If you live in a ESA collaborating country you might also start
>>> >> >> >> your
>>> >> >> >> application for SoCiS.
>>> >> >> >
>>> >> >> > I found your Melange application.
>>> >> >>
>>> >> >> Piotr,
>>> >> >>
>>> >> >> I read both of your proposals. They look good. However, it would be
>>> >> >> good if you could give more details on your approach to the problem.
>>> >> >> I think your first steps were really good: get in touch, look at the
>>> >> >> code, try to learn what you need. At some point it would be good to
>>> >> >> know how you plan the activities (it wont be written in stone, is
>>> >> >> just
>>> >> >> to have a guideline and an impression of how you want to proceed).
>>> >> >> We
>>> >> >> can iterate over that plan before you start coding.
>>> >> >>
>>> >> >> I personally prefer a solid start rather than volume of work. That
>>> >> >> is,
>>> >> >> porting a small fraction of TISEAN would be enough, as long as it is
>>> >> >> well documented and the decision on what will be ported by linking
>>> >> >> FORTRAN code, what will be re-implemented in C++, and what will be
>>> >> >> implemented as m-files is clear.
>>> >> >> For example, you could go through the TISEAN manual and choose the
>>> >> >> chapter that interest you the most (e.g. Testing for nonlinearity)
>>> >> >> and
>>> >> >> list the functions that are needed. Then you could make a list of
>>> >> >> those functions, indicating which are in FORTRAN and which in C and
>>> >> >> which wont be ported because they can be implemented directly as
>>> >> >> m-file using GNU Octave code. This would be a nice addition to your
>>> >> >> proposal (do not over do it!).
>>> >> >>
>>> >> >> Regarding learning how to do a package you can start in the manual
>>> >> >> https://www.gnu.org/software/octave/doc/interpreter/Packages.html
>>> >> >> And of course look at the many examples in the projects
>>> >> >> https://sourceforge.net/p/octave/_list/hg?source=navbar
>>> >> >> I recommend you look at signal, control, image, linear-algebra
>>> >> >> and/or
>>> >> >> optim. I made geometry based on older packages (I think it will need
>>> >> >> an improvement for Octave 4.0.0), and it uses the idea of
>>> >> >> subpackages
>>> >> >> (which is a very useful hack).
>>> >> >>
>>> >> >> Another package that uses FORTRAN (90) is optiminterp. The package
>>> >> >> odepgk shows how to apply patches to FORTRAN files before compiling
>>> >> >> for octave. There might be other FORTRAN packages but I am not aware
>>> >> >> of them. Feel free to crawl the repository.
>>> >> >>
>>> >> >> Please do share the function henon function. If you haven't set a
>>> >> >> repository yet you can upload the file to Agora
>>> >> >> http://agora.octave.org/snippet/
>>> >> >> and send the link
>>> >> >>
>>> >> >> Cheers
>>> >> >
>>> >> >
>>> >> > Juan,
>>> >> >
>>> >> > Thanks a lot for all the information you sent me. I took a close look
>>> >> > at
>>> >> > the
>>> >> > control package today.
>>> >> >
>>> >> > I posted my henon.m file in my project in the elaborate section. Here
>>> >> > is
>>> >> > also a link to it:
>>> >> >
>>> >> > http://agora.octave.org/snippet/hOzw/
>>> >> >
>>> >> > Could you comment on where the usage should be placed? Should it be
>>> >> > in
>>> >> > the
>>> >> > comments section under texinfo, or should it be displayed the way I
>>> >> > did
>>> >> > it?
>>> >> > Or is this irrelevant?
>>> >> >
>>> >> > I decided to start working on the Nonlinear noise reduction, since
>>> >> > prior
>>> >> > to
>>> >> > getting your email I already started working on project.f. I started
>>> >> > looking
>>> >> > at how to make packages. And I have a few questions:
>>> >> >
>>> >> > 1. I have only used doxygen for documentation so far. From the
>>> >> > packages
>>> >> > I
>>> >> > have looked at, they use texinfo, is that what I should be using as
>>> >> > well? If
>>> >> > yes, how do I actually generate the documentation? I tried to use
>>> >> > 'makeinfo'
>>> >> > on a m-file from the control package and it didn't really do
>>> >> > anything.
>>> >> >
>>> >> > 2. As looked at the documentation I noticed some people, like the
>>> >> > author
>>> >> > of
>>> >> > the Control package gives writes down each files version in the
>>> >> > comments
>>> >> > section, whereas other people like the author of actuarial does not.
>>> >> > Which
>>> >> > route should I take? Does it matter?
>>> >> >
>>> >> > 3. My plan was to make the fortran code "stupid" and do the input
>>> >> > validation
>>> >> > in the on the side of the C++ files. Is that a good approach?
>>> >> >
>>> >> > :)
>>> >>
>>> >> Piotr,
>>> >>
>>> >> Please always keep the mailing list in copy!
>>> >>
>>> >> Thanks for the function. To learn how to use texinfo just look at
>>> >> other functions. Check signal or geometry for example.
>>> >> Also the error message is a little verbose. You better add that to the
>>> >> help of the function and just use the error function.
>>> >>
>>> >> At the moment we are going through the proposals and evaluating them.
>>> >> The faster you can provide code, the more chances you have that we can
>>> >> decide fairly.
>>> >> Start the mercurial repo and upload your work, so at least we can see
>>> >> what you have done, and how.
>>> >>
>>> >> 1. Texinfo is used for all octave. The documentation is parsed with
>>> >> the function help on the prompt or if you want to have a separate
>>> >> manual (which I wouldn't recommend since TISEAN already has a manual,
>>> >> you can just have an user guide) check how it is done in the control
>>> >> package, folder doc. We can do this at a later stage. for now try to
>>> >> show us how you are able to code/port some functions.
>>> >>
>>> >> 2. It is up to you, usually those comments are automatically generated
>>> >> with the tool you use to write the functions. I just keep the creation
>>> >> date, the version is given by the version control system (i.e.
>>> >> mercurial).
>>> >>
>>> >> 3. I am not sure I understand your idea. Best way to go is to provide
>>> >> an example.
>>> >>
>>> >> Cheers
>>> >
>>> >
>>> >  Juan,
>>> >
>>> > Thanks for all of the info you sent me.
>>> >
>>> > I have linked to 2 programs from the TISEAN library. You can view my
>>> > progress here:
>>> >
>>> > https://bitbucket.org/josiah425/tisean
>>> >
>>> > I have put a link on my wiki, but it's not in a visible place. Should I
>>> > put
>>> > a link in section "E: Coding Experience" on my wiki?
>>> Make it visible, also add a
>>> >
>>> > In the next couple of days I intend to add gkss and makefiles.
>>> > Afterwards I
>>> > will try to make it into a package.
>>> >
>>> > If you have any comments on the code I would like to hear them.
>>> >
>>> > :)
>>> >
>>>
>>> This code is a great way to show you can do it!
>>> Looks good. You are getting there.
>>> Do add a link to your wiki in a visible place. This repo will be your
>>> devel repo. I will clone it asap and check while you develop.
>>>
>>> Check the makefiles in signal and image, they simplify the process a lot
>>> https://sourceforge.net/p/octave/signal/ci/default/tree/Makefile
>>>
>>> As I mentioned before, there are many functions in the package that
>>> should be deprecated and use Octave's own implementation instead. For
>>> example random numbers with gausssian distributions. Do not use
>>> Tisean's, those are bad and we already have randn and mvrand
>>> (hopefully using the mersenne twister).
>>>
>>> After you provide a makefile for what you have done so far, please
>>> freeze the code and provide a plan (in wiki, and comment with link in
>>> your application in melange). The plan should contain a list with the
>>> functions you will work on (your choice was Nonlinear noise reduction,
>>> add it to your plan!), specifying which one you will use octave's (and
>>> maybe wrapper to avoid re-coding), which one you will re-write from
>>> scratch (in C++ or mFile) and which one will be linked with directly
>>> via octave API, as you have done now.
>>>
>>> After this just wait for the evaluation. Remember this year we are
>>> tight with students slots. If things go sad and you are not taken for
>>> GSoC, please do not be discouraged. It would be great if we kept
>>> working on TISEAN and try next year. But is up to you, of course. Lets
>>> see how it goes.
>>>
>>> Cheers
>>
>>
>> Juan,
>>
>> I have written out my plan on my elaborated proposal wiki page:
>> http://wiki.octave.org/User:Josiah425:TISEAN_Package
>>
>> I have also added a makefile to my bitbucket and added a link on melange.
>>
>> The reason I linked to 'addnoise' is because it was something easy I could
>> start with. I see that octave has a lot of different functions that are
>> better that what TISEAN has in regards of random numbers. I will try to use
>> those instead from now.
>>
>> :)
>>
>>
>
> Good work Piotr!
>
> I give you some help for the next phase: To know which TISEAN's
> functions can be ignored.
> Remember that you are helping the community so paving the road is as
> important as taking the trip.
> A good roadmap will help you and the people after you.
>
> == TISEAN's function has the same name as na Octave's function ==
>
> Find if the name exist and what it does by runnning the following
> command in Octave:
>
> help <function-name>
>
> If the function name exists you will get the help for it and you can
> check if it does what TISEAN needs. If it does the same, then you can
> ignore that function from TISEAN, and just need to replace all
> occurrences of TISEAN's by Octave's (other ideas welcomed).
>
> In your function list (ideally in the wiki) you can mark this as
> IGNORE and provide Octave's function name (so far we are checking on
> core...we will discuss packages below)
>
> == TISEAN's func. is named differently but there is an Octave's func
> that does the same thing ==
>
> To find if octave has the functionality under a different name you can
> use keywords and use lookfor. For example, lets find what is there
> about correlations
>
> lookfor -all correlation
>
> Lets say you want to check for a keyword key1 then you do
>
> lookfor -all key1
>
> If you find a potentially matching function you can check what it does
> with help. If it does what you need (and since we tested matching name
> in the previous search, this function doesn't have the same name as it
> TISEAN equivalent) we need to write a wrapper.
>
> You can mark this function in the list as WRAP and give the
> corresponding octave's function.
>
> == Octave forge packages ==
> The full list of functions is here
> http://octave.sourceforge.net/function_list.html
> So you can search for matching names without installing and loading the 
> package.
>
> To check for functionality beyond name I would just look in a set of
> selected packages: general, miscellaneous, specfun, linear-algebra,
> control, tsa, optim, optiminterp, statistics, signal.
> You can install all this packages with
>
> pkg install -forge general, miscellaneous, specfun, linear-algebra,
> control, tsa, optim, optiminterp, statistics, signal
>
> Then load them
>
> pkg load all
>
> and then use the lookfor instructions given above.
>
>
> Ok, you do not need to do this for the whole TISEAN, you can just do
> it for the part you are working on. However the list of all TISEAN
> programs (This is not the full list of functions, but we can build the
> call tree from here) is in the folder
> Tisean_3.0.1/docs/alphabetical.html and it might be good to have it
> before starting.

Piotr,

Here is some information about the call tree of TISEAN's functions
written in C. Use it to produce your roadmap.
http://wiki.octave.org/User:KaKiLa#TISEAN