
From:  Jonathan Stickel 
Subject:  Re: Solving large matrix equations 
Date:  Tue, 07 Apr 2009 07:55:51 0600 
Useragent:  Thunderbird 2.0.0.21 (Macintosh/20090302) 
On 4/7/09 address@hidden wrote:
Date: Tue, 7 Apr 2009 09:55:57 +0200 From: "Martin Heller" <address@hidden> Subject: Solving large matrix equations To: address@hidden MessageID: <address@hidden>I am new to Octave and am trying to write a function for differentiating somedata based on this paper: <http://math.lanl.gov/Research/Publications/Docs/chartrand2007numerical.pdf> My attempt is shown below and it seems to work but my data sets contain 1000015000 points which is too much for Octave to handle when solving the problem using my simple minded approch. I was wondering if there is a standard way to solve large matrix problems in Octave that I could take advantage of? At the moment I just feed smaller chunks of data to my function and this gets the job done.
MartinI think you will need to provide more details to have your general question answered: the appropriate method for solving large matrix equations of course depends on the system. What do you mean when you say your system is "too much for Octave to handle"? Are you running out of memory? Does it take too long? There are people much more qualified to help you with these general issues than I am.
However, I have a particular interest in your application. I have studied data smoothing and differentiation quite a bit in the last year and have recently submitted a paper on the subject (I can share offlist if you like). Is your "real data" (not your example test data) discontinuous or does it have a discontinuous derivative? If not, then you likely can use a squared term for your roughness rather than the absolute value. This case has an analytical solution that does not require an iterative method (Eilers P.H.C, 2003, Anal. Chem. 75, 36313636). I have implemented this method and provided it in the Octave package datasmoothing (http://octave.sourceforge.net/datasmoothing/index.html). However, I have found that my implementation is often illconditioned for data with more than ~1000 points.
At a first glance, it looks like you have made a good start with Chartrand's method. Would you consider submitting your function to OctaveForge? I think it would make a nice addition to the datasmoothing package.
Regards, Jonathan
[Prev in Thread]  Current Thread  [Next in Thread] 