[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
## Re: OT: finding the weights used in weighted least squares regression

**From**: |
Mike Miller |

**Subject**: |
Re: OT: finding the weights used in weighted least squares regression |

**Date**: |
Thu, 28 Apr 2011 11:56:02 -0500 (CDT) |

**User-agent**: |
Alpine 2.00 (DEB 1167 2008-08-23) |

On Tue, 26 Apr 2011, Kamaraju S Kusumanchi wrote:

The weighted least squares regression is done by solving
W A x = W b
for x, where W is nxn, A is nxm, x is mx1, b is nx1. W is a diagonal matrix.

`My question is that, in the above set up, if we are given A, x, b is it
``possible to solve for the W?
`

We usually have an equation of this form:
y = X*b + e

`The y vector (nx1) and X matrix (nxp) are given and we want to solve for b
``(px1) attempting to minimize the variance of hypothetical e (nx1) -- the
``least squares solution, but we assume that the variance-covariance matrix
``of e is of the form eye(n)*s^2 for some s. We can estimate s.
`

`But what if the var-covar matrix of e is of a different form? Let's give
``it the general form V (nxn, not necessarily diagonal) where V is symmetric
``non-negative definite. It turns out that if V is positive definite (all
``positive eigenvalues), you can produce a matrix U=chol(inv(V)) such that
`
U*y = U*X*b + U*e

`where U*e has var-covar matrix eye(n). This makes it possible to get a
``proper solution for b like so:
`
b = (U*X)\(U*y)

`I assume that your "A x = b" is what I would call "X b = y". In other
``words, I think the W you want is the U I've given.
`

`This means that you have to know something about the structure of your
``residuals. Are there groups of observations that allow you to estimate
``residual variances for the elements of e? If so, you can use an iterative
``feasible generalized least squares method:
`
(1) b = X\y ; [implicitly assumes cov(e) = eye(n)*s^2]
(2) eOLS = y - X*b ; [compute residuals]

`(3) compute var-covar matrix V based on eOLS -- I can't tell you how
``because I don't know the structure of your data.
`
(4) U = chol(inv(V)) ;
(5) b = (U*X)\(U*y) ;
(6) eGLS = y - X*b ;

`(7) compute var-covar matrix V based on eGLS -- again, I can't tell you
``how because I don't know the structure of your data.
`

`(8) repeat steps 4-7 until b stops changing. It should not require many
``iterations.
`
Mike
--
Michael B. Miller, Ph.D.
Bioinformatics Specialist
Minnesota Center for Twin and Family Research
Department of Psychology
University of Minnesota

**OT: finding the weights used in weighted least squares regression**, *Kamaraju S Kusumanchi*, `2011/04/26`
**Re: OT: finding the weights used in weighted least squares regression**, *Ben Abbott*, `2011/04/26`
**Re: OT: finding the weights used in weighted least squares regression**, *Kamaraju S Kusumanchi*, `2011/04/27`
**Re: OT: finding the weights used in weighted least squares regression**, *James Sherman Jr.*, `2011/04/27`
**Re: OT: finding the weights used in weighted least squares regression**, *Kamaraju S Kusumanchi*, `2011/04/27`
**Re: OT: finding the weights used in weighted least squares regression**, *Ben Abbott*, `2011/04/27`
**Re: OT: finding the weights used in weighted least squares regression**, *Kamaraju S Kusumanchi*, `2011/04/27`
**Re: OT: finding the weights used in weighted least squares regression**, *Ben Abbott*, `2011/04/27`
**Re: OT: finding the weights used in weighted least squares regression**, *Kamaraju S Kusumanchi*, `2011/04/27`

**Re: OT: finding the weights used in weighted least squares regression**,
*Mike Miller* **<=**