Talk:Recursive least squares filter

That was good fun... The derivation of the RLS algorithm is a bit lengthy. I went for a clear instead of a brief description. This work needs some proofreading. Everybody is more then welcome. Faust o 16:22, 22 January 2006 (UTC)

Badly written introductory sentence

This sentence is misleading: "The benefit of the RLS algorithm is that there is no need to invert matrices, thereby saving computational power."

However, the final solution to the RLS algorithm includes a matrix inversion. So this sentence is just wrong, or I am misunderstanding something about the final solution. I came here to understand more about RLS, but I'm left more confused. I assume the point is that n >> p, so that there is some benefit to iterating over n items with a p order filter. However this is never explicitly stated in the introduction or in the summary of the algorithm (which is what I would want to read and understand before going through a proof). Maybe this sentence could be changed to read "The benefit of the RLS algorithm is that we need only compute many small matrix inversions, rather than one larger matrix inversion, thereby saving computational power." However, not fully understanding RLS I didn't want to change this, since I don't know if it's accurate. — Preceding unsigned comment added by DeverLite (talk • contribs) 17:03, 15 October 2013 (UTC)

Summary Question

The calculation of e(n) and w(n) are interdependent. Should e(n) be calculated at e(n) = d(n) - wt(n-1) x(n)?

Yeah there are some errors. First, we calculate e(n) according to W(n) and then we update W(n) using W(n-1) >.< This needs some correction Touriste 10:39, 10 August 2007 (UTC)

There was just a mistake in the a priori estimate error. It's right now Touriste (usurped) 11:08, 10 August 2007 (UTC)

What is d(n)

¿What is 'd(n)'?. Text article doesn't mention it. 163.117.150.178 13:47, 13 July 2007 (UTC)

It represents the optimum signal (the one you are expecting to get closed to) but it's true a block diagram is missing :( Touriste (usurped) 10:51, 10 August 2007 (UTC)

It is the desired signal. I added the block diagram and an explanatory sentence.--Adoniscik (talk) 22:53, 3 January 2008 (UTC)

According to differernt system ,the corresponding desired signal is differernt. — Preceding unsigned comment added by 138.4.34.234 (talk) 15:39, 28 October 2011 (UTC)

Problems with the motivation and discussion sections

There seems to be some discussion about adaptive filters in general missing. For example, the discussion starts out with defining an optimization problem in e(n), but never defines e(n). Also, as mentioned above, d(n) is introduced in a sentence saying something like "by the definition of e(n)". This definition, again, isn't there.
Also, I think some discussion about the numerical stability properties (problems) of the RLS algorith is needed. And perhaps also some discussion about the choice of lambda.

The motivation is generally quite weak. Firstly, the formula y(n + 1) = wx(n) + e(n) doesn't make any sense unless w is a scalar, secondly, it is not stated which signals are known an which are not, thirdly, there are several more motivations for adaptive filters in general and RLS in particular. And the list really goes on here... The motivation should probably contain the motivation for adaptive filters in general, the motivation for using Least Squares methods for estimating optimal filters, and the motivation for making the Least Squares method recursive.
152.94.13.40 11:52, 12 October 2007 (UTC)

It's there now. That wx(n) was a typo. What is meant was a FIR filter 'w' that is a function of the input x(n). I rephrased it, hopefully dispelling the confusion.--Adoniscik (talk) 22:54, 3 January 2008 (UTC)

In the Motivation, it says "The benefit of the RLS algorithm is that there is no need to invert matrices" - but the computation of the gain g(n) clearly indicates a matrix inversion. Did you mean the LMS algorithm? —Preceding unsigned comment added by 192.150.10.200 (talk) 15:43, 24 January 2011 (UTC)

What is $x^{*}$?

What in the world are $e^{*}$, $w^{*}$, and $x^{*}$? Whatever this operator is, it should be mentioned in the derivation. —Preceding unsigned comment added by 76.175.175.2 (talk) 00:30, 6 April 2009 (UTC)

It means complex conjugation, but I really think it should be removed. Firstly, it is entirely OK to assume that the signal is real in the context of wikipedia, and rather refer the reader to the references for the (rarely used in practice) complex case. Secondly, the motivation part seems to assume that the filter is real, which makes it weird to later use complex variables (the article never actually mentions the word "complex"). Thirdly, this article uses conjugation in a very strange and unconventional way -- it keeps using real transposition of complex vectors, and conjugation without transposition of the same vectors - both quite rarely seen operations. For example, this article ends up with an outer product of a complex vector which is $x^{*}x^{T}$, instead of the conventional $xx^{H}$.

Overall, I think the article is still kind of a mess. Perhaps the whole motivation is best left to the "adaptive filter" article, while this article could explain the cost function and the resulting algorithm, without going into details about its derivation (or at least leaving that until the very end). 07:32, 23 May 2010 (UTC) —Preceding unsigned comment added by Short rai (talk • contribs)

I made several changes, revert them if you will.

1. I changed the introduction a bit. "Adaptive filter" usually refers to the whole algorithm, meaning RLS is an adaptive filter. So I found it strange that it said something along the lines of "RLS is used with adaptive filters".

2. I made some small cosmetic changes to the "motivation" part. However, this part makes no sense as it is: from the setup it is clear that one wants d(n), but the algorithm requires that we know d(n) so it is stupid to use RLS to estimate it from x(n). Yes, I am aware that this is similar to what one actually uses adaptive filters for, but the name "desired signal" for d(n) is usually misleading: One knows d(n) but wants to find d(n)-v(n), or simply v(n). In this example there doesn't seem to be any reason why one could not use d(n) as it is. Anyone willing to replace with a more sensible application?

3. I removed all the complex-specific stuff (all the conjugation, basically). It was all mangled, and there was no motivation for it.

I also think maybe the algorithm summary should come before all the horrible derivation that nobody is going to read through anyway :) Short rai (talk) 08:58, 23 May 2010 (UTC)

x(i) for negative times

In many formulas, for example in calculating the partial derivatives of cost, for the little values of i (when i<k or i<l) x(i-l) or x(i-k) refer to a negative time. I think it's fine to define them as zero, which should be mentioned in the description. Otherwise the lower limit of the summation should be changed to something, perhaps like min{k,l}. --A.joudaki (talk) 11:24, 21 December 2010 (UTC)

Clarity in terms

Maybe a slight mention the b term and convolution delay (from the echo) in the first equation? The d and v terms are covered, for the sake of clarity, why not describe the only remaining term?? edit: My mistake. It's a delay, not a convolution.

 — Preceding unsigned comment added by Brandon.irwin (talk • contribs) 17:53, 20 June 2011 (UTC) 

Applications?

We should mention some example applications in the lead of the article, to give the reader as idea of what problems the method is useful for, which also make it easier to understand how the method works as you more easily can put everything into context. Having skimmed through article, I am still a bit confused about what the method does or what it is useful for. —Kri (talk) 15:14, 1 August 2016 (UTC)

Inverses Exist?

The main analysis just assumes that several matrix inverses exist. For example, where the Woodbury matrix identity is used. In reality, this may not be the case. Certainly, when the recursion is getting started and there is not much data, the given matrix will not be invertible. Therefore the analysis cannot be correct. This needs to be discussed. — Preceding unsigned comment added by Austrartsua (talk • contribs) 21:52, 22 November 2021 (UTC)

Lattice sections notation is not in line with previous sections

Is v same as w before? Is N same as p, number of filter taps? These sections could be very useful, but need more detailed explanations. — Preceding unsigned comment added by Lukistrela (talk • contribs) 17:53, 27 July 2020 (UTC)

I just added a reference (the Diniz book) regarding the benefits of the lattice implementation. This should be a very useful reference if someone feels motivated to overhaul the whole section (which I agree is necessary). (Do I understand correctly that the lattice filters are not applicable to the online case, because they require iterating forward and backward multiple times? So when would one actually want to use them? As efficient large linear system solvers?) --2A02:8108:84C0:A24:4419:1EDD:F1FE:EF07 (talk) 10:31, 23 April 2021 (UTC)

Initialisation up to -p not reached

In "RLS algorithm summary", the initialisation sets x(-1) down to x(-p) but then the vector is formed with x(n) down to x(n-p) for n=1,2,... so it never reaches down to x(-p). I think something is off (by one) here.

Please note I updated the initialisation where I was certain; x(n) could not be initialised without establishing n; so I changed that to x(0).

Having seen the other problems, I am now suspecting that you want to iterate n = 0,1,2,... so d(n) also starts at 0.

In that case, the initialisation for x(0) and P(0) need to change to x(-1) and P(-1). But that's from reading the page alone, without referencing literature in the algorithm. Merely discussing the disparities. — Preceding unsigned comment added by 2001:980:93A5:1:6664:82C6:273C:593A (talk) 17:56, 21 February 2022 (UTC)