I’ve been fiddling around in python, implementing a Variational Bayesian Principal Component Analysis (VBPCA) algorithm. It’s true, I could do this happily in vibes , but where’s the fun in that? Besides, I think I learned more doing it this way.

Happily, there’s a nice paper by Chris Bishop, which explains clearly what’s going on, and gives you the update equations. For example:

which involves taking the expected value of , and . These three are straightforward:

The distribution for is , and so the expected value of is .

The distribution for is , and so the expected value of is simply .

The distribution for is a slightly more complex beast: is a non-square matrix, and we have a Gaussian distribution for each *row*. That is: . Bizarrely, the rows share a covariance matrix. The expected value of is simple: is just a matrix made of a stack of s.

The tricky bit comes in a different update equation, where we need to evaluate . The first thing to notice is that (where is a d by q matrix):

.

Since is Gaussian, . Now we can write:

Simple when you know how.

Edit: Anyone know how to get a bold ? I’ve tried {\bf \mu} and \mathbf{\mu}.