Recently, I am very interested in Mutual Information Estimator, especically, KSG estimator. However, I always concern that I haven’t fully understand the KSG estimator, i.e.

$$I^{2}(X, Y) = \psi(k) - 1/k - <\psi(n_x) + \psi(n_y)> + \psi(N),$$

where $n_x$ and $n_y$ by the number of points with $|x_i - x_j| \leq \epsilon_{x}(i)$ and $|y_i - y_j| \leq \epsilon_{y}(i)$. (For more detail, see Estimating mutual information)

The key loop for computing $I^{2}(X, Y)$ are demonstrated in the following. (The all code can be view at github)

1 | // update KSG estimator component : $<\psi(n_x) + \psi(n_y)>$: |