spkit.show_farmulas

spkit.show_farmulas()

Usuful Formulas



Usuful Formulas


Differential Entropy of Normally distribuated Multivariant X

for .. math:: x ∼ N(μ,Σ)

entropy in nats

H(x)=(1/2)ln|Σ|+(1/2)n+(n/2)ln(2π)

(1/2)n + (n/2)ln(2π) => are constant values for fixed dimension

code:

H_x = entropy_differential(x,is_multidim=True, emb_dim=1, delay=1,)

Self-Conditional Entropy

Information of X(i+1) given X(i)

H(Xi+1|Xi)=H(Xi+1,Xi)H(Xi)

using: H(X|Y)=H(X,Y)H(Y)

code:

H_x1x = entropy_diff_cond_self(X, present_first=True)

Conditional Entropy

Information of X(i+1) given X(i) and Y(i)

H(Xi+1|Xi,Yi)=H(Xi+1,Xi,Yi)H(Xi,Yi)

code::

H_x1xy = entropy_diff_cond(X,Y,present_first=True)

Joint Entropy

H(X,Y)

code:

H_xy = entropy_diff_joint(X,Y)

Joint-Conditional Entropy

H(Xi+1,Yi+1|Xi,Yi)=H(Xi+1,Yi+1,Xi,Yi)H(Xi,Yi)

code:

H_xy1xy = entropy_diff_joint_cond(X,Y,present_first=True)

Self Mutual Information

Predictibility of X(i+1) given X(i)

I(Xi+1;Xi)=H(Xi+1)H(Xi+1|Xi)

code:

I_xx = mutual_info_diff_self(X,present_first=True)

Mutual Information

Predictibility of X(i+1) given X(i) and Y(i)

I(Xi+1;Xi,Yi)=H(Xi+1)H(Xi+1|Xi,Yi)

H(Xi+1|Xi,Yi)=H(Xi+1,Xi,Yi)H(Xi,Yi)

code:

I_xy = mutual_info_diff(X,Y,present_first=True)

Transfer Entropy

TEX>Y=I(Yi+1,Xi|Yi)

TEX>Y=H(Yi+1|Yi)H(Yi+1|Xi,Yi) [Eq1]

TEX>Y=H(Yi+1,Yi)H(Yi)H(Yi+1,Xi,Yi)+H(Xi,Yi)

TEX>Y=H(Xi,Yi)+H(Yi+1,Yi)H(Yi+1,Xi,Yi)H(Yi) [Eq2]

Using: H(Xi+1|Xi)=H(Xi+1,Xi)H(Xi) | entropy_diff_cond_self(X) H(Xi+1|Xi,Yi)=H(Xi+1,Xi,Yi)H(Xi,Yi) | entropy_diff_cond(X,Y)

code:

TE_x2y = transfer_entropy(X,Y,present_first=True)

Partial Transfer Entropy Or Conditional Transfer Entopry

TEX>Y|Z=I(Yi+1,Xi|Yi,Zi)

TEX>Y|Z=H(Xi,Yi,Zi)+H(Yi+1,Yi,Zi)H(Yi+1,Xi,Yi,Zi)H(Yi,Zi)

code:

TE_x2y1z = partial_transfer_entropy(X,Y,Z,present_first=True,verbose=False)

Granger Causality based on Differential Entropy

  1. GC_XY (X–>Y) : H(Yi+1|Yi)H(Yi+1|Xi,Yi)

  2. GC_YX (Y–>X) : H(Xi+1|Xi)H(Xi+1|Xi,Yi)

  3. GC_XdY (X.Y) : H(Yi+1|Xi,Yi)+H(Xi+1|Xi,Yi)H(Xi+1,Yi+1|Xi,Yi)

if normalize True

GCXY=GCXY/(I(Yi+1;Yi)+GCXY) GCYX=GCYX/(I(Xi+1;Xi)+GCYX)

Using:: H(Yi+1|Yi)=H(Yi+1,Yi)H(Yi) H(Xi+1|Xi)=H(Xi+1,Xi)H(Xi) H(Yi+1|Xi,Yi)=H(Yi+1,Xi,Yi)H(Xi,Yi) H(Xi+1|Xi,Yi)=H(Xi+1,Xi,Yi)H(Xi,Yi) H(Xi+1,Yi+1|Xi,Yi)=H(Xi+1,Yi+1,Xi,Yi)H(Xi,Yi)

I(Xi+1;Xi)=H(Xi+1)H(Xi+1|Xi):math:I(Yi+1;Yi)=H(Yi+1)H(Yi+1|Yi)

code:

gc_xy, gc_yx,gc_xdy = entropy_granger_causality(X,Y,present_first=True, normalize=False)