spkit.fill_nans_2d

spkit.fill_nans_2d(X, pkind='linear', filter_size=3, method='conv', clip_range=[None, None], restore_original_values=True)

Fill nan values with interpolation/exterpolation for 2D

Fill nan values with interpolation/exterpolation for 2D

This function applies

1. Inter/exter-polation to estimate missing values

2. Smooting with filter_size, if filter_size>0

3. Restore original values, if restore_original_values True

4. Finally, clipping values if clip_range is provided.

This function uses ‘fill_nans_1d’ for each column and each row. This results two inter/exter-polated values for each missing value

To fill the missing value, function takes average of both, which reduces the variability along both axis.

Further to remove any artifacts created by new values, smoothing is applied. However, original values are restored.

Finally, if clip_range is passed, values in new matrix are clipped using it.

Parameters:

X: 2d-array

• array with missing values, denoted by np.nan

pkindkind of interpolation, default=’linear

• one of {‘linear’, ‘nearest’, ‘nearest-up’, ‘zero’, ‘slinear’, ‘quadratic’, ‘cubic’, ‘previous’, or ‘next’. ‘zero’}

filter_size: int

• A 2d-filter size to apply smoothing, a kernel of filter_size x filter_size

• is created and convoled with matrix

method: str, {‘conv’, ‘conv_nan’}, default = ‘conv’

• convolution method

• if method=’conv’, conventional convolution is applied, in this case,

• ‘filter_size’ can be of any length>1

• if method=’conv_nan’, a convolution operation, that can handle NaN values is used.

  For this, filter_size should be an odd number, if even number is passed, 1 is added to make it odd

restore_original_values: bool, default =True

• if True, original values are restored

clip_range: list of two [l0,l1]

• After (1) inter-exter-polation, (2) applied smotthing,(3) restoring original values

• matrix values are clipped with clip_range.

• This is only applied if at least of the clip_range values is not None.

Returns:

XINew Matrix,

• where NaN are filled, but original values are left un-changed, except clipping

XkNew Matrix,

• same as XI, except, original values are not restored, or clipped.

See also

conv2d_nan, fill_nans_1d

Examples

>>> import numpy as np
>>> import spkit as sp
>>> np.random.seed(seed=2)
>>> X  = np.random.randint(0,10,[5,5])
>>> r = 1*(np.abs(np.random.randn(5,5))<1.4).astype(float)
>>> r[r==0]=np.nan
>>> X_nan = X*r
>>> print(X_nan)

array([[ 8.,  8.,  6.,  2., nan],
       [ 7.,  2., nan,  5.,  4.],
       [nan,  5.,  7.,  3.,  6.],
       [ 4.,  3.,  7.,  6.,  1.],
       [nan,  5.,  8.,  4., nan]])

>>> X_filled, X_smooth = sp.fill_nans_2d(X_nan)
>>> print(X_filled.round(1))

array([[8. , 8. , 6. , 2. , 1.9],
       [7. , 2. , 4.8, 5. , 4. ],
       [4.5, 5. , 7. , 3. , 6. ],
       [4. , 3. , 7. , 6. , 1. ],
       [3.3, 5. , 8. , 4. , 0.9]])

>>> #####################################
>>> #sp.fill_nans_2d
>>> import numpy as np
>>> import matplotlib.pyplot as plt
>>> import seaborn as sns
>>> import scipy
>>> import spkit as sp
>>> seed = 2
>>> I = (sp.create_signal_2d(n=10,seed=seed,sg_winlen=3)*10).round(0)
>>> np.random.seed(seed)
>>> r = np.random.rand(*I.shape)
>>> I[r<0.05] = np.nan
>>> np.random.seed(None)
>>> kernel = np.ones([2,2])/4
>>> I_spkit = sp.conv2d_nan(I.copy(), kernel, boundary='reflect',fillvalue=0)
>>> I_fill, I_sm  = sp.fill_nans_2d(I.copy())
>>> plt.figure(figsize=(12,4))
>>> plt.subplot(131)
>>> sns.heatmap(I, annot=True,cbar=False,xticklabels='', yticklabels='')
>>> plt.title('Matrix with NaNs')
>>> plt.subplot(132)
>>> sns.heatmap(I_fill, annot=True,cbar=False,xticklabels='', yticklabels='')
>>> plt.title('Filling NaNs with \n (sp.fill_nans_2d)')
>>> plt.subplot(133)
>>> sns.heatmap(I_spkit, annot=True,cbar=False,xticklabels='', yticklabels='')
>>> plt.title('Smoothing with \n (sp.conv2d_nan)')
>>> plt.tight_layout()
>>> plt.show()

Examples using spkit.fill_nans_2d

Release Highlights for spkit 0.0.9.7

Release Highlights for spkit 0.0.9.7