Question:
numpy returns nan . If there are two different arrays, then nan fly out:
np.corrcoef([0.1, 0.1, 0.1], [0.9, 0.9, 0.9])
array([[ 1., nan],
[ nan, nan]])
If two arrays are the same, then this does not happen:
np.corrcoef([0.1, 0.1, 0.1], [0.1, 0.1, 0.1])
array([[ 1., 1.],
[ 1., 1.]])
It is also not clear why corrcoef all nan for zeros:
np.corrcoef([0, 0, 0], [0.1, 0.1, 0.1])
array([[ nan, nan],
[ nan, nan]])
Answer:
If we simulate what happens inside the np.corrcoef() function :
In [37]: x=[0.1, 0.1, 0.1]; y=[0.9,0.9,0.9]
# расчет ковариционной матрицы
In [38]: c = np.cov(x,y)
# для указанных значений - она имеет нулевые значения для 3-х из 4-х элементов
# в результате `np.corrcoef()` на месте этих элементов будут стоять NaN
In [39]: c
Out[39]:
array([[ 2.88889492e-34, 0.00000000e+00],
[ 0.00000000e+00, 0.00000000e+00]])
In [40]: d = np.diag(c)
In [41]: d
Out[41]: array([ 2.88889492e-34, 0.00000000e+00])
In [42]: stddev = np.sqrt(d.real)
In [43]: stddev
Out[43]: array([ 1.69967494e-17, 0.00000000e+00])
In the next line, we get NaN as the result of division by zero:
In [44]: c /= stddev[:, None]
...\py36\Scripts\ipython3:1: RuntimeWarning: invalid value encountered in true_divide
In [45]: stddev[:, None]
Out[45]:
array([[ 1.69967494e-17],
[ 0.00000000e+00]])
In [46]: c
Out[46]:
array([[ 1.69967494e-17, 0.00000000e+00],
[ nan, nan]])
The way around this is to add a very small number to one or more elements of the second array:
In [109]: x=[0.1, 0.1, 0.1]; y=[0.9,0.9,0.9+1e-16]
In [110]: np.corrcoef(x,y)
Out[110]:
array([[ 1. , -0.57735027],
[-0.57735027, 1. ]])
or
In [111]: x=np.array([0.1, 0.1, 0.1]); y=np.array([0.9,0.9,0.9])+1e-16
In [112]: np.corrcoef(x,y)
Out[112]:
array([[ 1., -1.],
[-1., 1.]])