stats._stats_mstats_common

Module Contents

Functions

linregress(x,y=None) Calculate a linear least-squares regression for two sets of measurements.
theilslopes(y,x=None,alpha=0.95) r
_find_repeats(arr)
linregress(x, y=None)

Calculate a linear least-squares regression for two sets of measurements.

x, y : array_like
Two sets of measurements. Both arrays should have the same length. If only x is given (and y=None), then it must be a two-dimensional array where one dimension has length 2. The two sets of measurements are then found by splitting the array along the length-2 dimension.
slope : float
slope of the regression line
intercept : float
intercept of the regression line
rvalue : float
correlation coefficient
pvalue : float
two-sided p-value for a hypothesis test whose null hypothesis is that the slope is zero, using Wald Test with t-distribution of the test statistic.
stderr : float
Standard error of the estimated gradient.
scipy.optimize.curve_fit() : Use non-linear
least squares to fit a function to data.
scipy.optimize.leastsq() : Minimize the sum of
squares of a set of equations.
>>> import matplotlib.pyplot as plt
>>> from scipy import stats
>>> np.random.seed(12345678)
>>> x = np.random.random(10)
>>> y = np.random.random(10)
>>> slope, intercept, r_value, p_value, std_err = stats.linregress(x, y)

To get coefficient of determination (r_squared)

>>> print("r-squared:", r_value**2)
r-squared: 0.080402268539

Plot the data along with the fitted line

>>> plt.plot(x, y, 'o', label='original data')
>>> plt.plot(x, intercept + slope*x, 'r', label='fitted line')
>>> plt.legend()
>>> plt.show()
theilslopes(y, x=None, alpha=0.95)

r Computes the Theil-Sen estimator for a set of points (x, y).

theilslopes implements a method for robust linear regression. It computes the slope as the median of all slopes between paired values.

y : array_like
Dependent variable.
x : array_like or None, optional
Independent variable. If None, use arange(len(y)) instead.
alpha : float, optional
Confidence degree between 0 and 1. Default is 95% confidence. Note that alpha is symmetric around 0.5, i.e. both 0.1 and 0.9 are interpreted as “find the 90% confidence interval”.
medslope : float
Theil slope.
medintercept : float
Intercept of the Theil line, as median(y) - medslope*median(x).
lo_slope : float
Lower bound of the confidence interval on medslope.
up_slope : float
Upper bound of the confidence interval on medslope.

The implementation of theilslopes follows [1]. The intercept is not defined in [1], and here it is defined as median(y) - medslope*median(x), which is given in [3]. Other definitions of the intercept exist in the literature. A confidence interval for the intercept is not given as this question is not addressed in [1].

[1](1, 2, 3) P.K. Sen, “Estimates of the regression coefficient based on Kendall’s tau”, J. Am. Stat. Assoc., Vol. 63, pp. 1379-1389, 1968.
[2]H. Theil, “A rank-invariant method of linear and polynomial regression analysis I, II and III”, Nederl. Akad. Wetensch., Proc. 53:, pp. 386-392, pp. 521-525, pp. 1397-1412, 1950.
[3]W.L. Conover, “Practical nonparametric statistics”, 2nd ed., John Wiley and Sons, New York, pp. 493.
>>> from scipy import stats
>>> import matplotlib.pyplot as plt
>>> x = np.linspace(-5, 5, num=150)
>>> y = x + np.random.normal(size=x.size)
>>> y[11:15] += 10  # add outliers
>>> y[-5:] -= 7

Compute the slope, intercept and 90% confidence interval. For comparison, also compute the least-squares fit with linregress:

>>> res = stats.theilslopes(y, x, 0.90)
>>> lsq_res = stats.linregress(x, y)

Plot the results. The Theil-Sen regression line is shown in red, with the dashed red lines illustrating the confidence interval of the slope (note that the dashed red lines are not the confidence interval of the regression as the confidence interval of the intercept is not included). The green line shows the least-squares fit for comparison.

>>> fig = plt.figure()
>>> ax = fig.add_subplot(111)
>>> ax.plot(x, y, 'b.')
>>> ax.plot(x, res[1] + res[0] * x, 'r-')
>>> ax.plot(x, res[1] + res[2] * x, 'r--')
>>> ax.plot(x, res[1] + res[3] * x, 'r--')
>>> ax.plot(x, lsq_res[1] + lsq_res[0] * x, 'g-')
>>> plt.show()
_find_repeats(arr)