package sklearn

Get an attribute of this module as a Py.Object.t. This is useful to pass a Python function to another function.

Compute elastic net path with coordinate descent.

The elastic net optimization function varies for mono and multi-outputs.

For mono-output tasks it is::

1 / (2 * n_samples) * ||y - Xw||^2_2

1. alpha * l1_ratio * ||w||_1
2. 0.5 * alpha * (1 - l1_ratio) * ||w||^2_2

For multi-output tasks it is::

(1 / (2 * n_samples)) * ||Y - XW||^Fro_2

1. alpha * l1_ratio * ||W||_21
2. 0.5 * alpha * (1 - l1_ratio) * ||W||_Fro^2

Where::

||W||_21 = \sum_i \sqrt\sum_j w_{ij^2

}

i.e. the sum of norm of each row.

Read more in the :ref:`User Guide <elastic_net>`.

Parameters ---------- X : array-like, sparse matrix of shape (n_samples, n_features) Training data. Pass directly as Fortran-contiguous data to avoid unnecessary memory duplication. If ``y`` is mono-output then ``X`` can be sparse.

y : array-like, sparse matrix of shape (n_samples,) or (n_samples, n_outputs) Target values.

l1_ratio : float, default=0.5 Number between 0 and 1 passed to elastic net (scaling between l1 and l2 penalties). ``l1_ratio=1`` corresponds to the Lasso.

eps : float, default=1e-3 Length of the path. ``eps=1e-3`` means that ``alpha_min / alpha_max = 1e-3``.

n_alphas : int, default=100 Number of alphas along the regularization path.

alphas : ndarray, default=None List of alphas where to compute the models. If None alphas are set automatically.

precompute : 'auto', bool or array-like of shape (n_features, n_features), default='auto' Whether to use a precomputed Gram matrix to speed up calculations. If set to ``'auto'`` let us decide. The Gram matrix can also be passed as argument.

Xy : array-like of shape (n_features,) or (n_features, n_outputs), default=None Xy = np.dot(X.T, y) that can be precomputed. It is useful only when the Gram matrix is precomputed.

copy_X : bool, default=True If ``True``, X will be copied; else, it may be overwritten.

coef_init : ndarray of shape (n_features, ), default=None The initial values of the coefficients.

verbose : bool or int, default=False Amount of verbosity.

return_n_iter : bool, default=False Whether to return the number of iterations or not.

positive : bool, default=False If set to True, forces coefficients to be positive. (Only allowed when ``y.ndim == 1``).

check_input : bool, default=True Skip input validation checks, including the Gram matrix when provided assuming there are handled by the caller when check_input=False.

**params : kwargs Keyword arguments passed to the coordinate descent solver.

Returns ------- alphas : ndarray of shape (n_alphas,) The alphas along the path where models are computed.

coefs : ndarray of shape (n_features, n_alphas) or (n_outputs, n_features, n_alphas) Coefficients along the path.

dual_gaps : ndarray of shape (n_alphas,) The dual gaps at the end of the optimization for each alpha.

n_iters : list of int The number of iterations taken by the coordinate descent optimizer to reach the specified tolerance for each alpha. (Is returned when ``return_n_iter`` is set to True).

See Also -------- MultiTaskElasticNet MultiTaskElasticNetCV ElasticNet ElasticNetCV

Notes ----- For an example, see :ref:`examples/linear_model/plot_lasso_coordinate_descent_path.py <sphx_glr_auto_examples_linear_model_plot_lasso_coordinate_descent_path.py>`.

Compute Least Angle Regression or Lasso path using LARS algorithm 1

The optimization objective for the case method='lasso' is::

(1 / (2 * n_samples)) * ||y - Xw||^2_2 + alpha * ||w||_1

in the case of method='lars', the objective function is only known in the form of an implicit equation (see discussion in 1)

Read more in the :ref:`User Guide <least_angle_regression>`.

Parameters ---------- X : None or array-like of shape (n_samples, n_features) Input data. Note that if X is None then the Gram matrix must be specified, i.e., cannot be None or False.

y : None or array-like of shape (n_samples,) Input targets.

Xy : array-like of shape (n_samples,) or (n_samples, n_targets), default=None Xy = np.dot(X.T, y) that can be precomputed. It is useful only when the Gram matrix is precomputed.

Gram : None, 'auto', array-like of shape (n_features, n_features), default=None Precomputed Gram matrix (X' * X), if ``'auto'``, the Gram matrix is precomputed from the given X, if there are more samples than features.

max_iter : int, default=500 Maximum number of iterations to perform, set to infinity for no limit.

alpha_min : float, default=0 Minimum correlation along the path. It corresponds to the regularization parameter alpha parameter in the Lasso.

method : 'lar', 'lasso', default='lar' Specifies the returned model. Select ``'lar'`` for Least Angle Regression, ``'lasso'`` for the Lasso.

copy_X : bool, default=True If ``False``, ``X`` is overwritten.

eps : float, optional The machine-precision regularization in the computation of the Cholesky diagonal factors. Increase this for very ill-conditioned systems. By default, ``np.finfo(np.float).eps`` is used.

copy_Gram : bool, default=True If ``False``, ``Gram`` is overwritten.

verbose : int, default=0 Controls output verbosity.

return_path : bool, default=True If ``return_path==True`` returns the entire path, else returns only the last point of the path.

return_n_iter : bool, default=False Whether to return the number of iterations.

positive : bool, default=False Restrict coefficients to be >= 0. This option is only allowed with method 'lasso'. Note that the model coefficients will not converge to the ordinary-least-squares solution for small values of alpha. Only coefficients up to the smallest alpha value (``alphas_alphas_ > 0..min()`` when fit_path=True) reached by the stepwise Lars-Lasso algorithm are typically in congruence with the solution of the coordinate descent lasso_path function.

Returns ------- alphas : array-like of shape (n_alphas + 1,) Maximum of covariances (in absolute value) at each iteration. ``n_alphas`` is either ``max_iter``, ``n_features`` or the number of nodes in the path with ``alpha >= alpha_min``, whichever is smaller.

active : array-like of shape (n_alphas,) Indices of active variables at the end of the path.

coefs : array-like of shape (n_features, n_alphas + 1) Coefficients along the path

n_iter : int Number of iterations run. Returned only if return_n_iter is set to True.

See also -------- lars_path_gram lasso_path lasso_path_gram LassoLars Lars LassoLarsCV LarsCV sklearn.decomposition.sparse_encode

References ---------- .. 1 'Least Angle Regression', Efron et al. http://statweb.stanford.edu/~tibs/ftp/lars.pdf

.. 2 `Wikipedia entry on the Least-angle regression <https://en.wikipedia.org/wiki/Least-angle_regression>`_

.. 3 `Wikipedia entry on the Lasso <https://en.wikipedia.org/wiki/Lasso_(statistics)>`_

lars_path in the sufficient stats mode 1

The optimization objective for the case method='lasso' is::

(1 / (2 * n_samples)) * ||y - Xw||^2_2 + alpha * ||w||_1

in the case of method='lars', the objective function is only known in the form of an implicit equation (see discussion in 1)

Read more in the :ref:`User Guide <least_angle_regression>`.

Parameters ---------- Xy : array-like of shape (n_samples,) or (n_samples, n_targets) Xy = np.dot(X.T, y).

Gram : array-like of shape (n_features, n_features) Gram = np.dot(X.T * X).

n_samples : int or float Equivalent size of sample.

max_iter : int, default=500 Maximum number of iterations to perform, set to infinity for no limit.

alpha_min : float, default=0 Minimum correlation along the path. It corresponds to the regularization parameter alpha parameter in the Lasso.

method : 'lar', 'lasso', default='lar' Specifies the returned model. Select ``'lar'`` for Least Angle Regression, ``'lasso'`` for the Lasso.

copy_X : bool, default=True If ``False``, ``X`` is overwritten.

eps : float, optional The machine-precision regularization in the computation of the Cholesky diagonal factors. Increase this for very ill-conditioned systems. By default, ``np.finfo(np.float).eps`` is used.

copy_Gram : bool, default=True If ``False``, ``Gram`` is overwritten.

verbose : int, default=0 Controls output verbosity.

return_path : bool, default=True If ``return_path==True`` returns the entire path, else returns only the last point of the path.

return_n_iter : bool, default=False Whether to return the number of iterations.

positive : bool, default=False Restrict coefficients to be >= 0. This option is only allowed with method 'lasso'. Note that the model coefficients will not converge to the ordinary-least-squares solution for small values of alpha. Only coefficients up to the smallest alpha value (``alphas_alphas_ > 0..min()`` when fit_path=True) reached by the stepwise Lars-Lasso algorithm are typically in congruence with the solution of the coordinate descent lasso_path function.

Returns ------- alphas : array-like of shape (n_alphas + 1,) Maximum of covariances (in absolute value) at each iteration. ``n_alphas`` is either ``max_iter``, ``n_features`` or the number of nodes in the path with ``alpha >= alpha_min``, whichever is smaller.

active : array-like of shape (n_alphas,) Indices of active variables at the end of the path.

coefs : array-like of shape (n_features, n_alphas + 1) Coefficients along the path

n_iter : int Number of iterations run. Returned only if return_n_iter is set to True.

See also -------- lars_path lasso_path lasso_path_gram LassoLars Lars LassoLarsCV LarsCV sklearn.decomposition.sparse_encode

References ---------- .. 1 'Least Angle Regression', Efron et al. http://statweb.stanford.edu/~tibs/ftp/lars.pdf

.. 2 `Wikipedia entry on the Least-angle regression <https://en.wikipedia.org/wiki/Least-angle_regression>`_

.. 3 `Wikipedia entry on the Lasso <https://en.wikipedia.org/wiki/Lasso_(statistics)>`_

Compute Lasso path with coordinate descent

The Lasso optimization function varies for mono and multi-outputs.

For mono-output tasks it is::

(1 / (2 * n_samples)) * ||y - Xw||^2_2 + alpha * ||w||_1

For multi-output tasks it is::

(1 / (2 * n_samples)) * ||Y - XW||^2_Fro + alpha * ||W||_21

Where::

||W||_21 = \sum_i \sqrt\sum_j w_{ij^2

}

i.e. the sum of norm of each row.

Read more in the :ref:`User Guide <lasso>`.

Parameters ---------- X : array-like, sparse matrix of shape (n_samples, n_features) Training data. Pass directly as Fortran-contiguous data to avoid unnecessary memory duplication. If ``y`` is mono-output then ``X`` can be sparse.

y : array-like, sparse matrix of shape (n_samples,) or (n_samples, n_outputs) Target values

eps : float, default=1e-3 Length of the path. ``eps=1e-3`` means that ``alpha_min / alpha_max = 1e-3``

n_alphas : int, default=100 Number of alphas along the regularization path

alphas : ndarray, default=None List of alphas where to compute the models. If ``None`` alphas are set automatically

precompute : 'auto', bool or array-like of shape (n_features, n_features), default='auto' Whether to use a precomputed Gram matrix to speed up calculations. If set to ``'auto'`` let us decide. The Gram matrix can also be passed as argument.

Xy : array-like of shape (n_features,) or (n_features, n_outputs), default=None Xy = np.dot(X.T, y) that can be precomputed. It is useful only when the Gram matrix is precomputed.

copy_X : bool, default=True If ``True``, X will be copied; else, it may be overwritten.

coef_init : ndarray of shape (n_features, ), default=None The initial values of the coefficients.

verbose : bool or int, default=False Amount of verbosity.

return_n_iter : bool, default=False whether to return the number of iterations or not.

positive : bool, default=False If set to True, forces coefficients to be positive. (Only allowed when ``y.ndim == 1``).

**params : kwargs keyword arguments passed to the coordinate descent solver.

Returns ------- alphas : ndarray of shape (n_alphas,) The alphas along the path where models are computed.

coefs : ndarray of shape (n_features, n_alphas) or (n_outputs, n_features, n_alphas) Coefficients along the path.

dual_gaps : ndarray of shape (n_alphas,) The dual gaps at the end of the optimization for each alpha.

n_iters : list of int The number of iterations taken by the coordinate descent optimizer to reach the specified tolerance for each alpha.

Notes ----- For an example, see :ref:`examples/linear_model/plot_lasso_coordinate_descent_path.py <sphx_glr_auto_examples_linear_model_plot_lasso_coordinate_descent_path.py>`.

To avoid unnecessary memory duplication the X argument of the fit method should be directly passed as a Fortran-contiguous numpy array.

Note that in certain cases, the Lars solver may be significantly faster to implement this functionality. In particular, linear interpolation can be used to retrieve model coefficients between the values output by lars_path

Examples --------

Comparing lasso_path and lars_path with interpolation:

>>> X = np.array([1, 2, 3.1], [2.3, 5.4, 4.3]).T >>> y = np.array(1, 2, 3.1) >>> # Use lasso_path to compute a coefficient path >>> _, coef_path, _ = lasso_path(X, y, alphas=5., 1., .5) >>> print(coef_path) [0. 0. 0.46874778] [0.2159048 0.4425765 0.23689075]

>>> # Now use lars_path and 1D linear interpolation to compute the >>> # same path >>> from sklearn.linear_model import lars_path >>> alphas, active, coef_path_lars = lars_path(X, y, method='lasso') >>> from scipy import interpolate >>> coef_path_continuous = interpolate.interp1d(alphas::-1, ... coef_path_lars:, ::-1) >>> print(coef_path_continuous(5., 1., .5)) [0. 0. 0.46915237] [0.2159048 0.4425765 0.23668876]

See also -------- lars_path Lasso LassoLars LassoCV LassoLarsCV sklearn.decomposition.sparse_encode

Orthogonal Matching Pursuit (OMP)

Solves n_targets Orthogonal Matching Pursuit problems. An instance of the problem has the form:

When parametrized by the number of non-zero coefficients using `n_nonzero_coefs`: argmin ||y - X\gamma||^2 subject to ||\gamma||_0 <= n_nonzero coefs

When parametrized by error using the parameter `tol`: argmin ||\gamma||_0 subject to ||y - X\gamma||^2 <= tol

Read more in the :ref:`User Guide <omp>`.

Parameters ---------- X : array, shape (n_samples, n_features) Input data. Columns are assumed to have unit norm.

y : array, shape (n_samples,) or (n_samples, n_targets) Input targets

n_nonzero_coefs : int Desired number of non-zero entries in the solution. If None (by default) this value is set to 10% of n_features.

tol : float Maximum norm of the residual. If not None, overrides n_nonzero_coefs.

precompute : True, False, 'auto', Whether to perform precomputations. Improves performance when n_targets or n_samples is very large.

copy_X : bool, optional Whether the design matrix X must be copied by the algorithm. A false value is only helpful if X is already Fortran-ordered, otherwise a copy is made anyway.

return_path : bool, optional. Default: False Whether to return every value of the nonzero coefficients along the forward path. Useful for cross-validation.

return_n_iter : bool, optional default False Whether or not to return the number of iterations.

Returns ------- coef : array, shape (n_features,) or (n_features, n_targets) Coefficients of the OMP solution. If `return_path=True`, this contains the whole coefficient path. In this case its shape is (n_features, n_features) or (n_features, n_targets, n_features) and iterating over the last axis yields coefficients in increasing order of active features.

n_iters : array-like or int Number of active features across every target. Returned only if `return_n_iter` is set to True.

See also -------- OrthogonalMatchingPursuit orthogonal_mp_gram lars_path decomposition.sparse_encode

Notes ----- Orthogonal matching pursuit was introduced in S. Mallat, Z. Zhang, Matching pursuits with time-frequency dictionaries, IEEE Transactions on Signal Processing, Vol. 41, No. 12. (December 1993), pp. 3397-3415. (http://blanche.polytechnique.fr/~mallat/papiers/MallatPursuit93.pdf)

This implementation is based on Rubinstein, R., Zibulevsky, M. and Elad, M., Efficient Implementation of the K-SVD Algorithm using Batch Orthogonal Matching Pursuit Technical Report - CS Technion, April 2008. https://www.cs.technion.ac.il/~ronrubin/Publications/KSVD-OMP-v2.pdf

Gram Orthogonal Matching Pursuit (OMP)

Solves n_targets Orthogonal Matching Pursuit problems using only the Gram matrix X.T * X and the product X.T * y.

Read more in the :ref:`User Guide <omp>`.

Parameters ---------- Gram : array, shape (n_features, n_features) Gram matrix of the input data: X.T * X

Xy : array, shape (n_features,) or (n_features, n_targets) Input targets multiplied by X: X.T * y

n_nonzero_coefs : int Desired number of non-zero entries in the solution. If None (by default) this value is set to 10% of n_features.

tol : float Maximum norm of the residual. If not None, overrides n_nonzero_coefs.

norms_squared : array-like, shape (n_targets,) Squared L2 norms of the lines of y. Required if tol is not None.

copy_Gram : bool, optional Whether the gram matrix must be copied by the algorithm. A false value is only helpful if it is already Fortran-ordered, otherwise a copy is made anyway.

copy_Xy : bool, optional Whether the covariance vector Xy must be copied by the algorithm. If False, it may be overwritten.

return_path : bool, optional. Default: False Whether to return every value of the nonzero coefficients along the forward path. Useful for cross-validation.

return_n_iter : bool, optional default False Whether or not to return the number of iterations.

Returns ------- coef : array, shape (n_features,) or (n_features, n_targets) Coefficients of the OMP solution. If `return_path=True`, this contains the whole coefficient path. In this case its shape is (n_features, n_features) or (n_features, n_targets, n_features) and iterating over the last axis yields coefficients in increasing order of active features.

n_iters : array-like or int Number of active features across every target. Returned only if `return_n_iter` is set to True.

See also -------- OrthogonalMatchingPursuit orthogonal_mp lars_path decomposition.sparse_encode

Notes ----- Orthogonal matching pursuit was introduced in G. Mallat, Z. Zhang, Matching pursuits with time-frequency dictionaries, IEEE Transactions on Signal Processing, Vol. 41, No. 12. (December 1993), pp. 3397-3415. (http://blanche.polytechnique.fr/~mallat/papiers/MallatPursuit93.pdf)

This implementation is based on Rubinstein, R., Zibulevsky, M. and Elad, M., Efficient Implementation of the K-SVD Algorithm using Batch Orthogonal Matching Pursuit Technical Report - CS Technion, April 2008. https://www.cs.technion.ac.il/~ronrubin/Publications/KSVD-OMP-v2.pdf

Solve the ridge equation by the method of normal equations.

Read more in the :ref:`User Guide <ridge_regression>`.

Parameters ---------- X : ndarray, sparse matrix, LinearOperator of shape (n_samples, n_features) Training data

y : ndarray of shape (n_samples,) or (n_samples, n_targets) Target values

alpha : float or array-like of shape (n_targets,) Regularization strength; must be a positive float. Regularization improves the conditioning of the problem and reduces the variance of the estimates. Larger values specify stronger regularization. Alpha corresponds to ``1 / (2C)`` in other linear models such as :class:`~sklearn.linear_model.LogisticRegression` or :class:`sklearn.svm.LinearSVC`. If an array is passed, penalties are assumed to be specific to the targets. Hence they must correspond in number.

sample_weight : float or array-like of shape (n_samples,), default=None Individual weights for each sample. If given a float, every sample will have the same weight. If sample_weight is not None and solver='auto', the solver will be set to 'cholesky'.

.. versionadded:: 0.17

solver : 'auto', 'svd', 'cholesky', 'lsqr', 'sparse_cg', 'sag', 'saga', default='auto' Solver to use in the computational routines:

• 'auto' chooses the solver automatically based on the type of data.

• 'svd' uses a Singular Value Decomposition of X to compute the Ridge coefficients. More stable for singular matrices than 'cholesky'.

• 'cholesky' uses the standard scipy.linalg.solve function to obtain a closed-form solution via a Cholesky decomposition of dot(X.T, X)

• 'sparse_cg' uses the conjugate gradient solver as found in scipy.sparse.linalg.cg. As an iterative algorithm, this solver is more appropriate than 'cholesky' for large-scale data (possibility to set `tol` and `max_iter`).

• 'lsqr' uses the dedicated regularized least-squares routine scipy.sparse.linalg.lsqr. It is the fastest and uses an iterative procedure.

• 'sag' uses a Stochastic Average Gradient descent, and 'saga' uses its improved, unbiased version named SAGA. Both methods also use an iterative procedure, and are often faster than other solvers when both n_samples and n_features are large. Note that 'sag' and 'saga' fast convergence is only guaranteed on features with approximately the same scale. You can preprocess the data with a scaler from sklearn.preprocessing.

All last five solvers support both dense and sparse data. However, only 'sag' and 'sparse_cg' supports sparse input when `fit_intercept` is True.

.. versionadded:: 0.17 Stochastic Average Gradient descent solver. .. versionadded:: 0.19 SAGA solver.

max_iter : int, default=None Maximum number of iterations for conjugate gradient solver. For the 'sparse_cg' and 'lsqr' solvers, the default value is determined by scipy.sparse.linalg. For 'sag' and saga solver, the default value is 1000.

tol : float, default=1e-3 Precision of the solution.

verbose : int, default=0 Verbosity level. Setting verbose > 0 will display additional information depending on the solver used.

random_state : int, RandomState instance, default=None Used when ``solver`` == 'sag' or 'saga' to shuffle the data. See :term:`Glossary <random_state>` for details.

return_n_iter : bool, default=False If True, the method also returns `n_iter`, the actual number of iteration performed by the solver.

.. versionadded:: 0.17

return_intercept : bool, default=False If True and if X is sparse, the method also returns the intercept, and the solver is automatically changed to 'sag'. This is only a temporary fix for fitting the intercept with sparse data. For dense data, use sklearn.linear_model._preprocess_data before your regression.

.. versionadded:: 0.17

check_input : bool, default=True If False, the input arrays X and y will not be checked.

.. versionadded:: 0.21

Returns ------- coef : ndarray of shape (n_features,) or (n_targets, n_features) Weight vector(s).

n_iter : int, optional The actual number of iteration performed by the solver. Only returned if `return_n_iter` is True.

intercept : float or ndarray of shape (n_targets,) The intercept of the model. Only returned if `return_intercept` is True and if X is a scipy sparse array.

Notes ----- This function won't compute the intercept.