package sklearn

Get an attribute of this module as a Py.Object.t. This is useful to pass a Python function to another function.

Compute elastic net path with coordinate descent.

The elastic net optimization function varies for mono and multi-outputs.

For mono-output tasks it is::

1 / (2 * n_samples) * ||y - Xw||^2_2

1. alpha * l1_ratio * ||w||_1
2. 0.5 * alpha * (1 - l1_ratio) * ||w||^2_2

For multi-output tasks it is::

(1 / (2 * n_samples)) * ||Y - XW||^Fro_2

1. alpha * l1_ratio * ||W||_21
2. 0.5 * alpha * (1 - l1_ratio) * ||W||_Fro^2

Where::

||W||_21 = \sum_i \sqrt\sum_j w_{ij^2

}

i.e. the sum of norm of each row.

Read more in the :ref:`User Guide <elastic_net>`.

Parameters ---------- X : array-like, shape (n_samples, n_features) Training data. Pass directly as Fortran-contiguous data to avoid unnecessary memory duplication. If ``y`` is mono-output then ``X`` can be sparse.

y : ndarray, shape (n_samples,) or (n_samples, n_outputs) Target values.

l1_ratio : float, optional Number between 0 and 1 passed to elastic net (scaling between l1 and l2 penalties). ``l1_ratio=1`` corresponds to the Lasso.

eps : float Length of the path. ``eps=1e-3`` means that ``alpha_min / alpha_max = 1e-3``.

n_alphas : int, optional Number of alphas along the regularization path.

alphas : ndarray, optional List of alphas where to compute the models. If None alphas are set automatically.

precompute : True | False | 'auto' | array-like Whether to use a precomputed Gram matrix to speed up calculations. If set to ``'auto'`` let us decide. The Gram matrix can also be passed as argument.

Xy : array-like, optional Xy = np.dot(X.T, y) that can be precomputed. It is useful only when the Gram matrix is precomputed.

copy_X : bool, optional, default True If ``True``, X will be copied; else, it may be overwritten.

coef_init : array, shape (n_features, ) | None The initial values of the coefficients.

verbose : bool or int Amount of verbosity.

return_n_iter : bool Whether to return the number of iterations or not.

positive : bool, default False If set to True, forces coefficients to be positive. (Only allowed when ``y.ndim == 1``).

check_input : bool, default True Skip input validation checks, including the Gram matrix when provided assuming there are handled by the caller when check_input=False.

**params : kwargs Keyword arguments passed to the coordinate descent solver.

Returns ------- alphas : array, shape (n_alphas,) The alphas along the path where models are computed.

coefs : array, shape (n_features, n_alphas) or (n_outputs, n_features, n_alphas) Coefficients along the path.

dual_gaps : array, shape (n_alphas,) The dual gaps at the end of the optimization for each alpha.

n_iters : array-like, shape (n_alphas,) The number of iterations taken by the coordinate descent optimizer to reach the specified tolerance for each alpha. (Is returned when ``return_n_iter`` is set to True).

See Also -------- MultiTaskElasticNet MultiTaskElasticNetCV ElasticNet ElasticNetCV

Notes ----- For an example, see :ref:`examples/linear_model/plot_lasso_coordinate_descent_path.py <sphx_glr_auto_examples_linear_model_plot_lasso_coordinate_descent_path.py>`.

Compute Least Angle Regression or Lasso path using LARS algorithm 1

The optimization objective for the case method='lasso' is::

(1 / (2 * n_samples)) * ||y - Xw||^2_2 + alpha * ||w||_1

in the case of method='lars', the objective function is only known in the form of an implicit equation (see discussion in 1)

Read more in the :ref:`User Guide <least_angle_regression>`.

Parameters ---------- X : None or array-like of shape (n_samples, n_features) Input data. Note that if X is None then the Gram matrix must be specified, i.e., cannot be None or False.

.. deprecated:: 0.21

The use of ``X`` is ``None`` in combination with ``Gram`` is not ``None`` will be removed in v0.23. Use :func:`lars_path_gram` instead.

y : None or array-like of shape (n_samples,) Input targets.

Xy : array-like of shape (n_samples,) or (n_samples, n_targets), default=None Xy = np.dot(X.T, y) that can be precomputed. It is useful only when the Gram matrix is precomputed.

Gram : None, 'auto', array-like of shape (n_features, n_features), default=None Precomputed Gram matrix (X' * X), if ``'auto'``, the Gram matrix is precomputed from the given X, if there are more samples than features.

.. deprecated:: 0.21

The use of ``X`` is ``None`` in combination with ``Gram`` is not None will be removed in v0.23. Use :func:`lars_path_gram` instead.

max_iter : int, default=500 Maximum number of iterations to perform, set to infinity for no limit.

alpha_min : float, default=0 Minimum correlation along the path. It corresponds to the regularization parameter alpha parameter in the Lasso.

method : 'lar', 'lasso', default='lar' Specifies the returned model. Select ``'lar'`` for Least Angle Regression, ``'lasso'`` for the Lasso.

copy_X : bool, default=True If ``False``, ``X`` is overwritten.

eps : float, optional The machine-precision regularization in the computation of the Cholesky diagonal factors. Increase this for very ill-conditioned systems. By default, ``np.finfo(np.float).eps`` is used.

copy_Gram : bool, default=True If ``False``, ``Gram`` is overwritten.

verbose : int, default=0 Controls output verbosity.

return_path : bool, default=True If ``return_path==True`` returns the entire path, else returns only the last point of the path.

return_n_iter : bool, default=False Whether to return the number of iterations.

positive : bool, default=False Restrict coefficients to be >= 0. This option is only allowed with method 'lasso'. Note that the model coefficients will not converge to the ordinary-least-squares solution for small values of alpha. Only coefficients up to the smallest alpha value (``alphas_alphas_ > 0..min()`` when fit_path=True) reached by the stepwise Lars-Lasso algorithm are typically in congruence with the solution of the coordinate descent lasso_path function.

Returns ------- alphas : array-like of shape (n_alphas + 1,) Maximum of covariances (in absolute value) at each iteration. ``n_alphas`` is either ``max_iter``, ``n_features`` or the number of nodes in the path with ``alpha >= alpha_min``, whichever is smaller.

active : array-like of shape (n_alphas,) Indices of active variables at the end of the path.

coefs : array-like of shape (n_features, n_alphas + 1) Coefficients along the path

n_iter : int Number of iterations run. Returned only if return_n_iter is set to True.

See also -------- lars_path_gram lasso_path lasso_path_gram LassoLars Lars LassoLarsCV LarsCV sklearn.decomposition.sparse_encode

References ---------- .. 1 'Least Angle Regression', Efron et al. http://statweb.stanford.edu/~tibs/ftp/lars.pdf

.. 2 `Wikipedia entry on the Least-angle regression <https://en.wikipedia.org/wiki/Least-angle_regression>`_

.. 3 `Wikipedia entry on the Lasso <https://en.wikipedia.org/wiki/Lasso_(statistics)>`_

lars_path in the sufficient stats mode 1

The optimization objective for the case method='lasso' is::

(1 / (2 * n_samples)) * ||y - Xw||^2_2 + alpha * ||w||_1

in the case of method='lars', the objective function is only known in the form of an implicit equation (see discussion in 1)

Read more in the :ref:`User Guide <least_angle_regression>`.

Parameters ---------- Xy : array-like of shape (n_samples,) or (n_samples, n_targets) Xy = np.dot(X.T, y).

Gram : array-like of shape (n_features, n_features) Gram = np.dot(X.T * X).

n_samples : int or float Equivalent size of sample.

max_iter : int, default=500 Maximum number of iterations to perform, set to infinity for no limit.

alpha_min : float, default=0 Minimum correlation along the path. It corresponds to the regularization parameter alpha parameter in the Lasso.

method : 'lar', 'lasso', default='lar' Specifies the returned model. Select ``'lar'`` for Least Angle Regression, ``'lasso'`` for the Lasso.

copy_X : bool, default=True If ``False``, ``X`` is overwritten.

eps : float, optional The machine-precision regularization in the computation of the Cholesky diagonal factors. Increase this for very ill-conditioned systems. By default, ``np.finfo(np.float).eps`` is used.

copy_Gram : bool, default=True If ``False``, ``Gram`` is overwritten.

verbose : int, default=0 Controls output verbosity.

return_path : bool, default=True If ``return_path==True`` returns the entire path, else returns only the last point of the path.

return_n_iter : bool, default=False Whether to return the number of iterations.

positive : bool, default=False Restrict coefficients to be >= 0. This option is only allowed with method 'lasso'. Note that the model coefficients will not converge to the ordinary-least-squares solution for small values of alpha. Only coefficients up to the smallest alpha value (``alphas_alphas_ > 0..min()`` when fit_path=True) reached by the stepwise Lars-Lasso algorithm are typically in congruence with the solution of the coordinate descent lasso_path function.

Returns ------- alphas : array-like of shape (n_alphas + 1,) Maximum of covariances (in absolute value) at each iteration. ``n_alphas`` is either ``max_iter``, ``n_features`` or the number of nodes in the path with ``alpha >= alpha_min``, whichever is smaller.

active : array-like of shape (n_alphas,) Indices of active variables at the end of the path.

coefs : array-like of shape (n_features, n_alphas + 1) Coefficients along the path

n_iter : int Number of iterations run. Returned only if return_n_iter is set to True.

See also -------- lars_path lasso_path lasso_path_gram LassoLars Lars LassoLarsCV LarsCV sklearn.decomposition.sparse_encode

References ---------- .. 1 'Least Angle Regression', Efron et al. http://statweb.stanford.edu/~tibs/ftp/lars.pdf

.. 2 `Wikipedia entry on the Least-angle regression <https://en.wikipedia.org/wiki/Least-angle_regression>`_

.. 3 `Wikipedia entry on the Lasso <https://en.wikipedia.org/wiki/Lasso_(statistics)>`_

Compute Lasso path with coordinate descent

The Lasso optimization function varies for mono and multi-outputs.

For mono-output tasks it is::

(1 / (2 * n_samples)) * ||y - Xw||^2_2 + alpha * ||w||_1

For multi-output tasks it is::

(1 / (2 * n_samples)) * ||Y - XW||^2_Fro + alpha * ||W||_21

Where::

||W||_21 = \sum_i \sqrt\sum_j w_{ij^2

}

i.e. the sum of norm of each row.

Read more in the :ref:`User Guide <lasso>`.

Parameters ---------- X : array-like, sparse matrix, shape (n_samples, n_features) Training data. Pass directly as Fortran-contiguous data to avoid unnecessary memory duplication. If ``y`` is mono-output then ``X`` can be sparse.

y : ndarray, shape (n_samples,), or (n_samples, n_outputs) Target values

eps : float, optional Length of the path. ``eps=1e-3`` means that ``alpha_min / alpha_max = 1e-3``

n_alphas : int, optional Number of alphas along the regularization path

alphas : ndarray, optional List of alphas where to compute the models. If ``None`` alphas are set automatically

precompute : True | False | 'auto' | array-like Whether to use a precomputed Gram matrix to speed up calculations. If set to ``'auto'`` let us decide. The Gram matrix can also be passed as argument.

Xy : array-like, optional Xy = np.dot(X.T, y) that can be precomputed. It is useful only when the Gram matrix is precomputed.

copy_X : boolean, optional, default True If ``True``, X will be copied; else, it may be overwritten.

coef_init : array, shape (n_features, ) | None The initial values of the coefficients.

verbose : bool or integer Amount of verbosity.

return_n_iter : bool whether to return the number of iterations or not.

positive : bool, default False If set to True, forces coefficients to be positive. (Only allowed when ``y.ndim == 1``).

**params : kwargs keyword arguments passed to the coordinate descent solver.

Returns ------- alphas : array, shape (n_alphas,) The alphas along the path where models are computed.

coefs : array, shape (n_features, n_alphas) or (n_outputs, n_features, n_alphas) Coefficients along the path.

dual_gaps : array, shape (n_alphas,) The dual gaps at the end of the optimization for each alpha.

n_iters : array-like, shape (n_alphas,) The number of iterations taken by the coordinate descent optimizer to reach the specified tolerance for each alpha.

Notes ----- For an example, see :ref:`examples/linear_model/plot_lasso_coordinate_descent_path.py <sphx_glr_auto_examples_linear_model_plot_lasso_coordinate_descent_path.py>`.

To avoid unnecessary memory duplication the X argument of the fit method should be directly passed as a Fortran-contiguous numpy array.

Note that in certain cases, the Lars solver may be significantly faster to implement this functionality. In particular, linear interpolation can be used to retrieve model coefficients between the values output by lars_path

Examples --------

Comparing lasso_path and lars_path with interpolation:

>>> X = np.array([1, 2, 3.1], [2.3, 5.4, 4.3]).T >>> y = np.array(1, 2, 3.1) >>> # Use lasso_path to compute a coefficient path >>> _, coef_path, _ = lasso_path(X, y, alphas=5., 1., .5) >>> print(coef_path) [0. 0. 0.46874778] [0.2159048 0.4425765 0.23689075]

>>> # Now use lars_path and 1D linear interpolation to compute the >>> # same path >>> from sklearn.linear_model import lars_path >>> alphas, active, coef_path_lars = lars_path(X, y, method='lasso') >>> from scipy import interpolate >>> coef_path_continuous = interpolate.interp1d(alphas::-1, ... coef_path_lars:, ::-1) >>> print(coef_path_continuous(5., 1., .5)) [0. 0. 0.46915237] [0.2159048 0.4425765 0.23668876]

See also -------- lars_path Lasso LassoLars LassoCV LassoLarsCV sklearn.decomposition.sparse_encode

DEPRECATED: logistic_regression_path was deprecated in version 0.21 and will be removed in version 0.23.0

Compute a Logistic Regression model for a list of regularization parameters.

This is an implementation that uses the result of the previous model to speed up computations along the set of solutions, making it faster than sequentially calling LogisticRegression for the different parameters. Note that there will be no speedup with liblinear solver, since it does not handle warm-starting.

.. deprecated:: 0.21 ``logistic_regression_path`` was deprecated in version 0.21 and will be removed in 0.23.

Read more in the :ref:`User Guide <logistic_regression>`.

Parameters ---------- X : array-like or sparse matrix, shape (n_samples, n_features) Input data.

y : array-like, shape (n_samples,) or (n_samples, n_targets) Input data, target values.

pos_class : int, None The class with respect to which we perform a one-vs-all fit. If None, then it is assumed that the given problem is binary.

Cs : int | array-like, shape (n_cs,) List of values for the regularization parameter or integer specifying the number of regularization parameters that should be used. In this case, the parameters will be chosen in a logarithmic scale between 1e-4 and 1e4.

fit_intercept : bool Whether to fit an intercept for the model. In this case the shape of the returned array is (n_cs, n_features + 1).

max_iter : int Maximum number of iterations for the solver.

tol : float Stopping criterion. For the newton-cg and lbfgs solvers, the iteration will stop when ``max |g_i | i = 1, ..., n <= tol`` where ``g_i`` is the i-th component of the gradient.

verbose : int For the liblinear and lbfgs solvers set verbose to any positive number for verbosity.

solver : 'lbfgs', 'newton-cg', 'liblinear', 'sag', 'saga' Numerical solver to use.

coef : array-like, shape (n_features,), default None Initialization value for coefficients of logistic regression. Useless for liblinear solver.

class_weight : dict or 'balanced', optional Weights associated with classes in the form ``class_label: weight``. If not given, all classes are supposed to have weight one.

The 'balanced' mode uses the values of y to automatically adjust weights inversely proportional to class frequencies in the input data as ``n_samples / (n_classes * np.bincount(y))``.

Note that these weights will be multiplied with sample_weight (passed through the fit method) if sample_weight is specified.

dual : bool Dual or primal formulation. Dual formulation is only implemented for l2 penalty with liblinear solver. Prefer dual=False when n_samples > n_features.

penalty : str, 'l1', 'l2', or 'elasticnet' Used to specify the norm used in the penalization. The 'newton-cg', 'sag' and 'lbfgs' solvers support only l2 penalties. 'elasticnet' is only supported by the 'saga' solver.

intercept_scaling : float, default 1. Useful only when the solver 'liblinear' is used and self.fit_intercept is set to True. In this case, x becomes x, self.intercept_scaling, i.e. a 'synthetic' feature with constant value equal to intercept_scaling is appended to the instance vector. The intercept becomes ``intercept_scaling * synthetic_feature_weight``.

Note! the synthetic feature weight is subject to l1/l2 regularization as all other features. To lessen the effect of regularization on synthetic feature weight (and therefore on the intercept) intercept_scaling has to be increased.

multi_class : 'ovr', 'multinomial', 'auto', default='auto' If the option chosen is 'ovr', then a binary problem is fit for each label. For 'multinomial' the loss minimised is the multinomial loss fit across the entire probability distribution, *even when the data is binary*. 'multinomial' is unavailable when solver='liblinear'. 'auto' selects 'ovr' if the data is binary, or if solver='liblinear', and otherwise selects 'multinomial'.

.. versionadded:: 0.18 Stochastic Average Gradient descent solver for 'multinomial' case. .. versionchanged:: 0.22 Default changed from 'ovr' to 'auto' in 0.22.

random_state : int, RandomState instance or None, optional, default None The seed of the pseudo random number generator to use when shuffling the data. If int, random_state is the seed used by the random number generator; If RandomState instance, random_state is the random number generator; If None, the random number generator is the RandomState instance used by `np.random`. Used when ``solver`` == 'sag' or 'liblinear'.

check_input : bool, default True If False, the input arrays X and y will not be checked.

max_squared_sum : float, default None Maximum squared sum of X over samples. Used only in SAG solver. If None, it will be computed, going through all the samples. The value should be precomputed to speed up cross validation.

sample_weight : array-like, shape(n_samples,) optional Array of weights that are assigned to individual samples. If not provided, then each sample is given unit weight.

l1_ratio : float or None, optional (default=None) The Elastic-Net mixing parameter, with ``0 <= l1_ratio <= 1``. Only used if ``penalty='elasticnet'``. Setting ``l1_ratio=0`` is equivalent to using ``penalty='l2'``, while setting ``l1_ratio=1`` is equivalent to using ``penalty='l1'``. For ``0 < l1_ratio <1``, the penalty is a combination of L1 and L2.

Returns ------- coefs : ndarray, shape (n_cs, n_features) or (n_cs, n_features + 1) List of coefficients for the Logistic Regression model. If fit_intercept is set to True then the second dimension will be n_features + 1, where the last item represents the intercept. For ``multiclass='multinomial'``, the shape is (n_classes, n_cs, n_features) or (n_classes, n_cs, n_features + 1).

Cs : ndarray Grid of Cs used for cross-validation.

n_iter : array, shape (n_cs,) Actual number of iteration for each Cs.

Notes ----- You might get slightly different results with the solver liblinear than with the others since this uses LIBLINEAR which penalizes the intercept.

.. versionchanged:: 0.19 The 'copy' parameter was removed.

Orthogonal Matching Pursuit (OMP)

Solves n_targets Orthogonal Matching Pursuit problems. An instance of the problem has the form:

When parametrized by the number of non-zero coefficients using `n_nonzero_coefs`: argmin ||y - X\gamma||^2 subject to ||\gamma||_0 <= n_nonzero coefs

When parametrized by error using the parameter `tol`: argmin ||\gamma||_0 subject to ||y - X\gamma||^2 <= tol

Read more in the :ref:`User Guide <omp>`.

Parameters ---------- X : array, shape (n_samples, n_features) Input data. Columns are assumed to have unit norm.

y : array, shape (n_samples,) or (n_samples, n_targets) Input targets

n_nonzero_coefs : int Desired number of non-zero entries in the solution. If None (by default) this value is set to 10% of n_features.

tol : float Maximum norm of the residual. If not None, overrides n_nonzero_coefs.

precompute : True, False, 'auto', Whether to perform precomputations. Improves performance when n_targets or n_samples is very large.

copy_X : bool, optional Whether the design matrix X must be copied by the algorithm. A false value is only helpful if X is already Fortran-ordered, otherwise a copy is made anyway.

return_path : bool, optional. Default: False Whether to return every value of the nonzero coefficients along the forward path. Useful for cross-validation.

return_n_iter : bool, optional default False Whether or not to return the number of iterations.

Returns ------- coef : array, shape (n_features,) or (n_features, n_targets) Coefficients of the OMP solution. If `return_path=True`, this contains the whole coefficient path. In this case its shape is (n_features, n_features) or (n_features, n_targets, n_features) and iterating over the last axis yields coefficients in increasing order of active features.

n_iters : array-like or int Number of active features across every target. Returned only if `return_n_iter` is set to True.

See also -------- OrthogonalMatchingPursuit orthogonal_mp_gram lars_path decomposition.sparse_encode

Notes ----- Orthogonal matching pursuit was introduced in S. Mallat, Z. Zhang, Matching pursuits with time-frequency dictionaries, IEEE Transactions on Signal Processing, Vol. 41, No. 12. (December 1993), pp. 3397-3415. (http://blanche.polytechnique.fr/~mallat/papiers/MallatPursuit93.pdf)

This implementation is based on Rubinstein, R., Zibulevsky, M. and Elad, M., Efficient Implementation of the K-SVD Algorithm using Batch Orthogonal Matching Pursuit Technical Report - CS Technion, April 2008. https://www.cs.technion.ac.il/~ronrubin/Publications/KSVD-OMP-v2.pdf

Gram Orthogonal Matching Pursuit (OMP)

Solves n_targets Orthogonal Matching Pursuit problems using only the Gram matrix X.T * X and the product X.T * y.

Read more in the :ref:`User Guide <omp>`.

Parameters ---------- Gram : array, shape (n_features, n_features) Gram matrix of the input data: X.T * X

Xy : array, shape (n_features,) or (n_features, n_targets) Input targets multiplied by X: X.T * y

n_nonzero_coefs : int Desired number of non-zero entries in the solution. If None (by default) this value is set to 10% of n_features.

tol : float Maximum norm of the residual. If not None, overrides n_nonzero_coefs.

norms_squared : array-like, shape (n_targets,) Squared L2 norms of the lines of y. Required if tol is not None.

copy_Gram : bool, optional Whether the gram matrix must be copied by the algorithm. A false value is only helpful if it is already Fortran-ordered, otherwise a copy is made anyway.

copy_Xy : bool, optional Whether the covariance vector Xy must be copied by the algorithm. If False, it may be overwritten.

return_path : bool, optional. Default: False Whether to return every value of the nonzero coefficients along the forward path. Useful for cross-validation.

return_n_iter : bool, optional default False Whether or not to return the number of iterations.

Returns ------- coef : array, shape (n_features,) or (n_features, n_targets) Coefficients of the OMP solution. If `return_path=True`, this contains the whole coefficient path. In this case its shape is (n_features, n_features) or (n_features, n_targets, n_features) and iterating over the last axis yields coefficients in increasing order of active features.

n_iters : array-like or int Number of active features across every target. Returned only if `return_n_iter` is set to True.

See also -------- OrthogonalMatchingPursuit orthogonal_mp lars_path decomposition.sparse_encode

Notes ----- Orthogonal matching pursuit was introduced in G. Mallat, Z. Zhang, Matching pursuits with time-frequency dictionaries, IEEE Transactions on Signal Processing, Vol. 41, No. 12. (December 1993), pp. 3397-3415. (http://blanche.polytechnique.fr/~mallat/papiers/MallatPursuit93.pdf)

This implementation is based on Rubinstein, R., Zibulevsky, M. and Elad, M., Efficient Implementation of the K-SVD Algorithm using Batch Orthogonal Matching Pursuit Technical Report - CS Technion, April 2008. https://www.cs.technion.ac.il/~ronrubin/Publications/KSVD-OMP-v2.pdf

Solve the ridge equation by the method of normal equations.

Read more in the :ref:`User Guide <ridge_regression>`.

Parameters ---------- X : ndarray, sparse matrix, LinearOperator of shape (n_samples, n_features) Training data

y : ndarray of shape (n_samples,) or (n_samples, n_targets) Target values

alpha : float or array-like of shape (n_targets,) Regularization strength; must be a positive float. Regularization improves the conditioning of the problem and reduces the variance of the estimates. Larger values specify stronger regularization. Alpha corresponds to ``C^-1`` in other linear models such as LogisticRegression or LinearSVC. If an array is passed, penalties are assumed to be specific to the targets. Hence they must correspond in number.

sample_weight : float or array-like of shape (n_samples,), default=None Individual weights for each sample. If given a float, every sample will have the same weight. If sample_weight is not None and solver='auto', the solver will be set to 'cholesky'.

.. versionadded:: 0.17

solver : 'auto', 'svd', 'cholesky', 'lsqr', 'sparse_cg', 'sag', 'saga', default='auto' Solver to use in the computational routines:

• 'auto' chooses the solver automatically based on the type of data.

• 'svd' uses a Singular Value Decomposition of X to compute the Ridge coefficients. More stable for singular matrices than 'cholesky'.

• 'cholesky' uses the standard scipy.linalg.solve function to obtain a closed-form solution via a Cholesky decomposition of dot(X.T, X)

• 'sparse_cg' uses the conjugate gradient solver as found in scipy.sparse.linalg.cg. As an iterative algorithm, this solver is more appropriate than 'cholesky' for large-scale data (possibility to set `tol` and `max_iter`).

• 'lsqr' uses the dedicated regularized least-squares routine scipy.sparse.linalg.lsqr. It is the fastest and uses an iterative procedure.

• 'sag' uses a Stochastic Average Gradient descent, and 'saga' uses its improved, unbiased version named SAGA. Both methods also use an iterative procedure, and are often faster than other solvers when both n_samples and n_features are large. Note that 'sag' and 'saga' fast convergence is only guaranteed on features with approximately the same scale. You can preprocess the data with a scaler from sklearn.preprocessing.

All last five solvers support both dense and sparse data. However, only 'sag' and 'sparse_cg' supports sparse input when`fit_intercept` is True.

.. versionadded:: 0.17 Stochastic Average Gradient descent solver. .. versionadded:: 0.19 SAGA solver.

max_iter : int, default=None Maximum number of iterations for conjugate gradient solver. For the 'sparse_cg' and 'lsqr' solvers, the default value is determined by scipy.sparse.linalg. For 'sag' and saga solver, the default value is 1000.

tol : float, default=1e-3 Precision of the solution.

verbose : int, default=0 Verbosity level. Setting verbose > 0 will display additional information depending on the solver used.

random_state : int, RandomState instance, default=None The seed of the pseudo random number generator to use when shuffling the data. If int, random_state is the seed used by the random number generator; If RandomState instance, random_state is the random number generator; If None, the random number generator is the RandomState instance used by `np.random`. Used when ``solver`` == 'sag'.

return_n_iter : bool, default=False If True, the method also returns `n_iter`, the actual number of iteration performed by the solver.

.. versionadded:: 0.17

return_intercept : bool, default=False If True and if X is sparse, the method also returns the intercept, and the solver is automatically changed to 'sag'. This is only a temporary fix for fitting the intercept with sparse data. For dense data, use sklearn.linear_model._preprocess_data before your regression.

.. versionadded:: 0.17

check_input : bool, default=True If False, the input arrays X and y will not be checked.

.. versionadded:: 0.21

Returns ------- coef : ndarray of shape (n_features,) or (n_targets, n_features) Weight vector(s).

n_iter : int, optional The actual number of iteration performed by the solver. Only returned if `return_n_iter` is True.

intercept : float or ndarray of shape (n_targets,) The intercept of the model. Only returned if `return_intercept` is True and if X is a scipy sparse array.

Notes ----- This function won't compute the intercept.