Metrics¶
classification¶
-
longling.ML.metrics.classification.classification_report(y_true, y_pred=None, y_score=None, labels=None, metrics=None, sample_weight=None, average_options=None, multiclass_to_multilabel=False, logger=<module 'logging' from '/home/docs/.pyenv/versions/3.7.9/lib/python3.7/logging/__init__.py'>, **kwargs)[源代码]¶ Currently support binary and multiclasss classification.
参数: - y_true (list, 1d array-like, or label indicator array / sparse matrix) -- Ground truth (correct) target values.
- y_pred (list or None, 1d array-like, or label indicator array / sparse matrix) -- Estimated targets as returned by a classifier.
- y_score (array or None, shape = [n_samples] or [n_samples, n_classes]) -- Target scores, can either be probability estimates of the positive class, confidence values, or non-thresholded measure of decisions (as returned by "decision_function" on some classifiers). For binary y_true, y_score is supposed to be the score of the class with greater label.
- labels (array, shape = [n_labels]) -- Optional list of label indices to include in the report.
- metrics (list of str,) -- Support: precision, recall, f1, support, accuracy, auc, aupoc.
- sample_weight (array-like of shape = [n_samples], optional) -- Sample weights.
- average_options (str or list) -- default to macro, choices (one or many): "micro", "macro", "samples", "weighted"
- multiclass_to_multilabel (bool) --
- logger --
实际案例
>>> import numpy as np >>> # binary classification >>> y_true = np.array([0, 0, 1, 1, 0]) >>> y_pred = np.array([0, 1, 0, 1, 0]) >>> classification_report(y_true, y_pred) precision recall f1 support 0 0.666667 0.666667 0.666667 3 1 0.500000 0.500000 0.500000 2 macro_avg 0.583333 0.583333 0.583333 5 accuracy: 0.600000 >>> y_true = np.array([0, 0, 1, 1]) >>> y_score = np.array([0.1, 0.4, 0.35, 0.8]) >>> classification_report(y_true, y_score=y_score) # doctest: +NORMALIZE_WHITESPACE macro_auc: 0.750000 macro_aupoc: 0.833333 >>> y_true = np.array([0, 0, 1, 1]) >>> y_pred = [0, 0, 0, 1] >>> y_score = np.array([0.1, 0.4, 0.35, 0.8]) >>> classification_report(y_true, y_pred, y_score=y_score) # doctest: +NORMALIZE_WHITESPACE precision recall f1 support 0 0.666667 1.00 0.800000 2 1 1.000000 0.50 0.666667 2 macro_avg 0.833333 0.75 0.733333 4 accuracy: 0.750000 macro_auc: 0.750000 macro_aupoc: 0.833333 >>> # multiclass classification >>> y_true = [0, 1, 2, 2, 2] >>> y_pred = [0, 0, 2, 2, 1] >>> classification_report(y_true, y_pred) precision recall f1 support 0 0.5 1.000000 0.666667 1 1 0.0 0.000000 0.000000 1 2 1.0 0.666667 0.800000 3 macro_avg 0.5 0.555556 0.488889 5 accuracy: 0.600000 >>> # multiclass in multilabel >>> y_true = np.array([0, 0, 1, 1, 2, 1]) >>> y_pred = np.array([2, 1, 0, 2, 1, 0]) >>> y_score = np.array([ ... [0.15, 0.4, 0.45], ... [0.1, 0.9, 0.0], ... [0.33333, 0.333333, 0.333333], ... [0.15, 0.4, 0.45], ... [0.1, 0.9, 0.0], ... [0.33333, 0.333333, 0.333333] ... ]) >>> classification_report( ... y_true, y_pred, y_score, ... multiclass_to_multilabel=True, ... metrics=["aupoc"] ... ) aupoc 0 0.291667 1 0.416667 2 0.166667 macro_avg 0.291667 >>> classification_report( ... y_true, y_pred, y_score, ... multiclass_to_multilabel=True, ... metrics=["auc", "aupoc"] ... ) auc aupoc 0 0.250000 0.291667 1 0.055556 0.416667 2 0.100000 0.166667 macro_avg 0.135185 0.291667 macro_auc: 0.194444 >>> y_true = np.array([0, 1, 1, 1, 2, 1]) >>> y_pred = np.array([2, 1, 0, 2, 1, 0]) >>> y_score = np.array([ ... [0.45, 0.4, 0.15], ... [0.1, 0.9, 0.0], ... [0.33333, 0.333333, 0.333333], ... [0.15, 0.4, 0.45], ... [0.1, 0.9, 0.0], ... [0.33333, 0.333333, 0.333333] ... ]) >>> classification_report( ... y_true, y_pred, ... y_score, ... multiclass_to_multilabel=True, ... ) # doctest: +NORMALIZE_WHITESPACE precision recall f1 auc aupoc support 0 0.000000 0.000000 0.000000 1.00 1.000000 1 1 0.500000 0.250000 0.333333 0.25 0.583333 4 2 0.000000 0.000000 0.000000 0.10 0.166667 1 macro_avg 0.166667 0.083333 0.111111 0.45 0.583333 6 accuracy: 0.166667 macro_auc: 0.437500 >>> classification_report( ... y_true, y_pred, ... y_score, ... labels=[0, 1], ... multiclass_to_multilabel=True, ... ) # doctest: +NORMALIZE_WHITESPACE precision recall f1 auc aupoc support 0 0.00 0.000 0.000000 1.00 1.000000 1 1 0.50 0.250 0.333333 0.25 0.583333 4 macro_avg 0.25 0.125 0.166667 0.45 0.583333 5 accuracy: 0.166667 macro_auc: 0.437500
regression¶
-
longling.ML.metrics.regression.regression_report(y_true, y_pred, metrics=None, sample_weight=None, multioutput='uniform_average', average_options=None, key_prefix='', key_suffix='', verbose=True)[源代码]¶ 参数: - y_true (array-like of shape (n_samples,) or (n_samples, n_outputs)) -- Ground truth (correct) target values.
- y_pred (array-like of shape (n_samples,) or (n_samples, n_outputs)) -- Estimated target values.
- metrics (list of str,) -- Support: evar(explained_variance), mse, rmse, mae, r2
- sample_weight (array-like of shape (n_samples,), optional) -- Sample weights.
- multioutput (string in ['raw_values', 'uniform_average', 'variance_weighted'], list) --
or array-like of shape (n_outputs) Defines aggregating of multiple output values. Disabled when verbose is True. Array-like value defines weights used to average errors. 'raw_values' :
Returns a full set of errors in case of multioutput input.- 'uniform_average' :
- Errors of all outputs are averaged with uniform weight. Alias: "macro"
- 'variance_weighted':
- Only support in evar and r2. Scores of all outputs are averaged, weighted by the variances of each individual output. Alias: "vw"
- average_options (str or list) -- default to macro, choices (one or many): "macro", "vw"
- key_prefix (str) --
- key_suffix (str) --
- verbose (bool) --
返回: - evar (explained variance)
- mse (mean squared error)
- rmse (root mean squared error)
- mae (mean absolute error)
- r2 (r2 score)
实际案例
>>> y_true = [[0.5, 1, 1], [-1, 1, 1], [7, -6, 1]] >>> y_pred = [[0, 2, 1], [-1, 2, 1], [8, -5, 1]] >>> regression_report(y_true, y_pred) # doctest: +NORMALIZE_WHITESPACE evar mse rmse mae r2 0 0.967742 0.416667 0.645497 0.5 0.965438 1 1.000000 1.000000 1.000000 1.0 0.908163 2 1.000000 0.000000 0.000000 0.0 1.000000 uniform_average 0.989247 0.472222 0.548499 0.5 0.957867 variance_weighted 0.983051 0.472222 0.548499 0.5 0.938257 >>> regression_report(y_true, y_pred, verbose=False) # doctest: +NORMALIZE_WHITESPACE evar: 0.989247 mse: 0.472222 rmse: 0.548499 mae: 0.500000 r2: 0.957867 >>> regression_report( ... y_true, y_pred, multioutput="variance_weighted", verbose=False ... ) # doctest: +NORMALIZE_WHITESPACE evar: 0.983051 mse: 0.472222 rmse: 0.548499 mae: 0.500000 r2: 0.938257 >>> regression_report(y_true, y_pred, multioutput=[0.3, 0.6, 0.1], verbose=False) # doctest: +NORMALIZE_WHITESPACE evar: 0.990323 mse: 0.725000 rmse: 0.793649 mae: 0.750000 r2: 0.934529 >>> regression_report(y_true, y_pred, verbose=True) # doctest: +NORMALIZE_WHITESPACE evar mse rmse mae r2 0 0.967742 0.416667 0.645497 0.5 0.965438 1 1.000000 1.000000 1.000000 1.0 0.908163 2 1.000000 0.000000 0.000000 0.0 1.000000 uniform_average 0.989247 0.472222 0.548499 0.5 0.957867 variance_weighted 0.983051 0.472222 0.548499 0.5 0.938257 >>> regression_report( ... y_true, y_pred, verbose=True, average_options=["macro", "vw", [0.3, 0.6, 0.1]] ... ) # doctest: +NORMALIZE_WHITESPACE evar mse rmse mae r2 0 0.967742 0.416667 0.645497 0.50 0.965438 1 1.000000 1.000000 1.000000 1.00 0.908163 2 1.000000 0.000000 0.000000 0.00 1.000000 uniform_average 0.989247 0.472222 0.548499 0.50 0.957867 variance_weighted 0.983051 0.472222 0.548499 0.50 0.938257 weighted 0.990323 0.725000 0.793649 0.75 0.934529
ranking¶
-
longling.ML.metrics.ranking.ranking_report(y_true, y_pred, k: (<class 'int'>, <class 'list'>) = None, continuous=False, coerce='ignore', pad_pred=-100, metrics=None, bottom=False, verbose=True) → longling.ML.metrics.utils.POrderedDict[源代码]¶ 参数: - y_true --
- y_pred --
- k --
- continuous --
- coerce --
- pad_pred --
- metrics --
- bottom --
- verbose --
实际案例
>>> y_true = [[1, 0, 0], [0, 0, 1]] >>> y_pred = [[0.75, 0.5, 1], [1, 0.2, 0.1]] >>> ranking_report(y_true, y_pred) # doctest: +NORMALIZE_WHITESPACE ndcg@k precision@k recall@k f1@k len@k support@k 1 1.000000 0.000000 0.0 0.0 1.0 2 3 0.565465 0.333333 1.0 0.5 3.0 2 5 0.565465 0.333333 1.0 0.5 3.0 2 10 0.565465 0.333333 1.0 0.5 3.0 2 auc: 0.250000 map: 0.416667 mrr: 0.416667 coverage_error: 2.500000 ranking_loss: 0.750000 len: 3.000000 support: 2 >>> ranking_report(y_true, y_pred, k=[1, 3, 5]) # doctest: +NORMALIZE_WHITESPACE ndcg@k precision@k recall@k f1@k len@k support@k 1 1.000000 0.000000 0.0 0.0 1.0 2 3 0.565465 0.333333 1.0 0.5 3.0 2 5 0.565465 0.333333 1.0 0.5 3.0 2 auc: 0.250000 map: 0.416667 mrr: 0.416667 coverage_error: 2.500000 ranking_loss: 0.750000 len: 3.000000 support: 2 >>> ranking_report(y_true, y_pred, bottom=True) # doctest: +NORMALIZE_WHITESPACE ndcg@k precision@k recall@k f1@k len@k support@k ndcg@k(B) \ 1 1.000000 0.000000 0.0 0.0 1.0 2 1.000000 3 0.565465 0.333333 1.0 0.5 3.0 2 0.806574 5 0.565465 0.333333 1.0 0.5 3.0 2 0.806574 10 0.565465 0.333333 1.0 0.5 3.0 2 0.806574 <BLANKLINE> precision@k(B) recall@k(B) f1@k(B) len@k(B) support@k(B) 1 0.500000 0.25 0.333333 1.0 2 3 0.666667 1.00 0.800000 3.0 2 5 0.666667 1.00 0.800000 3.0 2 10 0.666667 1.00 0.800000 3.0 2 auc: 0.250000 map: 0.416667 mrr: 0.416667 coverage_error: 2.500000 ranking_loss: 0.750000 len: 3.000000 support: 2 map(B): 0.708333 mrr(B): 0.750000 >>> ranking_report(y_true, y_pred, bottom=True, metrics=["auc"]) # doctest: +NORMALIZE_WHITESPACE auc: 0.250000 len: 3.000000 support: 2 >>> y_true = [[0.9, 0.7, 0.1], [0, 0.5, 1]] >>> y_pred = [[0.75, 0.5, 1], [1, 0.2, 0.1]] >>> ranking_report(y_true, y_pred, continuous=True) # doctest: +NORMALIZE_WHITESPACE ndcg@k len@k support@k 3 0.675647 3.0 2 5 0.675647 3.0 2 10 0.675647 3.0 2 mrr: 0.750000 len: 3.000000 support: 2 >>> y_true = [[1, 0], [0, 0, 1]] >>> y_pred = [[0.75, 0.5], [1, 0.2, 0.1]] >>> ranking_report(y_true, y_pred) # doctest: +NORMALIZE_WHITESPACE ndcg@k precision@k recall@k f1@k len@k support@k 1 1.00 0.500000 0.5 0.500000 1.0 2 3 0.75 0.416667 1.0 0.583333 2.5 2 5 0.75 0.416667 1.0 0.583333 2.5 2 10 0.75 0.416667 1.0 0.583333 2.5 2 auc: 0.500000 map: 0.666667 mrr: 0.666667 coverage_error: 2.000000 ranking_loss: 0.500000 len: 2.500000 support: 2 >>> ranking_report(y_true, y_pred, coerce="abandon") # doctest: +NORMALIZE_WHITESPACE ndcg@k precision@k recall@k f1@k len@k support@k 1 1.0 0.500000 0.5 0.5 1.0 2 3 0.5 0.333333 1.0 0.5 3.0 1 auc: 0.500000 map: 0.666667 mrr: 0.666667 coverage_error: 2.000000 ranking_loss: 0.500000 len: 2.500000 support: 2 >>> ranking_report(y_true, y_pred, coerce="padding") # doctest: +NORMALIZE_WHITESPACE ndcg@k precision@k recall@k f1@k len@k support@k 1 1.00 0.500000 0.5 0.500000 1.0 2 3 0.75 0.416667 1.0 0.583333 2.5 2 5 0.75 0.416667 1.0 0.583333 2.5 2 10 0.75 0.416667 1.0 0.583333 2.5 2 auc: 0.500000 map: 0.666667 mrr: 0.666667 coverage_error: 2.000000 ranking_loss: 0.500000 len: 2.500000 support: 2 >>> ranking_report(y_true, y_pred, bottom=True) # doctest: +NORMALIZE_WHITESPACE ndcg@k precision@k recall@k f1@k len@k support@k ndcg@k(B) \ 1 1.00 0.500000 0.5 0.500000 1.0 2 1.000000 3 0.75 0.416667 1.0 0.583333 2.5 2 0.846713 5 0.75 0.416667 1.0 0.583333 2.5 2 0.846713 10 0.75 0.416667 1.0 0.583333 2.5 2 0.846713 <BLANKLINE> precision@k(B) recall@k(B) f1@k(B) len@k(B) support@k(B) 1 0.500000 0.5 0.500000 1.0 2 3 0.583333 1.0 0.733333 2.5 2 5 0.583333 1.0 0.733333 2.5 2 10 0.583333 1.0 0.733333 2.5 2 auc: 0.500000 map: 0.666667 mrr: 0.666667 coverage_error: 2.000000 ranking_loss: 0.500000 len: 2.500000 support: 2 map(B): 0.791667 mrr(B): 0.750000 >>> ranking_report(y_true, y_pred, bottom=True, coerce="abandon") # doctest: +NORMALIZE_WHITESPACE ndcg@k precision@k recall@k f1@k len@k support@k ndcg@k(B) \ 1 1.0 0.500000 0.5 0.5 1.0 2 1.000000 3 0.5 0.333333 1.0 0.5 3.0 1 0.693426 <BLANKLINE> precision@k(B) recall@k(B) f1@k(B) len@k(B) support@k(B) 1 0.500000 0.5 0.5 1.0 2 3 0.666667 1.0 0.8 3.0 1 auc: 0.500000 map: 0.666667 mrr: 0.666667 coverage_error: 2.000000 ranking_loss: 0.500000 len: 2.500000 support: 2 map(B): 0.791667 mrr(B): 0.750000 >>> ranking_report(y_true, y_pred, bottom=True, coerce="padding") # doctest: +NORMALIZE_WHITESPACE ndcg@k precision@k recall@k f1@k len@k support@k ndcg@k(B) \ 1 1.00 0.500000 0.5 0.500000 1.0 2 1.000000 3 0.75 0.416667 1.0 0.583333 2.5 2 0.846713 5 0.75 0.416667 1.0 0.583333 2.5 2 0.846713 10 0.75 0.416667 1.0 0.583333 2.5 2 0.846713 <BLANKLINE> precision@k(B) recall@k(B) f1@k(B) len@k(B) support@k(B) 1 0.50 0.5 0.500000 1.0 2 3 0.50 1.0 0.650000 3.0 2 5 0.30 1.0 0.452381 5.0 2 10 0.15 1.0 0.257576 10.0 2 auc: 0.500000 map: 0.666667 mrr: 0.666667 coverage_error: 2.000000 ranking_loss: 0.500000 len: 2.500000 support: 2 map(B): 0.791667 mrr(B): 0.750000