# Model performance metrics, leakage and data alerts

*CRM Analytics and Einstein Discovery Consultant Study Guide · Einstein Discovery · Last updated November 3, 2026 · https://www.certifyforce.com/study-guide/crm-analytics-and-einstein-discovery-consultant/einstein-discovery/model-performance-metrics-leakage-and-data-alerts*

Reading AUC and R2 on a model's Performance overview, recognising data leakage and weak predictors, and turning each data alert (strongest predictors, proxy variable, multicollinearity, outliers and more) into a data improvement you can propose to the customer.

- Certification: Salesforce CRM Analytics and Einstein Discovery Consultant
- Exam section: Einstein Discovery
- Study guide: https://www.certifyforce.com/study-guide/crm-analytics-and-einstein-discovery-consultant

**Official exam objective:**

> Given business requirements, analyze the model results and propose data improvements to the customer.

## What the exam expects

- Interpret AUC (binary and multiclass) and R2 (numeric) on a model's Performance overview and say whether the model is ready to deploy.
- Read the evaluation statistics behind a recommendation: accuracy, MCC, the threshold trade-off, precision and recall, and numeric error.
- Recognise data leakage from a suspiciously strong metric or predictor, and name the variable to remove.
- Propose more or better explanatory data when predictors are weak, including historical values the source doesn't keep.
- Turn each data alert into an action: proxy variable, multicollinearity, outliers, imbalanced distribution and leakage alerts.
- Know where alerts are reviewed and what happens when you change a setting or ignore one.

Key terms: **AUC** (area under the curve, the rate of correct classification; 0.5 is random guessing); **R2** (a regression model's ability to explain variation in the outcome, from zero to one); **data leakage** (training data that contains the information you are trying to predict); **proxy variable** (a variable highly correlated with a sensitive variable); **multicollinearity** (two or more highly correlated variables with a duplicate impact on the outcome).

## Key facts

| Fact | Detail | Source |
|---|---|---|
| AUC of 0.5 | No better than random guessing | [Binary Overview Tab](https://help.salesforce.com/s/articleView?id=analytics.bi_edd_model_metrics_overview_category.htm&type=5&language=en_US) |
| AUC of 1.0 | Can indicate data leakage | [Binary Overview Tab](https://help.salesforce.com/s/articleView?id=analytics.bi_edd_model_metrics_overview_category.htm&type=5&language=en_US) |
| AUC quality alert | Below 0.6 or above 0.95 | [AUC Quality Alert](https://help.salesforce.com/s/articleView?id=analytics.bi_edd_quality_alert_auc.htm&type=5&language=en_US) |
| R2 range and alert | 0 to 1; alert below 0.1 or above 0.95 | [R2 Quality Alert](https://help.salesforce.com/s/articleView?id=analytics.bi_edd_quality_alert_r2.htm&type=5&language=en_US) |
| Threshold Evaluation | Slider, optimise a metric, or a false positive to false negative cost ratio | [Threshold Evaluation](https://help.salesforce.com/s/articleView?id=analytics.bi_edd_model_metrics_threshold_evaluation_category.htm&type=5&language=en_US) |
| Numeric error statistics | MAE (less outlier-sensitive) and RMSE | [Numeric Overall Performance](https://help.salesforce.com/s/articleView?id=analytics.bi_edd_model_metrics_overall_measure.htm&type=5&language=en_US) |
| Strongest Predictors alert | R-squared with the outcome above 0.3 | [Strongest Predictors Alert](https://help.salesforce.com/s/articleView?id=analytics.bi_edd_data_alert_correlation_high.htm&type=5&language=en_US) |
| Proxy detection | Only for sensitive variables; over 50% correlated | [Proxy Variable Alert](https://help.salesforce.com/s/articleView?id=analytics.bi_edd_data_alert_proxy_variables.htm&type=5&language=en_US) |
| Multicollinearity test | Cramér's V of 0.5 or higher | [Multicollinearity Alert](https://help.salesforce.com/s/articleView?id=analytics.bi_edd_data_alert_collinearity.htm&type=5&language=en_US) |
| Outlier definition | More than five standard deviations from the global mean | [Outliers Alert](https://help.salesforce.com/s/articleView?id=analytics.bi_edd_data_alert_outliers.htm&type=5&language=en_US) |
| Ignored alert | Moves to the Resolved tab | [Handle Quality Alerts](https://help.salesforce.com/s/articleView?id=analytics.bi_edd_quality_alert.htm&type=5&language=en_US) |
| Prior field values | Snapshots or prior-value custom fields | [Determine Data Requirements](https://help.salesforce.com/s/articleView?id=analytics.bi_edd_prep_requirements.htm&type=5&language=en_US) |

## Check yourself

- Which AUC values make Einstein Discovery raise the AUC quality alert?
- Why might a model show no proxy alert even though a field mirrors ethnicity?
- When does Einstein Discovery treat a numeric value as an outlier?
- What happens to an alert you take no action on?
- Why can a leaky model look excellent in training yet disappoint later?

## Sources

- [Overview Tab for Binary Classification Use Cases](https://help.salesforce.com/s/articleView?id=analytics.bi_edd_model_metrics_overview_category.htm&type=5&language=en_US)
- [Overview Tab for Numeric Use Cases](https://help.salesforce.com/s/articleView?id=analytics.bi_edd_model_metrics_overview_measure.htm&type=5&language=en_US)
- [Overview Tab for Multiclass Classification Use Cases](https://help.salesforce.com/s/articleView?id=analytics.bi_edd_model_metrics_overview_multiclass.htm&type=5&language=en_US)
- [Area Under the Curve (AUC) Quality Alert](https://help.salesforce.com/s/articleView?id=analytics.bi_edd_quality_alert_auc.htm&type=5&language=en_US)
- [R2 Quality Alert](https://help.salesforce.com/s/articleView?id=analytics.bi_edd_quality_alert_r2.htm&type=5&language=en_US)
- [Handle Quality Alerts](https://help.salesforce.com/s/articleView?id=analytics.bi_edd_quality_alert.htm&type=5&language=en_US)
- [Strongest Predictors Alert](https://help.salesforce.com/s/articleView?id=analytics.bi_edd_data_alert_correlation_high.htm&type=5&language=en_US)
- [Potential Data Leakage Alert](https://help.salesforce.com/s/articleView?id=analytics.bi_edd_data_alert_values_constant.htm&type=5&language=en_US)
- [Proxy Variable Alert](https://help.salesforce.com/s/articleView?id=analytics.bi_edd_data_alert_proxy_variables.htm&type=5&language=en_US)
- [Multicollinearity Alert](https://help.salesforce.com/s/articleView?id=analytics.bi_edd_data_alert_collinearity.htm&type=5&language=en_US)
- [Outliers Alert](https://help.salesforce.com/s/articleView?id=analytics.bi_edd_data_alert_outliers.htm&type=5&language=en_US)
- [Imbalanced Distribution Alert](https://help.salesforce.com/s/articleView?id=analytics.bi_edd_data_alert_imbalanced.htm&type=5&language=en_US)
- [Detect and Remove Bias from a Model](https://help.salesforce.com/s/articleView?id=analytics.bi_edd_story_bias.htm&type=5&language=en_US)
- [Determine Data Requirements](https://help.salesforce.com/s/articleView?id=analytics.bi_edd_prep_requirements.htm&type=5&language=en_US)
- [Define Your Target Outcome](https://help.salesforce.com/s/articleView?id=analytics.bi_edd_plan.htm&type=5&language=en_US)
- [Overall Performance Tab for Binary Classification Use Cases](https://help.salesforce.com/s/articleView?id=analytics.bi_edd_model_metrics_overall_category.htm&type=5&language=en_US)
- [Threshold Evaluation for Binary Classification Use Cases](https://help.salesforce.com/s/articleView?id=analytics.bi_edd_model_metrics_threshold_evaluation_category.htm&type=5&language=en_US)
- [Overall Performance Tab for Multiclass Classification Use Cases](https://help.salesforce.com/s/articleView?id=analytics.bi_edd_model_metrics_overall_multiclass.htm&type=5&language=en_US)
- [Overall Performance Tab for Numeric Use Cases](https://help.salesforce.com/s/articleView?id=analytics.bi_edd_model_metrics_overall_measure.htm&type=5&language=en_US)
- [Einstein Discovery Glossary](https://help.salesforce.com/s/articleView?id=analytics.bi_edd_glossary.htm&type=5&language=en_US)

## About the full unit

The "Core concepts", "Exam traps" and "Worked scenario" sections of this unit, its 8 linked practice questions, and unlimited timed mock exams are included with the CRM Analytics and Einstein Discovery Consultant pack (USD $19): https://www.certifyforce.com/practice-exams