Hjem

Introduction xxi
Chapter 1 The Two Essential Algorithms for Making Predictions 1 Why are These Two Algorithms So Useful? 2 What are Penalized Regression Methods? 7 What are Ensemble Methods? 9 How to Decide Which Algorithm to Use 11 The Process Steps for Building a Predictive Model 13 Framing a Machine Learning Problem 15 Feature Extraction and Feature Engineering 17 Determining Performance of a Trained Model 18
Chapter Contents and Dependencies 18 Summary 20
Chapter 2 Understand the Problem by Understanding the Data 23 The Anatomy of a New Problem 24 Different Types of Attributes and Labels Drive Modeling Choices 26 Things to Notice about Your New Data Set 27 Classification Problems: Detecting Unexploded Mines Using Sonar 28 Physical Characteristics of the Rocks Versus Mines Data Set 29 Statistical Summaries of the Rocks Versus Mines Data Set 32 Visualization of Outliers Using a Quantile-Quantile Plot 34 Statistical Characterization of Categorical Attributes 35 How to Use Python Pandas to Summarize the Rocks Versus Mines Data Set 36 Visualizing Properties of the Rocks Versus Mines Data Set 39 Visualizing with Parallel Coordinates Plots 39 Visualizing Interrelationships between Attributes and Labels 41 Visualizing Attribute and Label Correlations Using a Heat Map 48 Summarizing the Process for Understanding the Rocks Versus Mines Data Set 50 Real-Valued Predictions with Factor Variables: How Old is Your Abalone? 50 Parallel Coordinates for Regression Problems--Visualize Variable Relationships for the Abalone Problem 55 How to Use a Correlation Heat Map for Regression--Visualize Pair-Wise Correlations for the Abalone Problem 59 Real-Valued Predictions Using Real-Valued Attributes: Calculate How Your Wine Tastes 61 Multiclass Classification Problem: What Type of Glass is That? 67 Using PySpark to Understand Large Data Sets 72 Summary 75
Chapter 3 Predictive Model Building: Balancing Performance, Complexity, and Big Data 77 The Basic Problem: Understanding Function Approximation 78 Working with Training Data 79 Assessing Performance of Predictive Models 81 Factors Driving Algorithm Choices and Performance--Complexity and Data 82 Contrast between a Simple Problem and a Complex Problem 82 Contrast between a Simple Model and a Complex Model 85 Factors Driving Predictive Algorithm Performance 89 Choosing an Algorithm: Linear or Nonlinear? 90 Measuring the Performance of Predictive Models 91 Performance Measures for Different Types of Problems 91 Simulating Performance of Deployed Models 105 Achieving Harmony between Model and Data 107 Choosing a Model to Balance Problem Complexity, Model Complexity, and Data Set Size 107 Using Forward Stepwise Regression to Control Overfitting 109 Evaluating and Understanding Your Predictive Model 114 Control Overfitting by Penalizing Regression Coefficients--Ridge Regression 116 Using PySpark for Training Penalized Regression Models on Extremely Large Data Sets 124 Summary 127
Chapter 4 Penalized Linear Regression 129 Why Penalized Linear Regression Methods are So Useful 130 Extremely Fast Coefficient Estimation 130 Variable Importance Information 131 Extremely Fast Evaluation When Deployed 131 Reliable Performance 131 Sparse Solutions 132 Problem May Require Linear Model 132 When to Use Ensemble Methods 132 Penalized Linear Regression: Regulating Linear Regression for Optimum Performance 132 Training Linear Models: Minimizing Errors and More 135 Adding a Coefficient Penalty to the OLS Formulation 136 Other Useful Coefficient Penalties--Manhattan and ElasticNet 137 Why Lasso Penalty Leads to Sparse Coefficient Vectors 138 ElasticNet Penalty Includes Both Lasso and Ridge 140 Solving the Penalized Linear Regression Problem 141 Understanding Least Angle Regression and Its Relationship to Forward Stepwise Regression 141 How LARS Generates Hundreds of Models of Varying Complexity 145 Choosing the Best Model from the Hundreds LARS Generates 147 Using Glmnet: Very Fast and Very General 152 Comparison of the Mechanics of Glmnet and LARS Algorithms 153 Initializing and Iterating the Glmnet Algorithm 153 Extension of Linear Regression to Classification Problems 157 Solving Classification Problems with Penalized Regression 157 Working with Classification Problems Having More Than Two Outcomes 161 Understanding Basis Expansion: Using Linear Methods on Nonlinear Problems 161 Incorporating Non-Numeric Attributes into Linear Methods 163 Summary 166
Chapter 5 Building Predictive Models Using Penalized Linear Methods 169 Python Packages for Penalized Linear Regression 170 Multivariable Regression: Predicting Wine Taste 171 Building and Testing a Model to Predict Wine Taste 172 Training on the Whole Data Set before Deployment 175 Basis Expansion: Improving Performance by Creating New Variables from Old Ones 179 Binary Classification: Using Penalized Linear Regression to Detect Unexploded Mines 182 Build a Rocks Versus Mines Classifier for Deployment 191 Multiclass Classification: Classifying Crime Scene Glass Samples 200 Linear Regression and Classification Using PySpark 203 Using PySpark to Predict Wine Taste 204 Logistic Regression with PySpark: Rocks Versus Mines 208 Incorporating Categorical Variables in a PySpark Model: Predicting Abalone Rings 213 Multiclass Logistic Regression with Meta Parameter Optimization 217 Summary 219
Chapter 6 Ensemble Methods 221 Binary Decision Trees 222 How a Binary Decision Tree Generates Predictions 224 How to Train a Binary Decision Tree 225 Tree Training Equals Split Point Selection 227 How Split Point Selection Affects Predictions 228 Algorithm for Selecting Split Points 229 Multivariable Tree Training--Which Attribute to Split? 229 Recursive Splitting for More Tree Depth 230 Overfitting Binary Trees 231 Measuring Overfit with Binary Trees 231 Balancing Binary Tree Complexity for Best Performance 232 Modifi cations for Classification and Categorical Features 235 Bootstrap Aggregation: "Bagging" 235 How Does the Bagging Algorithm Work? 236 Bagging Performance--Bias Versus Variance 239 How Bagging Behaves on Multivariable Problem 241 Bagging Needs Tree Depth for Performance 245 Summary of Bagging 246 Gradient Boosting 246 Basic Principle of Gradient Boosting Algorithm 246 Parameter Settings for Gradient Boosting 249 How Gradient Boosting Iterates toward a Predictive Model 249 Getting the Best Performance from Gradient Boosting 250 Gradient Boosting on a Multivariable Problem 253 Summary for Gradient Boosting 256 Random Forests 256 Random Forests: Bagging Plus Random Attribute Subsets 259 Random Forests Performance Drivers 260 Random Forests Summary 261 Summary 262
Chapter 7 Building Ensemble Models with Python 265 Solving Regression Problems with Python Ensemble Packages 265 Using Gradient Boosting to Predict Wine Taste 266 Using the Class Constructor for GradientBoostingRegressor 266 Using GradientBoostingRegressor to Implement a Regression Model 268 Assessing the Performance of a Gradient Boosting Model 271 Building a Random Forest Model to Predict Wine Taste 272 Constructing a RandomForestRegressor Object 273 Modeling Wine Taste with RandomForestRegressor 275 Visualizing the Performance of a Random Forest Regression Model 279 Incorporating Non-Numeric Attributes in Python Ensemble Models 279 Coding the Sex of Abalone for Gradient Boosting Regression in Python 280 Assessing Performance and the Importance of Coded Variables with Gradient Boosting 282 Coding the Sex of Abalone for Input to Random Forest Regression in Python 284 Assessing Performance and the Importance of Coded Variables 287 Solving Binary Classification Problems with Python Ensemble Methods 288 Detecting Unexploded Mines with Python Gradient Boosting 288 Determining the Performance of a Gradient Boosting Classifier 291 Detecting Unexploded Mines with Python Random Forest 292 Constructing a Random Forest Model to Detect Unexploded Mines 294 Determining the Performance of a Random Forest Classifier 298 Solving Multiclass Classification Problems with Python Ensemble Methods 300 Dealing with Class Imbalances 301 Classifying Glass Using Gradient Boosting 301 Determining the Performance of the Gradient Boosting Model on Glass Classification 306 Classifying Glass with Random Forests 307 Determining the Performance of the Random Forest Model on Glass Classification 310 Solving Regression Problems with PySpark Ensemble Packages 311 Predicting Wine Taste with PySpark Ensemble Methods 312 Predicting Abalone Age with PySpark Ensemble Methods 317 Distinguishing Mines from Rocks with PySpark Ensemble Methods 321 Identifying Glass Types with PySpark Ensemble Methods 325 Summary 327 Index 329