4  Performing cross-validation with the caret package

并包装卡雷特进行交叉验证

caret packageThe Caret (classification and regression training) package contains many functions in regard to the training process for regression and classification problems. Similar to the e1071 package, it also contains a function to perform the k-fold cross validation. In this recipe, we will demonstrate how to the perform k-fold cross validation using the caret package.

（分类和回归训练）包中所包含的关于回归和分类问题的训练过程中的许多功能。类似的e1071包，它还包含了一个函数来实现交叉验证。在这个食谱中，我们将演示如何执行交叉验证使用

Getting ready

准备

In this recipe, we will continue to use the telecom churn dataset as the input data source to perform the k-fold cross validation

在这个食谱中，我们将继续使用电信客户流失数据集作为输入数据源进行交叉验证

How to do it...

怎么做

Perform the following steps to perform the k-fold cross-validation with the caret package:执行以下步骤并封装进行交叉验证：

1. First, set up the control parameter to train with the 10-fold cross validation in 3

repetitions:

> control = trainControl(method="repeatedcv", number=10,

 repeats=3)

2. Then, you can train the classification model on telecom churn data with rpart:

2。然后，你可以训练分类模型对电信客户流失数据：

> model = train(churn~., data=http://www.mamicode.com/trainset, method="rpart",

 

preProcess="scale", trControl=control)

3. Finally, you can examine the output of the generated model:

3。最后，您可以检查生成的模型的输出：

> model

 CART

 2315 samples

16 predictor

 2 classes: ‘yes‘, ‘no‘

 Pre-processing: scaled

 Resampling: Cross-Validated (10 fold, repeated 3 times)

 Summary of sample sizes: 2084, 2083, 2082, 2084, 2083, 2084, ...

 Resampling results across tuning parameters:

  cp Accuracy Kappa Accuracy SD Kappa SD

 0.0556 0.904 0.531 0.0236 0.155

 0.0746 0.867 0.269 0.0153 0.153

 0.0760 0.860 0.212 0.0107 0.141

  Accuracy was used to select the optimal model using the largest value.

The final value used for the model was cp = 0.05555556.

How it works...

如何工作

In this recipe, we demonstrate how convenient it is to conduct the k-fold cross-validation using the caret package. In the first step, we set up the training control and select the option to perform the 10-fold cross-validation in three repetitions. The process of repeating the k-fold validation is called repeated k-fold validation, which is used to test the stability of the model. If

the model is stable, one should get a similar test result. Then, we apply rpart on the training dataset with the option to scale the data and to train the model with the options configured in

the previous step.

它如何工作…在这个食谱中，我们演示了使用符号包进行交叉验证是如何方便的。在第一步中，我们设置了训练控制，并选择选项执行10倍交叉验证在三次重复。重复折验证的过程称为重复折验证，这是用来测试的稳定性

See also

f You can configure the resampling function in trainControl, in which you can

specify boot, boot632, cv, repeatedcv, LOOCV, LGOCV, none, oob, adaptive_

cv, adaptive_boot, or adaptive_LGOCV. To view more detailed information of

how to choose the resampling method, view the trainControl document:

> ?trainControl

又见F可以控制、配置重采样功能，您可以在其中指定启动，boot632，CV，repeatedcv，LOOCV，lgocv，无，OOB，adaptive_ CV，adaptive_boot，或adaptive_lgocv。查看更详细的信息，如何选择重采样方法，查看控制、文档：>？列控

5.

Ranking the variable importance with the rminer package

排序的变量的重要性与rminer包

Besides using the caret package to generate variable importance, you can use the rminerpackage to generate the variable importance of a classification model. In the following recipe, we will illustrate how to use rminer to obtain the variable importance of a fitted model.Getting readyIn this recipe, we will continue to use the telecom churn dataset as the input data source to rank the va

除了使用符号包产生变量的重要性，你可以使用rminer包产生一个分类模型的变量的重要性。在下面的食谱，我们将说明如何使用rminer获得拟合模型的变量的重要性。准备在这个食谱中，我们将继续使用的电信流失数据集作为输入数据源排名VA

How to do it...Perform the following steps to rank the variable importance with rminer:1. Install and load the package, rminer:> install.packages("rminer")> library(rminer)2. Fit the svm model with the training set:> model=fit(churn~.,trainset,model="svm")3. Use the Importance function to obtain the variable importance:> VariableImportance=Importance(model,trainset,method="sensv")4. Plot the variable importance ranked by the variance:> L=list(runs=1,sen=t(VariableImportance$imp),sresponses=VariableImportance$sresponses)> mgraph(L,graph="IMP",leg=names(trainset),col="gray",Grid=10)Figure 2: The visualization of variable importance using the rminer package

Similar to the caret package, the rminer package can also generate the variable importance of a classification model. In this recipe, we first train the svm model on the training dataset, trainset, with the fit function. Then, we use the Importance function to rank the variable importance with a sensitivity measure. Finally, we use mgraph to plot the rank of the variable importance. Simila

类似于符号的rminer包，包也可以产生一个分类模型的变量的重要性。在这个食谱中，我们首先训练SVM模型的训练数据集，动车组，与拟合函数。然后，我们使用的重要性功能排名的变量重要性的敏感性措施。最后，我们使用MGraph绘制变量重要性排序。

6.Finding highly correlated features with the caret package

寻找高度相关的特征并包装

When performing regression or classification, some models perform better if highly correlated attributes are removed. The caret package provides the findCorrelation function, which can be used to find attributes that are highly correlated to each other. In this recipe, we will demonstrate how to find highly correlated features using the caret package.

当进行回归或分类，一些模型表现更好，如果高度相关的属性被删除。插入findcorrelation包提供的功能，它可以用来发现是彼此高度相关的属性。在这个食谱中，我们将展示如何找到高度相关的特征，用符号包。

How to do it...Perform the following steps to find highly correlated attributes:1. Remove the features that are not coded in numeric characters:> new_train = trainset[,! names(churnTrain) %in% c("churn", "international_plan", "voice_mail_plan")]2. Then, you can obtain the correlation of each attribute:>cor_mat = cor(new_train)3. Next, we use findCorrelation to search for highly correlated attributes with a cut off equal to 0.75:> highlyCorrelated = findCorrelation(cor_mat, cutoff=0.75)4. We then obtain the name of highly correlated attributes:> names(new_train)[highlyCorrelated][1] "total_intl_minutes" "total_day_charge" "total_eve_minutes" "total_night_minutes"

  In this recipe, we search for highly correlated attributes using the caret package. In order to retrieve the correlation of each attribute, one should first remove nonnumeric attributes. Then, we perform correlation to obtain a correlation matrix. Next, we use findCorrelation to find highly correlated attributes with the cut off set to 0.75. We finally obtain the names of highly correlated

在这个食谱中，我们寻找高度相关的属性使用插入符号包。为了检索每个属性的相关性，应先去除非数值属性。然后，我们执行相关，得到相关矩阵。接下来，我们使用findcorrelation找到高度相关的属性与切断设置为0.75。我们终于获得高度相关的名称。

小结：利用交叉验证可以更好的方便我们学习和工作，大数据时代带给我们越来越多的便捷。

                                                                                                                      ---------摘自百度翻译

                                                                                                                                                                                                                                                                                   李明玥

文章翻译第七章4-6