Yixuan Zhu assignment7

Assignment7.Rmd

@@ -1,7 +1,7 @@
 ---
 title: "Assignment 7 - Answers"
-author: "Charles Lang"
-date: "11/30/2016"
+author: "Leona Zhu"
+date: "11/30/2019"
 output: html_document
 ---
 
@@ -11,27 +11,46 @@ In the following assignment you will be looking at data from an one level of an
 
 #Upload data
 ```{r}
-
+library(purrr)
+library(tidyr)
+library(dplyr)
+library(ggplot2)
+#install.packages("GGally")
+library(GGally)
+library(rpart)
+D1 <- read.csv("online.data.csv")
 ```
 
 #Visualization 
 ```{r}
 #Start by creating histograms of the distributions for all variables (#HINT: look up "facet" in the ggplot documentation)
+D1 %>%keep(is.numeric)%>% gather() %>% 
+  ggplot(aes(value)) +
+    facet_wrap(~ key, scales = "free") +
+    geom_histogram()
 
 #Then visualize the relationships between variables
+ggpairs(select(D1, -id))
 
 #Try to capture an intution about the data and the relationships
 
+#messages have strong correlation with post test score and average assignment score. It shows that student send more messages talk about the assiggnment with other students get better score on post test and assignments.
 ```
 #Classification tree
 ```{r}
 #Create a classification tree that predicts whether a student "levels up" in the online course using three variables of your choice (As we did last time, set all controls to their minimums)
 
+ctree_1 <- rpart(level.up ~ pre.test.score + messages + forum.posts,
+                        data = D1)
+
 #Plot and generate a CP table for your tree 
+printcp(ctree_1)
 
 #Generate a probability value that represents the probability that a student levels up based your classification tree 
+ctree_2<- prune.rpart(ctree_1, cp = 0.01125)
+printcp(ctree_2)
 
-D1$pred <- predict(rp, type = "prob")[,2]#Last class we used type = "class" which predicted the classification for us, this time we are using type = "prob" to see the probability that our classififcation is based on.
+D1$pred <- predict(ctree_1, type = "prob")[,2]#Last class we used type = "class" which predicted the classification for us, this time we are using type = "prob" to see the probability that our classififcation is based on.
 ```
 ## Part II
 #Now you can generate the ROC curve for your model. You will need to install the package ROCR to do this.
@@ -40,28 +59,48 @@ library(ROCR)
 
 #Plot the curve
 pred.detail <- prediction(D1$pred, D1$level.up) 
-plot(performance(pred.detail, "tpr", "fpr"))
-abline(0, 1, lty = 2)
+plot(performance(pred.detail, "tpr", "fpr"))+ abline(0, 1, lty = 2)
 
 #Calculate the Area Under the Curve
-unlist(slot(performance(Pred2,"auc"), "y.values"))#Unlist liberates the AUC value from the "performance" object created by ROCR
+unlist(slot(performance(pred.detail,"auc"), "y.values"))#Unlist liberates the AUC value from the "performance" object created by ROCR
 
 #Now repeat this process, but using the variables you did not use for the previous model and compare the plots & results of your two models. Which one do you think was the better model? Why?
+ctree_3<-rpart((level.up)~post.test.score + av.assignment.score,method="class",data=D1)
+printcp(ctree_3)
+
+D1$pred1 <- predict(ctree_3, type = "prob")[,2] #what is [,2]?
+pred.detail1<-prediction(D1$pred1, D1$level.up)
+
+plot(performance(pred.detail1, "tpr", "fpr"))+abline(0, 1, lty = 2,)
+
+unlist(slot(performance(pred.detail1,"auc"), "y.values"))
+
 ```
+##The auc of first model is 0.88;The auc of second model is 1.0 and the xerror of second model is 0. Which means the second model is perfect. Therefore the second model is better.
+
+
 ## Part III
 #Thresholds
 ```{r}
 #Look at the ROC plot for your first model. Based on this plot choose a probability threshold that balances capturing the most correct predictions against false positives. Then generate a new variable in your data set that classifies each student according to your chosen threshold.
 
-threshold.pred1 <- 
+#I use 0.7 as my threshold.
+
+D1$threshold.pred1 <- ifelse(D1$pred >= 0.7, 1, 0)
 
 #Now generate three diagnostics:
 
-D1$accuracy.model1 <-
+D1$accuracy.model1 <- mean(ifelse(D1$level.up == D1$threshold.pred1, 1, 0))
 
-D1$precision.model1 <- 
+##True positive,false positive, false negative
+D1$truepos<-ifelse(D1$level.up == 1 & D1$threshold.pred1 == 1,1,0)
+D1$falsepos<-ifelse(D1$level.up == 0 & D1$threshold.pred1 == 1,1,0)
+D1$falseneg<-ifelse(D1$level.up == 1 & D1$threshold.pred1 == 0,1,0)
 
-D1$recall.model1 <- 
+
+D1$precision.model1 <- sum(D1$truepos.model1)/(sum(D1$truepos.model1) + sum(D1$falsepos.model1))
+
+D1$recall.model1 <- sum(D1$truepos.model1)/(sum(D1$truepos.model1) + sum(D1$falseneg.model1))
 
 #Finally, calculate Kappa for your model according to:
 
@@ -76,6 +115,29 @@ kappa(matrix1, exact = TRUE)/kappa(matrix1)
 
 #Now choose a different threshold value and repeat these diagnostics. What conclusions can you draw about your two thresholds?
 
+#I use 0.5 this time.
+
+D1$threshold.pred2 <- ifelse(D1$pred1 > 0.5, 1,0)
+D1$accuracy.model2 <- mean(ifelse(D1$level.up == D1$threshold.pred2, 1, 0))
+
+
+D1$truepos.model2 <- ifelse(D1$level.up == "1" & D1$threshold.pred2 == "1", 1, 0)
+D1$falsepos.model2 <- ifelse(D1$level.up == "0" & D1$threshold.pred2 == "1", 1,0)
+D1$falseneg.model2 <- ifelse(D1$level.up == "1" & D1$threshold.pred2 == "0", 1,0)
+D1$precision.model2 <- sum(D1$truepos.model2)/(sum(D1$truepos.model2) + sum(D1$falsepos.model2))
+D1$recall.model2 <-  sum(D1$truepos.model2)/(sum(D1$truepos.model2) + sum(D1$falseneg.model2))
+
+#Table
+table2 <- table(D1$level.up, D1$threshold.pred2)
+#Matrix
+matrix2 <- as.matrix(table2)
+#Kappa
+kappa(matrix2, exact = TRUE)/kappa(matrix2)
+
+matrix1
+matrix2
+
+##Model 1 kappa value is 1.04; Model 2 kappa value is 1.15. Theredore model 2 is better.
 ```
 
 ### To Submit Your Assignment