simplified iris task

Author: tenosel

README.md

@@ -1,6 +1,6 @@
 # k-Nearest Neighbors Classification Exercise
 
-Today we will get to know the package `scikit-learn` (sklearn). It has many different machine learning algorithms already implemented, so we will be using it for the next five exercise sheets. The first algorithm, which we are going to learn today is the k-nearest neighbor algorithm. It can be used for classification as well as for regression.
+Today we will get to know the package `scikit-learn` (sklearn). It has many different machine learning algorithms already implemented. The first algorithm, which we are going to learn today is the k-nearest neighbor algorithm. It can be used for classification as well as for regression.
 
 Take a look at the file `src/nn_iris.py`. We will implement the TODOs step by step:
 
@@ -13,25 +13,15 @@ Take a look at the file `src/nn_iris.py`. We will implement the TODOs step by st
    or directly via `pip install scikit-learn`.  
    The dataset <em>iris</em> is very popular amongst machine learners in example tasks. For this reason it can be found directly in the sklearn package.
 
-2. Navigate to the `__main__` function of `src/nn_iris.py` and load the iris dataset from `sklearn.datasets`.  
+2. Navigate to the `__main__` function of `src/nn_iris.py`. At first we load the iris dataset from `sklearn.datasets`.  
    In the dataset there are several plants of different species of genus Iris. For each of the examples width and length of petal and sepal of the flower were measured.  
    ![A petal and a sepal of a flower (Wikipedia)](./figures/Petal_sepal.jpg)
 
 3. Find out how to access the attributes of the database (Hint: set a breakpoint and examine the variable). Print the shape of the data matrix and the number of the target entries. Print the names of the labels. Print the names of the features.
 
-### Task 2: Examining the data (optional)
-
-Your goal is to determine the species for an example, based on the dimensions of its petals and sepals. But first we need to inspect the dataset.
-
-1. Use a histogram (classes distribution) to check if the iris dataset is balanced. To plot a histogram you can for example use `pandas.Series.hist` or `matplotlib.pyplot.hist`.
-   Fortunately, the iris dataset is balanced, so it has the same number of samples for each species. Balanced datasets make it simple to proceed directly to the classification phase. In the opposite case we would have to take additional steps to reduce the negative effects (e.g. collect more data) or use other algorithms than the k-Nearest Neighbors (e.g. Random Forests).
-
-2. We also can use pandas `scatter_matrix` to visualize some trends in our data. A scatter matrix (pairs plot) compactly plots all the numeric variables we have in a dataset against each other.  
-   Plot the scatter matrix. To make the different species visually distinguishable use the parameter `c=iris.target` in `pandas.plotting.scatter_matrix` to colorize the datapoints according to their target species.  
-   In the scatter matrix you can see domains of values as well as the distributions of each of the attributes. It is also possible to compare groups in scatter plots over all pairs of attributes. From those it seems that groups are well separated, two of the groups slightly overlap. 
-
-### Task 3: Training
+### Task 2: Training
 
+Your goal is to determine the species for an example, based on the dimensions of its petals and sepals. 
 First, we need to split the dataset into train and test data. Then we are ready to train the model.
 
 1. Use `train_test_split` from `sklearn.model_selection` and create a train and a test set with the ratio 75:25. Print the dimensions of the train and the test set. You can use the parameter `random_state` to set the seed for the random number generator. That will make your results reproducible. Set this value to 29.
@@ -40,36 +30,30 @@ First, we need to split the dataset into train and test data. Then we are ready
 
 3. Train the classifier on the training set. The method `fit()` is present in all the estimators of the package `scikit-learn`.
 
-### Task 4: Prediction and Evalutation
+### Task 3: Prediction and Evalutation
 
 The trained model is now able to receive the input data and produce predictions of the labels.
 1. Predict the labels first for the train and then for the test data. 
 
 2. The comparison of a predicted and the true label can tell us valuable information about how well our model performs. The simplest performance measure is the ratio of correct predictions to all predictions, called accuracy. Implement a function `compute_accuracy` to calculate the accuracy of predictions. Use your function and evaluate your model by calculating the accuracy on the train set and the test set. Print both results.
 
-3. (Optional) To evaluate, whether our model performs well, its performance is compared to other models. Since we now only know one classifier, we will compare it to dummy models. Most frequent models always predict the label that occurs the most in our train set. If the train set is balanced, we choose one of the classes. Implement the function `accuracy_most_frequent` to compute the accuracy of the most frequent model. (Hint: the function `numpy.bincount` might be helpful.) Print the result.
-
-4. (Optional) Another dummy model is a stratified model. A stratified model assigns random labels based on the ratio of the labels in the train set. Implement the function `accuracy_stratified` to compute the accuracy of the stratified model. (Hint: `numpy.random.choice` might help.) Call the function several times and print the results. You see that the results are different. In order to reproduce the results, it is usefull to set a seed. Use `numpy.random.seed` before calling the function to set the seed. Set it to 29. 
-
-### Task 5: Confusion matrix
+### Task 4: Confusion matrix
 
 Another common method to evaluate the performance of a classifier is constructing a confusion matrix that shows not only accuracies for each of the classes (labels), but what classes the classifier is most confused about.
 
 1. Use the function `confusion_matrix` to compute the confusion matrix for the test set.
 
-2. (Optional) The accuracy of the prediction can be derived from the confusion matrix as sum of the matrix diagonal over the sum of the whole matrix. Compute the accuracy using the information obtained from the confusion matrix. Print the result.
-
-3. We can also visualize the confusion matrix in form of a heatmap. Use `ConfusionMatrixDisplay` to plot a heatmap of the confusion matrix for the test set. Use `display_labels=iris.target_names` for better visualization.
+2. We can also visualize the confusion matrix in form of a heatmap. Use `ConfusionMatrixDisplay` to plot a heatmap of the confusion matrix for the test set. Use `display_labels=iris.target_names` for better visualization.
 
-### Task 6: Hyperparameter tuning
+### Task 5: Hyperparameter tuning
 
 Now we need to find the best value for our hyperparameter `k`. We will use a common procedure called <em>grid search</em> to search the space of the possible values. Since our train dataset is small, we will perform cross-validation in order to compute the validation error for each value of `k`. Implement this hyperparameter tuning in the function `cv_knearest_classifier` following these steps:
 
 1. Define a second classifier `knn2`. Define a grid of parameter values for `k` from 1 to 25 (Hint: `numpy.arange`). This grid must be stored in a dictionary with `n_neighbors` as the key in order to use `GridSearchCV` with it.
 
 2. Use the class `GridSearchCV` to perform grid search. It gives you the possibility to perform n-fold cross-validation too, so use the attribute `cv` to set the number of folds to 3. When everything is set, you can train your `knn2`.
 
-### Task 7: Testing
+### Task 6: Testing
 
 After the training you can access the best parameter `best_params_`, the corresponding validation accuracy `best_score_` and the corresponding estimator `best_estimator_`.
 

src/nn_iris.py

@@ -23,37 +23,6 @@ def compute_accuracy(y: np.ndarray, y_pred: np.ndarray) -> float:
     # TODO: Implement me.
     return None
 
-def accuracy_most_frequent(y_train: np.ndarray, y_test: np.ndarray) -> float:
-    """Compute the accuracy of the most frequent model.
-
-    Most frequent models always predict the label that occurs the most in the train set.
-    They belong to the class of so-called "dummy models", because the prediction is
-    independent of the input. Such models usually serve as a baseline if
-    no other models are available.
-
-    Args:
-        y_train (np.ndarray): The array with the training labels.
-        y_test (np.ndarray): The array with the test labels.
-    Returns:
-        float: The accuracy of the prediction by the most frequent model on the test labels.
-    """
-    # TODO: Implement me.
-    return None
-
-# optional
-def accuracy_stratified(y_train: np.ndarray, y_test: np.ndarray) -> float:
-    """Compute the accuracy of the stratified model.
-
-    A stratified model assigns random labels based on the ratio of the labels in the train set.
-
-    Args:
-        y_train (np.ndarray): The array with the training labels.
-        y_test (np.ndarray): The array with the test labels.
-    Returns:
-        float: The accuracy of the prediction by the stratified model on the test labels.
-    """
-    # TODO: Implement me.
-    return None
 
 def cv_knearest_classifier(x_train: np.ndarray, y_train: np.ndarray) -> GridSearchCV:
     """Train and cross-validate a k-nearest neighbors classifier with the grid search.
@@ -80,26 +49,13 @@ def cv_knearest_classifier(x_train: np.ndarray, y_train: np.ndarray) -> GridSear
 
 if __name__ == "__main__":
     # load iris dataset
-    iris = # TODO
+    iris = load_iris()
 
     # print shape of data matrix and number of target entries
     # TODO
     # print names of labels and of features
     # TODO
 
-    # (optional) use classes distribution (histogram) to check if iris dataset is balanced:
-    # find out what the next two lines of code do
-    temp = pd.Series(iris.target)
-    target_str = temp.apply(lambda i: iris.target_names[i])
-    # and use 'pandas.Series.hist' function to plot histogram
-    # TODO
-
-    # (optional) use pandas 'scatter_matrix' to visualize some trends in data:
-    # represent iris as pandas data frame
-    # TODO
-    # create scatter matrix from dataframe, color by target; plot matrix
-    # TODO
-
     # create train and test split with the ratio 75:25 and print their dimensions
     # TODO
 
@@ -117,20 +73,9 @@ def cv_knearest_classifier(x_train: np.ndarray, y_train: np.ndarray) -> GridSear
     # print both accuracies
     # TODO
 
-    # implement and use 'accuracy_most_frequent' to compute accuracy of most frequent model
-    # TODO
-    # print result
-    # TODO
-
-    # (optional) implement and use 'accuracy_stratified' to compute and print accuracy of the stratified model
-    # TODO
-
     # compute confusion matrix for test set
     # TODO
 
-    # (optional) compute and print test set accuracy from confusion matrix
-    # TODO
-
     # plot heatmap of confusion matrix for test set
     # TODO
 

tests/test_nn_iris.py

@@ -8,7 +8,7 @@
 
 sys.path.insert(0, "./src/")
 
-from src.nn_iris import compute_accuracy, accuracy_most_frequent, accuracy_stratified, cv_knearest_classifier
+from src.nn_iris import compute_accuracy, cv_knearest_classifier
 
 
 def test_compute_accuracy():
@@ -17,19 +17,6 @@ def test_compute_accuracy():
     acc = compute_accuracy(y, y_pred)
     assert np.allclose(acc, 0.8)
 
-def test_accuracy_most_frequent():
-    ytrain = np.array([0, 1, 0, 1, 0, 1, 0, 1, 1])
-    ytest = np.array([1, 0, 1, 1, 1])
-    acc_mf = accuracy_most_frequent(ytrain, ytest)
-    assert np.allclose(acc_mf, 0.8)
-
-def test_accuracy_stratified():
-    np.random.seed(42)  # Set the random seed for reproducibility
-    ytrain = np.array([0, 1, 0, 1, 0, 1, 0, 1, 1])
-    ytest = np.array([1, 0, 1, 1, 1])
-    acc_strat = accuracy_stratified(ytrain, ytest)
-    assert np.allclose(acc_strat, 0.4)
-
 def test_cv_knearest_classifier():
     # Create a dummy dataset
     X, y = make_classification(n_samples=100, n_features=20, random_state=42)