CS539 Machine Learning. Spring 2007 
Project 1 - Using the Weka System to Preprocess Datasets

PROF. CAROLINA RUIZ 

DUE DATE: Thursday, Jan 25th 2007. Slides are due at 3:00 (by email) and a hardcopy of the written report is due at 4:00 pm (beginning of class). 

• Project Description
• Project Assignment
• Report Submission and Due Date

PROJECT DESCRIPTION

• to gain familiarity with the Weka system, its GUI, its code, and its input data format (arff).

• To gain experience "pre-processing" datasets to clean, normalize, and discretize data attributes, and, when needed, reduce the dimensionality of the data.

PROJECT ASSIGNMENT

1. You can download and use the latest stable GUI version of the system though I suggest to download the developer version (currently weka-3-5-4) which offers added functionality.

2. Read in detail the "Explorer Guide" and the "Experimenter Tutorial" provided with the Weka system. Browse through the "Package Documentation" to become familiar with it.

3. Datasets: Consider the following sets of data:
   1. The weather data (available in the data directory of the Weka system as the "weather.arff" file).
   2. The iris data (available in the data directory of the Weka system as the "iris.arff" file).
   3. The Automobile Database taken for the UCI Machine Learning Repository.
   4. A dataset of your choice. This dataset can consist of data that you use for your own research or work, a dataset taken from a public data repository (e.g., UCI Machine Learning Repository, or from the UCI KDD Archive), or data that you collect from public data sources. THIS DATASET CANNOT BE ONE OF THOSE INCLUDED IN THE WEKA SYSTEM.

4. Experiments: For each of the above datasets:

   1. Translate the dataset into the arff format if needed.

   2. Use the "Explorer" option of the Weka system to perform the following operations:
      • Open the dataset in Weka.
      • Preprocess the dataset attributes using Weka's filters. In particular,
        • explore different ways of discretizing continuous attributes. That is, convert numeric attributes into "nominal" ones by binning numeric values into intervals - See the weka.filter.DiscretizeFilter in Weka. Play with the filter and read the Java code implementing it.
        • explore different ways of removing missing values. Missing values in arff files are represented with the character "?". See the weka.filter.ReplaceMissingValuesFilter in Weka. Play with the filter and read the Java code implementing it.

   3. Use both the "Explorer" and the "Experimenter" options of the Weka system in turn to run the "ZeroR" and the "OneR" classifiers (under the "Classify" tab) over the above datasets. Use different ways of testing your results. That is, explore the following alternatives offered by the Weka system:
      • Testing your results over the training data.
      • Splitting your input file into two parts one for training and one for testing.
      • Using n-fold crossvalidation. Play with different values for n.
      Analyze the results obtained (i.e. interpret the meaning of the output produced by Weka). Read to the Java code implementing the ZeroR and the OneR classifiers.

   4. Run several experiments with your data and the system varying the parameters so that you gain familiarity with the system.

ORAL AND WRITTEN REPORTS AND DUE DATE

• Written Report. Please hand in a hardcopy of your report at the beginning of class when the project is due. Your report should contain the following sections with the corresponding discussions:

  1. Data: Describe the datasets that you used in terms of the attributes present in the data, the number of instances, missing values, and other relevant characteristics.

  2. Code Description: Describe to the extent possible any observations you made when looking at the Weka code implementing the filters you used and the ZeroR and OneR functions.

  3. Experiments: For each experiment you ran describe:
     • Instances: What data did you use for the experiments? That is, did you use the entire dataset of just a subset of it?
     • Any pre-processing done to the data. That is, did you remove any attributes? Did you discretize any continuous attribute? If so, what strategy did you use to bin the values? Did you replace missing values? If so, what strategy did you use to select a replacement of the missing values?
     • Your system parameters.
     • For the ZeroR function, analysis of results of the experiments you ran using different ways of testing the classifier (crossvalidation, etc.).
     • For the OneR function, analysis of results of the experiments you ran using different ways of testing the classifier (crossvalidation, etc.).

  4. Summary of Results
     • Discuss the strengths and the weaknesses of your project.

• Oral Report. We will discuss the results from the individual projects during the class on Thursday, Jan 27. Each of you will have approximately 7 minutes to present your report. Prepare detailed SLIDES with the results of your experiments. Your slides should be a good "preview" of your written report and should summarize the contents of the different sections of your written report as described above. Be ready to show your results and to discuss your project in class within the time allowed.