# Non-Personalized Recommender Assignment In this assignment, you will implement some non-personalized recommenders. In particular, you will implement raw and damped item mean recommenders and simple and advanced association rule recommenders. You will implement these recommenders in the LensKit toolkit. ## Downloads and Resources - Project template (from Coursera) - [LensKit for Teaching website](http://mooc.lenskit.org) (links to relevant documentation) - [JavaDoc for included code](http://mooc.lenskit.org/assignments/nonpers/javadoc/) - [Fastutil API docs](http://fastutil.di.unimi.it/docs/) documents the Fastutil optimized data structure classes that are used in portions of LensKit. The project template contains support code, the build file, and the input data that you will use. ## Input Data The input data contains the following files: - `ratings.csv` contains user ratings of movies - `movies.csv` contains movie titles - `movielens.yml` is a LensKit data manifest that describes the other input files ## Getting Started To get started with this assignment, unpack the template and import it in to your IDE as a Gradle project. The assignment video demonstrates how to do this in IntelliJ IDEA. ## Mean-Based Recommendation The first two recommenders you will implement will recommend items with the highest average rating. With LensKit's scorer-model-builder architecture, you will just need to write the recommendation logic once, and you will implement two different mechanisms for computing item mean ratings. You will work with the following classes: - `MeanItemBasedItemRecommender` (the *item recommender*) computes top-*N* recommendations based on mean ratings. You will implement the logic to compute such recommendation lists. - `ItemMeanModel` is a *model class* that stores precomputed item means. You will not need to modify this class, but you will write code to construct instances of it and use it in your item recommender implementation. - `ItemMeanModelProvider` computes item mean ratings from rating data and constructs the model. It computes raw means with no damping. - `DampedItemMeanModelProvider` is an alternate builder for item mean models that computes damped means instead of raw means. It takes the damping term as a parameter. The configuration file we provide you uses a damping term of 5. There are `// TODO` comments in all places where you need to write new code. ### Computing Item Means Modify the `ItemMeanModelProvider` class to compute the mean rating for each item. ### Recommending Items Modify the `MeanItemBasedItemRecommender` class to compute recommendations based on item mean ratings. For this, you need to: 1. Obtain the mean rating for each item 2. Order the items in decreasing order 3. Return the *N* highest-rated items ### Computing Damped Item Means Modify the `DampedItemMeanModelProvider` class to compute the damped mean rating for each item. This formula uses a damping factor $\alpha$, which is the number of 'fake' ratings at the global mean to assume for each item. In the Java code, this is available as the field `damping`. The damped mean formula, as you may recall, is: $$s(i) = \frac{\sum_{u \in U_i} r_{ui} + \alpha\mu}{|U_i| + \alpha}$$ where $\mu$ is the *global* mean rating. ### Example Outputs To help you see if your output is correct, we have provided the following example correct values: | ID | Title | Mean | Damped Mean | | :-: | :---- | :--: | :---------: | | 2959 | *Fight Club* | 4.259 | 4.252 | | 1203 | *12 Angry Men* | 4.246 | 4.227 | ## Association Rules In the second part of the assignment, you will implement two versions of an association rule recommender. The association rule implementation consists of the following code: - `AssociationItemBasedItemRecommender` recommends items using association rules. Unlike the mean recommenders, this recommender uses a *reference item* to compute the recommendations. - `AssociationModel` stores the association rule scores between pairs of items. You will not need to modify this class. - `BasicAssociationModelProvider` computes an association rule model using the basic association rule formula ($P(X \wedge Y) / P(X)$). - `LiftAssociationModelProvider` computes an association rule model using the lift formula ($P(X \wedge Y) / P(X) P(Y)$). ### Computing Association Scores Like with the mean-based recommender, we pre-compute product association scores and store them in a model before recommendation. We compute the scores between *all pairs* of items, so that the model can be used to score any item. When computing a single recommendation from the command line, this does not provide much benefit, but is useful in the general case so that the model can be used to very quickly compute many recommendations. The `BasicAssociationModelProvider` class computes the association rule scores using the following formula: $$P(i|j) = \frac{P(i \wedge j)}{P(j))} = \frac{|U_i \cap U_j|/|U|}{|U_j|/|U|}$$ In this case, $j$ is the *reference* item and $i$ is the item to be scored. We estimate probabilities by counting: $P(i)$ is the fraction of users in the system who purchased item $i$; $P(i \wedge j)$ is the fraction that purchased both $i$ and $j$. **Implement the association rule computation in this class.** ### Computing Recommendations Implement the recommendation logic in `AssociationItemBasedItemRecommender` to recommend items related to a given reference item. As with the mean recommender, it should compute the top *N* recommendations and return them. ### Computing Advanced Association Rules The `LiftAssociationModelProvider` recommender uses the *lift* metric that computes how much more likely someone is to rate a movie $i$ when they have rated $j$ than they would have if we do not know anything about whether they have rated $j$: $$s(i|j) = \frac{P(j \wedge i)}{P(i) P(j)}$$ ### Example Outputs Following is the correct output for the basic association rules with reference item 260 (*Star Wars*), as generated with `./gradlew runBasicAssoc -PreferenceItemm=260`: 2571 (Matrix, The (1999)): 0.916 1196 (Star Wars: Episode V - The Empire Strikes Back (1980)): 0.899 4993 (Lord of the Rings: The Fellowship of the Ring, The (2001)): 0.892 1210 (Star Wars: Episode VI - Return of the Jedi (1983)): 0.847 356 (Forrest Gump (1994)): 0.843 5952 (Lord of the Rings: The Two Towers, The (2002)): 0.841 7153 (Lord of the Rings: The Return of the King, The (2003)): 0.830 296 (Pulp Fiction (1994)): 0.828 1198 (Raiders of the Lost Ark (Indiana Jones and the Raiders of the Lost Ark) (1981)): 0.791 480 (Jurassic Park (1993)): 0.789 And lift-based association rules for item 2761 (*The Iron Giant*): 631 (All Dogs Go to Heaven 2 (1996)): 4.898 2532 (Conquest of the Planet of the Apes (1972)): 4.810 3615 (Dinosaur (2000)): 4.546 1649 (Fast, Cheap & Out of Control (1997)): 4.490 340 (War, The (1994)): 4.490 1016 (Shaggy Dog, The (1959)): 4.490 2439 (Affliction (1997)): 4.490 332 (Village of the Damned (1995)): 4.377 2736 (Brighton Beach Memoirs (1986)): 4.329 3213 (Batman: Mask of the Phantasm (1993)): 4.317 ## Running your code The Gradle build file we have provided is set up to automatically run all four of your recommenders. The following Gradle targets will do this: - `runMean` runs the raw mean recommender - `runDampedMean` runs the damped mean recommender - `runBasicAssoc` runs the basic association rule recommender - `runLiftAssoc` runs the advanced (lift-based) association rule recommender You can run these using the IntelliJ Gradle runner (open the Gradle panel, browse the tree to find a task, and double-click it), or from the command line: ./gradlew runMean The association rule recommenders can also take the reference item ID on the command line as a `referenceItem` parameter. For example: ./gradlew runLiftAssoc -PreferenceItem=1 The IntelliJ ‘Run Configuration’ dialog will allow you to specify additional ‘script parameters’ to the Gradle invocation. ### Debugging If you run the Gradle tasks using IntelliJ's Gradle runner, you can run them under the debugger to debug your code. The Gradle file also configures LensKit to write log output to log files under the `build` directory. If you use the SLF4J logger (the `logger` field on the classes we provide) to emit debug messages, you can find them there when you run one of the recommender tasks such as `runDampedMean`. ## Submitting You will submit a compiled `jar` file containing your solution. To prepare your project for submission, run the Gradle `prepareSubmission` task: ./gradlew prepareSubmission This will create file `nonpers-submission.jar` under `build/distributions` that contains your final solution code in a format the grader will understand. Upload this `jar` file to the Coursera assignment grader. ## Grading Your grade for each part will be based on two components: - Outputting items in the correct order: 75% - Computing correct scores for items (within an error tolerance): 25% The parts themselves are weighted equally.