A hybrid fuzzy-based Personalized Recommender System for Telecom Products/Services

5 experimental results

This dataset is collected by asking a new user to register as a member of the website and rate at least 15 movies that he/she has watched. The website will then recommend numerous movies to the new user based on his/her ratings. In our experiment, the MovieLens 100K dataset contains 100,000 rating records from 943 users for 1682 items, where the rating scale is from 1 to 5, and every user has rated at least 20 movies and all movies have been rated at least once. To validate the proposed fuzzy-based recommendation approach, the original rating scale 1 to 5 is fuzzified into NI, LI, I, MI, and SI respectively according to Table 2.

Two popular methods for measuring recommender systems are statistical accuracy metrics and decision support accuracy metrics. Statistical accuracy metrics compare the predicted ratings with the user-rated ratings. Commonly used statistical accuracy metrics methods include Mean Absolute Error (MAE), Root Mean Squared Error (RMSE) and Correlation. In the experiments, we select MAE as our evaluation method because it is easy to interpret directly and is more commonly used than others:

where is the predicted rating value of an item from a particular user i, is the actual rating of the item from this user (i=1,2,…N), ( ) is the distance measure between two fuzzy numbers which is calculated by formula (6), and is the total number of compared rating pairs.

To achieve accurate evaluation results, we randomly select the training dataset and testing dataset five times so that we have five training/testing dataset groups. For each group, the dataset is divided into one training dataset that contains 80% of all user ratings and one testing dataset that contains the remaining 20% ratings. For each group, the training dataset is used as the input data for the approach and all unrated ratings are to be predicted; MAE is then applied to compare all the records in the testing dataset with the predicted ratings.

The five training datasets are named u1base, u2base, u3base, u4base and u5base, while the five corresponding testing datasets are named u1test, u2test, u3test, u4test and u5test. To measure the effect of the number of neighbours on the accuracy of the approach, we calculate the MAE four times separately using 5, 10, 20 and 50 neighbours for each training/testing group. The testing result is illustrated in Figure 2.

As shown in Figure 2, only group 1 has a slightly higher average MAE than the rest of the groups, and the results of the other four groups are all very close to one another. Therefore, the performance of the approach is quite uniform across the MovieLens dataset. Figure 2 clearly shows that the average MAE falls, while the number of neighbours increases. As the number of neighbours increases from 5 to 10, there is a significant drop in average MAE which indicates a considerable increase in prediction accuracy. Therefore, considering both the accuracy and calculation efficiency, we decided that 10 neighbours are most suitable for our system.

Figure 2. Experiment results (MAE)

Missing rate %	Pearson CF	Model-Based CF	Content predictor	CBCF	JMCF	SMCF	FTCP-RS
94.64	0.8937	0.8921	0.9178	0.8705	0.8135	0.7785
94.96							0.784929
95.59	0.8858	0.8437	0.8669	0.8014	0.7836	0.7335