Desb, University of Utah

Download 17 Kb.
Size17 Kb.
ACCTG 6910, Spring 2003

DESB, University of Utah

Project Milestone 5 (April 3 – 17)
Question 1 (75%): Discover access patterns in web logs.
The supervisory council for University of Utah’s web portal has contacted the e.bis Research Lab to discover user access patterns from its web logs.
As a volunteer in the Lab, you have been asked to perform association rule and sequential pattern mining tasks on a small sample web log. It contains 4736 users, 10000 sessions and 11042 visit with the following attributes:
1-5 user id

7-11 session id

    1. URL id

Step 1: Download from the Project section in the class website the data set – weblog.txt and a text file – urlmapping.txt that describes mappings of URL codes in weblog.txt to URLs in UU’s web site.

Step 2: Use IBM Intelligent Miner to mine the data set for large item sets, association rules and large sequential patterns. Use 0.3 % for support level for association rule and sequential pattern mining and 50 % for confidence level for association rule mining. Mine the data set again using two different support levels for both association rule and sequential pattern mining.

Step 3: Report and analyze the results. Please identify 10 interesting association rules and 10 large sequential patterns respectively. Use the urlmapping.txt to help find the URLs that match URL ids in the rules/patterns Write up a short (one to two paragraphs) of analysis of these rules/patterns and any actions you recommend the supervisory council to consider.

1. Select five (instead of ten) different association rules with multiple items on the left-hand-side to interpret. If you don't have enough qualified rules, please adjust your support and/or confidence to find sufficient rules. Repeat steps 2 and 3 for each select rule.

2. Think of an explanation why the rule might exist (e.g., for A, B -> C, think of why would UU website visitors tend to access page C if they visit A and B.

3. Discuss your assessment of whether your explanation is or is not interesting.

4. Select five (instead of ten) large 3-sequences. If you don't have enough large 3-sequencies, please adjust the support level to find sufficient qualified sequences. For each select large sequence, repeat steps 2 and 3.

At 0.3% support and 50% confidence level, IM discovered 76 rules and 139 item-sets.

At 0.2% support and 50% confidence level, IM discovered 128 rules and 235 item-sets.

At 0.1% support and 50% confidence level, IM discovered 1072 rules and 781 item-sets.
Note: The objective of this milestone is for you to better understand what you may or may not expect from data mining and the efforts and domain knowledge required to interpret and leverage data mining results. Think about how much time it took you to work on the milestone 5 (A software like Link Selector can save a lot of web master’s time to interpret user access patterns for website redesign decisions.) Was it hard to interpret the results and make recommendations without some knowledge of website design/administration? When the data mining task is somewhat data-driven initially, you must find experts or acquire the relevant knowledge to analyze and leverage the patterns.
Here are some relevant knowledge and ways to interpret and analyze association rules and sequential patterns from a web log:
Web content management decisions include which links to be included in a page (especially the portal page).
An association rule, A, B -> C may suggest the following linkage because visits to A and B tend to go thru C:




A sequential pattern, <{A}, {B}, {C}> may suggest the following linkage because users tend to reference these urls throughout sessions:







Interpretations of interestingness of association rules and sequential patterns:

  1. Uninteresting if the patterns are induced by the design of a website only.

  2. Somewhat interesting if the patterns show common user interests that are well recognized and supported by the website design because they validate the effectiveness of the design.

  3. Interesting if the patterns show common user interests that are not well recognized and supported by the website design because some redesign actions may follow.

Association Rule Analysis:
1) [06155]+[06165]+[06128]==>[06153]
support = 0.3552% confidence = 100% lift = 266.68
corresponding URLs are
This rule exists because when users visited page /upap (utah physian assistant program) in utah website, the website would automatically load /upap/main.html, upap/top.html, and /upap/left.html and combine them into one html page as response. Uninteresting.

2) [00489]+[02049] ==> [02091]

support = 0.3158% confidence = 80% lift = 29.37

corresponding URLs are

[/academics/index.html]+[/graduate_school/admissions.html] ==>[/graduate_school/index.html]

The rule exists because if users visited academic program index page (/academics/index.html) and admissions information of graduate school (/graduate_school/admissions.html), they most probably used graduate school index page (/graduate_school/index.html) to navigate. Somewhat interesting.

3) [02087]+[02085] ==> [02091]
support = 0.2368% confidence = 85.71% lift = 31.47
corresponding URLs are

The rule exists because users clicked the link in the graduate school home page [/graduate_school/index.html] to browse the graduate handbook [/graduate_school/graduate_handbook/handbook.html], they then clicked the hyperlink in the handbook page to view the degrees available in UU. Somewhat interesting.

4) [00489]+[00680] ==> [00687]
support = 0.2171% confidence = 68.75% lift = 31.96
corresponding URLs are
The rule exists because users may visit the page of event of October 2002 [/calendar/oct2002.html] by clicking event calendar link [/calendar/index.html] in the academics program [/academics/index.html] home page. Somewhat interesting.

5) [04560]+[00566] ==> [03773]

support = 0.1184% confidence = 50% lift = 32.07


The rule exists because user may switch between the student index page [/students/index.html] and alumni visitor index page [/alumni_visitors/index.html] through the quick links index page [/quicklinks/index.html]. However, the quick links index page is removed now, and user will be redirected to the UU homepage if they still visit that page. The UU home page also contains the links to the student and alumni visitor homepages Somewhat interesting.

Under minimum support 0.1%, IM mined 773 sequential patterns. Five of them are selected to explain as follow.

1)<{[04560]}, {[00489]}, {[00472]}>
support = 0.313

corresponding URLs are

<{[/students/index.html]}, {[/academics/index.html]}, {[/a_z/index.html] }>

The pattern exists because users may check student information by student index page and browse academics information by academics index page, then using a-z index to quick locate specific information they may be interested in. Interesting.

2) <{[04560]}, {[04560]}, {[05955]}>

support = 0.281%

corresponding URLs are
<{[/students/index.html]}, {[[/students/index.html]}, {[/unews/releases/02/oct/cauldron.html]}>

The pattern exists because users may notice and visit the hot news link in the student index page after they visit the student index page twice. Interesting.

3) <{[00489]}, {[00489]}, {[00489]}>

support = 8.13%

corresponding URLs are
<{[/academics/index.html]}, {[/academics/index.html]}, {[/academics/index.html]}>

The pattern exists because some users used to use academics index page to locate academics information in the different sessions.

3) <{[00472]}, {[00489]}, {[04560]}>

support =0.219%

corresponding URLs are
<{[/a_z/index.html ]}, {[/academics/index.html]}, {[/students/index.html]}>

The pattern exists because some users may find it not easy to find the information they required through a-z index page. Therefore, they may choose to use academic and student index page to locate the information of their interest in the following sessions. Interesting.

4) <{[00472]}, {[00472]}, {[00680] [00687]}>

support =0.188%

corresponding URLs are
<{[[/a_z/index.html]}, {[[/a_z/index.html]},

{[/calendar/index.html] [/calendar/oct2002.html]}>

The pattern exists because users may like to review the news of UU after they browse the specific information of their interest through a-z index page. We can also derive that the web logs may be collected in October, 2002 since users visit the news index page of that time. Interesting.
5) <{[00489]}, {[01681]}, {[00489]}>
support =0.188%

corresponding URLs are

<{[/academics/index.html]}, {[/employment/index.html]}, {[/academics/index.html]}>

The pattern exists because students may want to find a job in UU to support their academics learning. Interesting.

Question 2 (25 %): If the data file includes referrer and visit duration information for each visit, please discuss how you might use clustering to help identify clusters in the data file.
Note: Clustering uses more than one attributes and doesn’t specify what the clusters (e.g., clusters of urls with average visit duration longer than 1 minute) should be.
The following clustering could produce interesting results of pages, users or sessions with similar patterns. To further analyze how they are similar, additional data mining such as association rule and sequential pattern, more granular clustering and classification may be applied.
Cluster by url with attributes - average visit duration, top 3 referrers, average # of visitors/day, # of in-links, and # of out-links

Cluster by visitor with attributes – location (e.g., zip code, coordinates, or wireless cell), frequent visit time of day (e.g., early am, mid am, late am, early pm, mid pm and late pm), average session duration, average page visit duration, top n large-item sets and top n sequential patterns.

Cluster by session with attributes – location of visitor, average session duration, time of day, average number of links, and top n large-item sets
Download 17 Kb.

Share with your friends:

The database is protected by copyright © 2020
send message

    Main page