April 12, 2006

---

Discussed Paper on Web Mining

---

• also gave short demo of real-time query expansion

---

Introduction to Data Mining

---

• process of finding interesting and useful patterns in massive amounts of data
• where databases meets AI
• several techniques: algorithms from machine learning (e.g., decision trees), clustering, or association-rule mining
• diapers -> beer
  • the hello world of data mining
  • read `people who buy diapers, buy beer'
  • called an association rule
• what can we do with association rules?
  • all sorts of marketingy things
  • this is where the computer science ends and the business begins

---

Association Rules

---

• we are dealing with a database of transactions
  
  

  Tx#	Items
  1	beer, diapers, chips, salsa
  2	french fries, ketchup, beer, diapers
  3	coffee, sugar, diapers, milk, beer
  4	tea, honey, peaches, cream, beer, diapers
  .	...
  .	...



• how do we know that `diapers -> beers' is a `good' association rule?
• two metrics, where X and Y are sets of items
  • support (X -> Y):
    #Tx's w/ X & Y

    ---

    #total Tx's
  • confidence (X -> Y):
    #Tx's w/ X & Y

    ---

    #Tx's w/ X
• a transaction here is a set of items in a market basket; not to be confused with transaction from the study of concurrency control
• once an item set has support, it is called a frequent item set
• 1% is a reasonable support threshold; think about how many items a grocery store might stock
• support says nothing of the direction of the arrow; that's where confidence comes in
• confidence determines whether `people who buy diapers buy beer' or `people who buy beers buy diapers'
• 30-40% is a good confidence threshold
• we are interested in finding rules with good support and good confidence
  • why are rules with low support, but high confidence less interesting?
• leads to a two-step procedure
  1. find all frequent item sets
  2. break up item sets into rules and identify those with confidence
     • e.g., all possible rules from frequent item set {b,d} are `b -> d' and `b -> d'

---

Identifying Frequent Item Sets

---

• how many candidate item sets are there: O(2ⁱ), where i is the total number of items in the database
• let's start by simply thinking of candidate pairs
• consider the relation Baskets(no, item)
• the naive approach using the following SQL query is perilous

  SELECT I.item, J.item, COUNT(I.no)
  FROM Baskets I, Baskets J
  WHERE I.no = J.no AND I.item < J.item
  GROUP BY I.item, J.item
  HAVING COUNT(I.no) >= s;
  

• example (courtesy [DBCB] example 20.23, p. 1095):
  • 10,000 items, 1 million transactions, 20 items/tx on average
  • means the Baskets relations contains 20 million tuples
  • 20C2 = 195 times 1 million = 195 million pairs to group & count!
    • toooooo many

---

The A-Priori Algorithm

---

• AI is all about `search' and one of the main themes in solving AI problems is `pruning the search space'
• pruning criterion: if a set of items X does not have support, then no superset of X will ever have support
• create a lattice of candidate item sets and search it bottom-up
• so we will first identify the singleton item sets with high support and then only examine supersets of those
• query to find frequent items

  INSERT INTO F1
     SELECT *
     FROM Baskets
     WHERE item IN (
        SELECT item
        FROM Baskets
        GROUP BY item
        HAVING COUNT(*) >= s);
  

• back to example (courtesy [DBCB] example 20.23, pp. 1095-1096):
  • consider a support of 1% (i.e., s=0.01 or 10,000 tuples)
  • this means that there can be at most 2,000 frequent items (20,000,000/10,000)
    • this is a worst-case analysis; in reality, probably a lot less than 2,000 will have support, but no more than 2,000 can have support
    • if we assume the baskets containing these frequent items have 10 items on average, then
      • we only have to group and count 45 million pairs of items
        • 10C2 = 45 times 1 million
        • that is about 1/4 of the number of tuples required to group and count when seeking frequent pairs without using the pruning criterion
  • corollary: we would expect the a-priori algorithm to run in about 1/4 the time of the naive algorithm
• query to get pair candidates

  SELECT I.item, J.item
  FROM F1 I, F1 J
  WHERE I.no = J.no AND I.item < J.item
  

• leads to a simple and effective procedure:
  1. query F_k-1, F_k-1 to get C_k
  2. query C_k to get F_k

---

A Simple Example

---

• 4 transactions:
  
  

  Tx#	Items
  1	a, c, d
  2	b, c, e
  3	a, b, c, e
  4	b, e

  
  
  • AIS algorithm (1993) adds a `d'
    • these are the initials of the authors
    • first is Agrawal
  • a-priori algorithm does not need a `d'
• query to get F₁
  
  

  Item set	Frequency
  a	2
  b	3
  c	3
  e	3



• query F₁, F₁ to get C₂
  
  

  Candidate Item set
  a, b
  a, c
  a, e
  b, c
  b, e
  c, e



• query C₂ to get F₂
  
  

  Item set	Frequency
  a, c	2
  b, c	2
  b, e	3
  c, e	2



• query F₂, F₂ to get C₃
  
  

  Candidate Item set
  a, b, c
  a, b, e
  a, c, e
  b, c, e



• query C₃ to get F₃
  
  

  Item set	Frequency
  b, c, e	3

---

References

[DBCB]	H. Garcia-Molina, J. D. Ullman, and J. Widom. Database Systems: The Complete Book. Prentice Hall, 2002.
[AIS]	R. Agrawal, T. Imielinski, and A. Swami. Mining Association Rules Between Sets of Items in Large Databases. In Proceedings of the ACM International Conference on Management of Data (SIGMOD), pp. 207-216, 1993.