Using Magnum Opus with attribute-value data: a tutorial introduction

The following tutorial introduction steps through the process of analyzing the example attribute-value data distributed with Magnum Opus. This tutorial is designed to be used with Magnum Opus. Carry out the instructions with Magnum Opus as you read through the tutorial. Illustrations in the text will show how Magnum Opus should appear at various stages. Please note that this tutorial assumes that the default settings for Magnum Opus have not been changed. If they have, you may need to set the values back to their defaults in order to follow this tutorial.

This constructed example represents the type of data that might be collected by a supermarket about customers. It is contained in two files: tutorial.nam and tutorial.data. The first is a names file. The second is a data file. The names file describes the attributes recorded in the data file. The data file contains the values of those attributes for each case.

The names file, tutorial.nam, contains the following.

Profitability99: numeric 3
Profitability98: numeric 3
Spend99: numeric 3
Spend98: numeric 3
NoVisits99: numeric 3
NoVisits98: numeric 3
Dairy: numeric 3
Deli: numeric 3
Bakery: numeric 3
Grocery: numeric 3
SocioEconomicGroup: categorical
Promotion1: t, f
Promotion2: t, f

Each line of the names file describes one of the attributes in the data file. Each line starts with the name of the attribute. The name is terminated with a colon ':'. This is followed by a description of the attribute type.

Most of these attributes are numeric. These numeric attributes have been designated numeric 3, indicating that they should be divided into three sub-ranges, each of which contains approximately the same number of cases. The profitability attributes represent respectively the profit made from a customer in 1999 and 1998. The spend attributes represent the total amount spent by a customer in each year. The NoVisits attributes represent the numbers of store visits in each year. The Dairy, Deli, Bakery, and Grocery attributes record the customer's total spend in each of four significant departments. The remaining three attributes are categorical. The SocioEconomicGroup attribute records an assessment of the customer's socio-economic group. The keyword categorical tells Magnum Opus to use whatever values it finds in the corresponding column in the data file. The final two attributes record whether the customer participated in each of two store promotions. The values that are allowed are listed. This allows error checking. If any other value appears in the column for the attribute an error message will be displayed.

This tutorial's example analysis process proceeds as follows (click on a heading to jump directly to that section).

1. Run Magnum Opus.

2. Import the data.

3. Select options for search by leverage.

4. Run the search by leverage.

5. View the output of the search by leverage.

6. Dissection of a rule.

7. Run the search by lift.

8. Output from search by lift.

9. Run search for itemsets.

10. Conclusion.

Run Magnum Opus

Either double click the Magnum Opus icon or select Magnum Opus from the Start menu.

Initially, the main window will be empty, indicating that no data has yet been imported for analysis.

Import the data

To import data you normally click on the import data toolbar button. For the purposes of this tutorial, however, to directly access the tutorial data click on the import tutorial data button. Select the file tutorial.nam. The Import Data Wizard next requests identification of the data import format.

Magnum Opus supports three data import formats. In this example we use the names file - data file format. Click on the names file - data file option and then click the Next > button.

Magnum Opus now requests that you select the data file to accompany tutorial.nam.

Magnum Opus suggests a file name formed by substituting '.nam' for '.data' from the names file. As this is the name of the data file, click Next > to proceed. Next you are requested to identify the delimiter between fields in the data file.

Fields in the tutorial data file use the default delimiter, comma, that is already selected, so click Next > to proceed. Next you are requested to select the proportion of cases to import. If you wish to randomly sample cases from the data file, select the percentage of cases to sample.

In this tutorial we wish to use half the 1000 cases for rule discovery and hence hence change the setting to 50%. Type 50 into the edit box.

sample50.jpeg (16092 bytes)

Then click Next > to proceed.

The final screen of the Data Import Wizard allows you to select data for holdout evaluation.

holdout.jpeg (16190 bytes)

Holdout evaluation assesses the rules that are discovered against previously unseen holdout data in order to determine statistical significance. One option for obtaining holdout data is to use out-of-sample data: the data that is not included in the sample specified in the previous screen. Another is to import the holdout data from a second data file that must be in the same format as the main data file. For this tutorial we wish to use the out of sample data for holdout evaluation. This is the data that was not included in the 50% sample specified on the previous screen. This is the default option, so proceed by clicking Import Data to commence the data import process.

Magnum Opus will now display the following screen, indicating that the requested data has been imported and allowing the user to select search options.

Note the following elements of the initial screen. At the top is information about the data that have been imported. Below this appear the currently selected values for

whether to search for rules or itemsets
the search mode
the filter mode
the maximum number of rules or itemsets to be returned by the search
the maximum size of a rule or itemset
the minimum values of leverage, coverage, support, strength, and lift for a rule
whether m-estimates are to be used for strength and lift

Finally appear two list boxes containing lists of all of the available attribute values. Only those attribute values selected in these list boxes will be able to appear on the LHS and RHS of a rule, respectively. Only values selected in the first of the list boxes will appear in an itemset.

Select options for search by leverage

For the first search we will search for rules using search by leverage. For ease of demonstration we will limit the search to find only ten rules. Because we are in Search by Leverage mode, these will be the ten rules with the highest values for leverage that fulfil all other specified constraints.

To limit the search to ten rules, change the value in the edit box labelled Maximum no. of rules to 10.

For the sake of the example we will assume that we are interested in interactions between other attributes and profitability. Therefore, we want to limit the attribute values that may appear on the RHS of a rule to the three values for profitability.

To do this, click on the first item in the list box of values allowed on the RHS (bottom right of screen). Then either control-click (click on the item while holding down the control key) each of the next two items or shift-click (click on the item while holding down the shift key) the last of the items to be added to the selection. The first three items in the list box should now be highlighted and the screen should appear as follows.

Running the search by leverage

Once you have specified all of the settings for the search, click on the GO button to start the search. Magnum Opus writes the rules that it discovers to an output file. A dialog is displayed to allow you to select the name of this file and the folder in which to store it. Once you have selected the output file, the search commences.

As the Search by Leverage mode is in effect, Magnum Opus will perform search finding rules with the highest values on leverage within the other constraints that have been specified. While the search executes, progress will be displayed in the Status bar at the bottom-left-hand of the Magnum Opus window. Note that different stages of the complex searches performed by Magnum Opus may have varying degrees of difficulty. Hence, the rate of progress may vary dramatically from time to time. During a long search it is possible to inspect progress by clicking on the SNAPSHOT button.

Viewing the output

When the search is completed, the output viewer is launched to display the output file that was created. The output file lists

the name of the names file,
the name of the data file,
the number of cases loaded from the data file,
the number of attribute values in the data,
the time at which the search commenced, and
the settings used for the search.

These are followed by a list of rules that were found.

The output of the search by leverage is as follows. A detailed examination of one of these rules is provided in the next section.

Magnum Opus - The leader in discovery technology. Version 3.0 Copyright (c) 1999-2005 G. I. Webb & Associates Pty Ltd. Names file: Tutorial.nam Data file: Tutorial.data [50% sample] 500 cases / 500 holdout cases / 39 values Sat Jul 23 14:00:00 2005 Search for rules Search by leverage Filter out rules that are insignificant, critical value=0.01 Maximum number of attributes on LHS = 4 Maximum number of rules = 10 Minimum leverage = -1.0 Minimum leverage count = -2147483647 Minimum coverage = 0.0 Minimum coverage count = 1 Minimum support = 0.0 Minimum support count = 0 Minimum lift = 0.0 Minimum strength = 0.0 All values allowed on LHS Values allowed on RHS: Profitability99<438 438<=Profitability99<=931 Profitability99>931 Found 10 rules The following 8 rules passed holdout evaluation Spend99<2200 is associated with Profitability99<438 with strength = 0.892 coverage = 0.334: 167 cases satisfy the LHS support = 0.298: 149 cases satisfy both the LHS and the RHS lift 2.67: the strength is 2.67 times greater than the strength if there were no association leverage = 0.1864: the support is 0.1864 (93.2 cases) greater than if there were no association Spend99<2200 & Grocery<912 are associated with Profitability99<438 with strength = 0.931 coverage = 0.288: 144 cases satisfy the LHS support = 0.268: 134 cases satisfy both the LHS and the RHS lift 2.79: the strength is 2.79 times greater than the strength if there were no association leverage = 0.1718: the support is 0.1718 (85.9 cases) greater than if there were no association Spend99>4464 is associated with Profitability99>931 with strength = 0.838 coverage = 0.334: 167 cases satisfy the LHS support = 0.280: 140 cases satisfy both the LHS and the RHS lift 2.51: the strength is 2.51 times greater than the strength if there were no association leverage = 0.1684: the support is 0.1684 (84.2 cases) greater than if there were no association Grocery<912 is associated with Profitability99<438 with strength = 0.838 coverage = 0.334: 167 cases satisfy the LHS support = 0.280: 140 cases satisfy both the LHS and the RHS lift 2.51: the strength is 2.51 times greater than the strength if there were no association leverage = 0.1684: the support is 0.1684 (84.2 cases) greater than if there were no association Grocery>2126 is associated with Profitability99>931 with strength = 0.814 coverage = 0.334: 167 cases satisfy the LHS support = 0.272: 136 cases satisfy both the LHS and the RHS lift 2.44: the strength is 2.44 times greater than the strength if there were no association leverage = 0.1604: the support is 0.1604 (80.2 cases) greater than if there were no association NoVisits99<37 is associated with Profitability99<438 with strength = 0.805 coverage = 0.328: 164 cases satisfy the LHS support = 0.264: 132 cases satisfy both the LHS and the RHS lift 2.41: the strength is 2.41 times greater than the strength if there were no association leverage = 0.1544: the support is 0.1544 (77.2 cases) greater than if there were no association NoVisits99<37 & Grocery<912 are associated with Profitability99<438 with strength = 0.921 coverage = 0.254: 127 cases satisfy the LHS support = 0.234: 117 cases satisfy both the LHS and the RHS lift 2.76: the strength is 2.76 times greater than the strength if there were no association leverage = 0.1492: the support is 0.1492 (74.6 cases) greater than if there were no association Profitability98<368 & Spend99<2200 are associated with Profitability99<438 with strength = 0.943 coverage = 0.244: 122 cases satisfy the LHS support = 0.230: 115 cases satisfy both the LHS and the RHS lift 2.82: the strength is 2.82 times greater than the strength if there were no association leverage = 0.1485: the support is 0.1485 (74.3 cases) greater than if there were no association The following 2 rules failed holdout evaluation, adjusted critical value = 0.005000 Spend99>4464 & Grocery>2126 are associated with Profitability99>931 with strength = 0.915 coverage = 0.260: 130 cases satisfy the LHS support = 0.238: 119 cases satisfy both the LHS and the RHS lift 2.74: the strength is 2.74 times greater than the strength if there were no association leverage = 0.1512: the support is 0.1512 (75.6 cases) greater than if there were no association Holdout coverage = 115, holdout support = 106, holdout strength = 0.922 Fails significant improvement with respect to Spend99>4464, p = 0.034094 Spend99<2200 & Deli<237 are associated with Profitability99<438 with strength = 0.943 coverage = 0.244: 122 cases satisfy the LHS support = 0.230: 115 cases satisfy both the LHS and the RHS lift 2.82: the strength is 2.82 times greater than the strength if there were no association leverage = 0.1485: the support is 0.1485 (74.3 cases) greater than if there were no association Holdout coverage = 153, holdout support = 140, holdout strength = 0.915 Fails significant improvement with respect to Spend99<2200, p = 0.136847
Note that the last two rules failed holdout evaluation. This means that statistical evaluation on the holdout data failed to support the association observed on the exploratory data. The ability to identify such spurious rules is very important, as exploratory data mining suffers from an extremely high risk of false discoveries, that is, of finding rules or itemsets that appear good but only do so due to chance. See holdout evaluation for further details.

Dissection of a rule

The first rule in the example output log is the following.

Spend99<2200 is associated with Profitability99<438 with strength = 0.892 coverage = 0.334: 167 cases satisfy the LHS support = 0.298: 149 cases satisfy both the LHS and the RHS lift 2.67: the strength is 2.67 times greater than the strength if there were no association leverage = 0.1864: the support is 0.1864 (93.2 cases) greater than if there were no association

This rule indicates that cases for which Spend99 has a value less than 2200 are associated with cases for which Profitability99 has a value less than 438 more frequently than would be expected if there were no association between these values.

The following information about the rule is presented.

Strength=0.892 This indicates the proportion of those cases that satisfy the LHS that also satisfy the RHS. The values for coverage and support reveal that 167 cases satisfy the LHS and 149 of those also satisfy the RHS, so Strength is calculated as 149/167. Strength can be considered as an estimate of the probability of the RHS if the LHS occurs.

Coverage=0.334 This value indicates the proportion of all cases that satisfy the LHS of the rule (Spend99 < 2200). The value following the colon indicates the absolute number of cases that this represents. 167 cases satisfy the LHS out of the 500 cases in the exploratory data sample.

Support=0.298 This value indicates the proportion of all cases that satisfy both the LHS and the RHS of the rule (Spend99 < 2200 andProfitability99 < 438). The value following the colon indicates the absolute number of cases that this represents. 149 cases satisfy both the LHS and RHS out of the 500 cases in the exploratory data sample.

Lift=2.67 Lift is a measure of how much stronger than normal is the association between the LHS and RHS. Out of all the data, 167 cases satisfy the RHS (this is the same as the number satisfying the LHS, because each attribute was split on a value that created three equal sized partitions). Therefore the strength of association that would be expected if the LHS and RHS were independent of each other is 167/500. Dividing the actual strength by this value we obtain (149/167)/(167/500) = 2.67.

Leverage=0.1864 Leverage is a measure of the magnitude of the effect created by the association. This is the support for the rule in excess of that which would be expected if the LHS and RHS were independent of each other. If the two attribute values were independent of each other than the expected support would be the proportion exhibiting the LHS (0.334) times the proportion exhibiting the RHS (0.334) = 0.1116. The actual support is 0.2980. The difference between these values is 0.1864. The figure displayed in brackets is the total number of cases that this represents.

Run a search by lift

When you have finished viewing the output, return to the Search Options page by clicking on the Search Options tab in the top-left of the main window.

The first search found the ten rules with the highest value for leverage. By changing the search mode, it is possible to alter the type of interactions between variables that the system identifies. This next search will find the ten rules with the highest value for lift.

First select Search By Lift mode by clicking the Search Mode ComboBox and then selecting LIFT. The screen should now appears as follows.

lift.jpg (103958 bytes)

For the first search by lift we will use the same settings as were used for the search by leverage (find best ten rules only; allow only values for Profitability99 on RHS). As these are already selected we can proceed to starting the search by clicking on the GO button. Magnum Opus will ask you to select an output file. Specify a file name and click OK to proceed.

Output from search by lift

The output of the search by lift is as follows.

Magnum Opus - The leader in discovery technology. Version 3.0 Copyright (c) 1999-2005 G. I. Webb & Associates Pty Ltd. Names file: Tutorial.nam Data file: Tutorial.data [50% sample] 500 cases / 500 holdout cases / 39 values Sat Jul 23 14:27:48 2005 Search for rules Search by lift Filter out rules that are insignificant, critical value=0.01 Maximum number of attributes on LHS = 4 Maximum number of rules = 10 Minimum leverage = -1.0 Minimum leverage count = -2147483647 Minimum coverage = 0.0 Minimum coverage count = 1 Minimum support = 0.0 Minimum support count = 0 Minimum lift = 0.0 Minimum strength = 0.0 All values allowed on LHS Values allowed on RHS: Profitability99<438 438<=Profitability99<=931 Profitability99>931 Found 10 rules The following 3 rules passed holdout evaluation Profitability98>754 & Deli>575 & Grocery>2126 are associated with Profitability99>931 with strength = 1.000 coverage = 0.124: 62 cases satisfy the LHS support = 0.124: 62 cases satisfy both the LHS and the RHS lift 2.99: the strength is 2.99 times greater than the strength if there were no association leverage = 0.0826: the support is 0.0826 (41.3 cases) greater than if there were no association Bakery<236 & Grocery<912 are associated with Profitability99<438 with strength = 0.981 coverage = 0.210: 105 cases satisfy the LHS support = 0.206: 103 cases satisfy both the LHS and the RHS lift 2.94: the strength is 2.94 times greater than the strength if there were no association leverage = 0.1359: the support is 0.1359 (67.9 cases) greater than if there were no association Deli<237 & Grocery<912 are associated with Profitability99<438 with strength = 0.972 coverage = 0.214: 107 cases satisfy the LHS support = 0.208: 104 cases satisfy both the LHS and the RHS lift 2.91: the strength is 2.91 times greater than the strength if there were no association leverage = 0.1365: the support is 0.1365 (68.3 cases) greater than if there were no association The following 7 rules failed holdout evaluation, adjusted critical value = 0.005000 Spend99<2200 & NoVisits99<37 & Dairy<250 & Bakery<236 are associated with Profitability99<438 with strength = 1.000 coverage = 0.178: 89 cases satisfy the LHS support = 0.178: 89 cases satisfy both the LHS and the RHS lift 2.99: the strength is 2.99 times greater than the strength if there were no association leverage = 0.1185: the support is 0.1185 (59.3 cases) greater than if there were no association Holdout coverage = 122, holdout support = 118, holdout strength = 0.967 Fails significant improvement with respect to Spend99<2200 & NoVisits99<37 & Bakery<236, p = 0.275834 Profitability98<368 & Spend99<2200 & NoVisits99<37 & Dairy<250 are associated with Profitability99<438 with strength = 0.989 coverage = 0.180: 90 cases satisfy the LHS support = 0.178: 89 cases satisfy both the LHS and the RHS lift 2.96: the strength is 2.96 times greater than the strength if there were no association leverage = 0.1179: the support is 0.1179 (58.9 cases) greater than if there were no association Holdout coverage = 124, holdout support = 121, holdout strength = 0.976 Fails significant improvement with respect to Profitability98<368 & Spend99<2200 & NoVisits99<37, p = 0.370032 Spend99<2200 & Dairy<250 & Bakery<236 are associated with Profitability99<438 with strength = 0.979 coverage = 0.194: 97 cases satisfy the LHS support = 0.190: 95 cases satisfy both the LHS and the RHS lift 2.93: the strength is 2.93 times greater than the strength if there were no association leverage = 0.1252: the support is 0.1252 (62.6 cases) greater than if there were no association Holdout coverage = 142, holdout support = 134, holdout strength = 0.944 Fails significant improvement with respect to Spend99<2200 & Bakery<236, p = 0.176359 Profitability98>754 & Spend99>4464 & Grocery>2126 are associated with Profitability99>931 with strength = 0.979 coverage = 0.190: 95 cases satisfy the LHS support = 0.186: 93 cases satisfy both the LHS and the RHS lift 2.93: the strength is 2.93 times greater than the strength if there were no association leverage = 0.1225: the support is 0.1225 (61.3 cases) greater than if there were no association Holdout coverage = 76, holdout support = 76, holdout strength = 1.000 Fails significant improvement with respect to Profitability98>754 & Spend99>4464, p = 1.000000 NoVisits99<37 & Dairy<250 & Bakery<236 are associated with Profitability99<438 with strength = 0.978 coverage = 0.182: 91 cases satisfy the LHS support = 0.178: 89 cases satisfy both the LHS and the RHS lift 2.93: the strength is 2.93 times greater than the strength if there were no association leverage = 0.1172: the support is 0.1172 (58.6 cases) greater than if there were no association Holdout coverage = 127, holdout support = 118, holdout strength = 0.929 Fails significant improvement with respect to NoVisits99<37 & Bakery<236, p = 0.184825 Spend99>4464 & Bakery>588 & Grocery>2126 are associated with Profitability99>931 with strength = 0.975 coverage = 0.160: 80 cases satisfy the LHS support = 0.156: 78 cases satisfy both the LHS and the RHS lift 2.92: the strength is 2.92 times greater than the strength if there were no association leverage = 0.1026: the support is 0.1026 (51.3 cases) greater than if there were no association Holdout coverage = 75, holdout support = 68, holdout strength = 0.907 Fails significant improvement with respect to Spend99>4464 & Grocery>2126, p = 0.886174 Spend99<2200 & NoVisits99<37 & Deli<237 are associated with Profitability99<438 with strength = 0.972 coverage = 0.216: 108 cases satisfy the LHS support = 0.210: 105 cases satisfy both the LHS and the RHS lift 2.91: the strength is 2.91 times greater than the strength if there were no association leverage = 0.1379: the support is 0.1379 (68.9 cases) greater than if there were no association Holdout coverage = 131, holdout support = 124, holdout strength = 0.947 Fails significant improvement with respect to Spend99<2200 & NoVisits99<37, p = 0.053907

Several points are worth noting. First, most rules have more conditions in the LHS than the previous search by leverage. This is because search by lift seeks rules with high strength without regard for coverage. Adding multiple conditions can readily increase strength but does so by reducing coverage. Thus, search by lift tends to find more strongly predictive rules than search by leverage, but rules that are less widely applicable.

A second point to note is that seven of the ten rules discovered have failed holdout evaluation. Search by lift is also more vulnerable to rules failing holdout evaluation than is search by leverage, again because it tends to find rules with lower coverage. Such rules are more likely to appear unrealistically strong on sample data, an effect that is detected by holdout evaluation.

Magnum Opus can guard against this effect by using Filter-out Insignificant rules. This discards rules that fail a significance test with respect to the exploratory data. Such a filter cannot be statistically sound, as it would need to be adjusted to allow for the extremely large number of potential rules that the system considers. Rather, it provides a convenient means of discarding rules that have little prospect of passing a statistically sound test with respect to the holdout data. The ability to filter rules in this manner is not provided by most rule discovery systems. This filter can be made more stringent by decreasing the significance level, in which case more rules are likely to pass holdout evaluation. If the filter is not used when searching by lift for 10 rules on our example data, all rules discovered fail holdout evaluation.

Run search for itemsets

Rules are useful when you wish to be able to predict the value on the RHS. Sometimes, however, the primary interest is in finding attribute-values that interact with each other and dividing those values into a LHS and an RHS provides no additional useful information. Indeed, it can even be a hindrance, as the same interaction between a set of attribute-values can result in many rules. Itemsets overcome this limitation by identifying groups of attribute-values that occur together. To perform a search for itemsets, select ITEMSETS in the Search for box.

As fewer options are applicable when searching for itemsets, some are deactivated. Note also that Search by Lift is not available when searching for Itemsets, and so Search by Leverage is reinstated. Now press the GO button again. Magnum Opus will ask you to select an output file. Do so and click OK.

Magnum Opus will perform the search with the new settings. On completion, the following output is displayed.

Conclusion

This concludes the introductory tutorial. To exit Magnum Opus click the close button for the main window or select Exit from the File menu. To open another file, select the Import Data button from the toolbar. To perform another search, first select the desired settings and then click the GO button.

Last updated Sept 2005

home

products

download

evaluations

prices

purchase

contact us