INTUITIVE STRATEGIES AND PRECONCEPTIONS ABOUT ASSOCIATION IN CONTINGENCY TABLES

Carmen Batanero, Antonio Estepa, Juan D. Godino, & David R. Green

Journal for Research in Mathematics Education, 27(2), 151-169

ABSTRACT

The aim of this research was to identify students' preconceptions concerning statistical association in contingency tables. An experimental study was carried out with 213 pre-university students, and it was based on students' responses to a written questionnaire including 2x2, 2x3 and 3x3 contingency tables. In this article, the students' judgments of association and solution strategies are compared with finding of previous psychological research on 2x2 contingency tables. We also present an original classification of students' strategies, from a mathematical point of view. Correspondence analysis is used to show the effect of item task variables on students' strategies. Finally, we include a qualitative analysis of the strategies of 51 students, which has served to characterize three misconceptions concerning statistical association.

The concepts and procedures involved in the study of correlation and regression are intended to determine statistical dependence relationships between numerical variables. The extension of the idea of correlation to qualitative variables has originated the general concept of association, the teaching of which is a fundamental topic in the statistics curricula of many different university degrees. In secondary education and pre-university courses, three different topics are included in the teaching of association: the analysis of contingency tables, the determination of correlation between quantitative variables, and the comparison of a numerical variable in two or more samples.

The concept of association or statistical dependence has great relevance to mathematics education, because it extends functional dependence and it is fundamental for many statistical methods, for it allows us to model numerous phenomena in different sciences (e.g., biology, economics, medicine, education). This topic has significant connections with research on functional thinking and other areas of mathematics education, such as probability and proportional reasoning. The main goal in many of these applications is to find causal explanations, that permit us to understand our environment. However, the association does not necessarily imply a causal relationship. Sometimes, due to the influence of concurrent factors, it is possible to find a high coefficient of association among variables when there is no causal link (spurious correlation).

Besides this epistemological difficulty, psychological research has shown that judging association is not an intuitive ability. Adults sometimes base their judgment on their previous beliefs about the type of association that ought to exist between the variables that are to be studied rather than on the empirical contingencies presented in the data. The existence of these preconceptions about the nature of the empirical relationship in problematic situations presents another difficulty for the teaching of association. Despite these epistemological and psychological issues, no previous research on this topic has been carried out in mathematics education, and most psychological research has only concentrated on 2x2 contingency tables.

The experimental study reported here, which was carried out with a sample of 213 pre-university students, sought to identify students' preconceptions concerning association in contingency tables. Their written responses to a questionnaire concerning 2x2, 2x3 and 3x3 contingency tables are analyzed from different points of view. First, we discuss the type of association perceived by the students for the different items, relating their association judgments to the task variables. Second, we analyze the students' solution strategies, comparing our results with previous research into 2x2 contingency tables. We present an original classification of these strategies, from a mathematical point of view, in which we identify concepts and theorems in action as described by Vergnaud (1982). Finally, we discuss the qualitative analysis of an additional sample of 51students' responses and characterize different misconceptions concerning statistical association.

As Confrey (1990) has pointed out, the relevance of research on students' conceptions lies in the fact that sometimes these conceptions differ in fundamental ways from the scientific concepts that we try to teach. Furthermore, students' conceptions are resistant to change in spite of instruction. Although many students in our study demonstrated correct or partially correct judgments and solutions strategies, the misconceptions and incorrect strategies presented in this article indicate a gap between the meaning of association that we try to teach, and the subjective meaning that students may attribute to this concept (Godino & Batanero, 1994). In the following sections all of these aspects will be described, starting with a summary of the psychological research on which our study was based.

PSYCHOLOGICAL RESEARCH ON CONTINGENCY TABLES

Piaget and Inhelder's Study

The study of reasoning about statistical association started with Inhelder and Piaget (1955), who considered the understanding of association as the final step in developing the idea of probability. According to them, the evolutionary developments of the concepts of association and probability are related, and understanding association has as prerequisites the comprehension of proportionality, probability, and combinatorics. Consequently, they only studied reasoning about association with children in their formal operation stage (IIIa and IIIb). They proposed to the subjects the problem of the association between eyes and hair color, and used a set of colored cards with drawings of faces as an experimental device (fair and brown hair; blue and dark eyes). In order to explain their results, Inhelder and Piaget classified the four possibilities of combining eyes and hair color according to the layout presented in Table 1, in which a, b, c and d represent the absolute frequencies in four cells (fair hair, blue eyes; fair hair, dark eyes; brown hair, blue eyes; brown hair, dark eyes). This is the simplest form of a contingency table or cross-tabulation involving two variables that is used to summarize the frequencies in a population or sample.

Table1. Typical Format for a 2x2 Contingency Table

 B Not B Total A a b a+b Not A c d cd Total a+c b+d a+b+c+d

Piaget and Inhelder found that at stage IIIa some adolescents only analyze the favorable positive cases in the association (cell [a] in Table 1). In other cases they only compare the cells two by two. When they admit that the cases in cell [d] (absence-absence) are also related to the existence of association, they do not understand that cells [a] and [d] have the same meaning concerning the association, and that they should compare [a] with [b] or [c] with [d] instead. This fact is explained by observing that, although stage IIIa subjects can compute single probabilities, understanding association requires considering quantities (a+d) as favorable to the association and (b+c) as opposed to it and also that it is necessary to consider the relation:

where R represents the difference between cases confirming the association (a+d) and cases opposed to it (b+c) compared to all the possibilities. According to Piaget and Inhelder, recognition of this fact only happens at 15 years of age (stage IIIb).

Subsequent Research into Judging Association in 2x2 Tables

Following Piaget and Inhelder, several psychologists have studied the judgment of association in 2x2 contingency tables in adults, using various kinds of tasks and, as a consequence, it has been noted that subjects perform poorly establishing judgments about association. For example, Smedslund (1963) found that some adult students based their judgment exclusively on cell [a] or by comparing [a] with [b].

The difficulty of this type of task is shown by the fact that, as Jenkins and Ward (1965) pointed out, even the strategy of comparing the diagonals in the table -considered correct by Piaget and Inhelder- is only valid in tables that have equal marginal frequencies for the independent variable. Nevertheless, in the research of Allan and Jenkins (1983) and Shaklee and Tucker (1980) this strategy was widely used by adults. For the general case, Jenkins and Ward proposed as the correct strategy the comparison of the difference between the two conditional probabilities, P(B|A) and P(B|NotA), i.e.,

Pérez Echevarría (1990) summarized the strategies that have been identified in psychological research. In her opinion these strategies can be divided into 5 levels of performance:

Level 1: using only one cell in the table; usually, cell [a].

Level 2: comparing [a] with [b] or [a] with [c].

Level 3: comparing [a] with [b] and [a] with [c].

Level 4: using all four cells in the table; but they use additive comparisons.

Level 5: using all four cells in the table; employing multiplicative comparisons.

Chapman and Chapman (1969) reported another finding having implications for this study. They found that subjects often hold expectations and beliefs about the relationship between the variables that suggest empirical contingencies. This phenomenon has been described as "illusory correlation" because people maintain their beliefs in spite of evidence of the independence between the variables. The subjects' life experiences and cultural environment contribute to the formation of these personal theories, which they use to interpret data and facts around them. Jennings, Amabile and Ross (1982), Wright and Murphy (1984), and Alloy and Tabachnik (1984), among others, have studied the effect that previous theories about the context of a problem have on judging association.

The general conclusion is that when there is agreement between data and previous expectations people increase their confidence in the attribution of co-variation. On the other hand, when data do not coincide with these expectations, there is a cognitive conflict and the accuracy of the perception of co-variation depends on the relative strength of the two sources of information. Finally, as Scholz (1991) described, later studies have shown that, for the same association problem structure, people change their strategies depending on the characteristics of the taks.

All of these results, as well as the importance of this concept in statistics, point to the need to carry out research on students' understanding of statistical association in order to plan appropriate instruction. In the following section, we describe our study of students' preconceptions concerning statistical association in contingency tables.

METHOD

The study we describe here is part of a larger research project concerning the effect of a teaching experiment, based on problem solving and the use of computers, on the learning of statistical association (Estepa, 1994). In this section we briefly review the sample, questionnaire, and data analysis procedure used in this study.

Sample

The sample consisted of 213 students in their last year of secondary school (17-18 year-old students) attending three different high schools in the city of Jaén (eight different student groups were used). It is at this level that the topic of association is introduced in the Spanish curriculum.The questionnaire was given to the students before the teaching of association began. So this study may be considered as research into students' preconceptions concerning statistical association (Confrey, 1990).

About half of the students (113) were males and half where females (110); 124 students had followed a scientifically-oriented curriculum and the rest a humanities-oriented curriculum. Although both groups of students had studied mathematics throughout their previous studies, in the first group a stronger emphasis was given to the subject and they had more opportunities to apply mathematics in other scientific subjects, such as physics or chemistry. Concerning statistics, both groups were taught frequency distribution, graphical representation, measurements of location, and the spread of univariate statistical variables. They also had been introduced to probability in the first year of secondary school (when they were 14-15 years old). This study has a quasi-experimental character, because of the non-random nature of the samples of students and problems.

Questionnaire

The questionnaire included five items concerning contingency tables and is presented as an appendix. These five items involved three task variables.

V1: Type of table: Because most research done in psychology has used 2x2 tables, we included three items of this kind. We also included a 2x3 table and a 3x3 table, to assess the similarities in difficulty level and solution strategies between 2x2 tables and higher dimension tables. Another reason to consider 2x3 and 3x3 tables is that the study of association in the curriculum is not restricted to 2x2 tables.

V2: Sign of the association: All three possible cases -direct, inverse and independence- were used in 2x2 tables. In the 2x3 table the association was between ordinal variables. The sign of association was not applicable to the 3x3 table used because it involved nominal variables.

V3: Relationship between context and prior belief: The association suggested by the context of the problem and the empirical association presented in the table may coincide (theory agrees with data) or not (theory goes against data). The precise values assigned to each of these variables in the different items are shown in Table 2.

Table 2. Values of task variables in the different items

 Item 1(Smoking) 2 (Diet) 3 (Allergy) 4 (Exam) 5 (Laterality) V1 2x2 2x2 2x2 2x3 3x3 V2 Independence Inverse Direct Direct Independence V3 Theory contradicted Unfamiliar context Unfamiliar context Theory supported Unfamiliar context

Note: V1: Type of Table; V2: Sign of the association; V3: Relationship between context and prior belief

Data analysis

In each item we analyzed the type of association perceived by the students (direct association, inverse association, or independence). In addition, the students' written responses to the questionnaire were categorized and, after several successive revisions, two different schemes were developed for classifying students' solution strategies. The first scheme was based on previous psychological research, especially Perez Echevarria's results (1990). For the second scheme we used Vergnaud's (1982) notion of theorem in action.

A pilot study with an additional sample of 51 students was used to check the reliability and the coding system for the students' answers. Since factor analysis of students' answers to the complete questionnaire had a multidimensional structure, a generalizability study (Brennan, 1983) was done, instead of computing an internal consistency index. For the complete questionnaire we obtained a generalizability index, G = 0.86, as a measure of the possibility of extending our conclusions to the hypothetical item population and another index, G = 0.94, of generalizability to the subjects' population. This pilot sample was also used to perform a qualitative study of the interrelation between strategy and the type of association perceived by the students, in order to characterize students' misconceptions concerning statistical association. The results of all of these analyses are presented in the following sections.

RESULTS AND DISCUSSION

Analysis of Association Judgments

Although many students provided a correct association judgment for the different items, three different factors influenced task difficulty: table size, the students' previous theories about the context of the problem, and the lack of perception of inverse association. In Table 3, we present the frequencies and percentages of the types of association perceived by the students in Items 1 through 4. On Item 5 (laterality) , in which the sign of association was not applicable, 129 students (60.6% of the total number of students) gave the correct answer (independence), 23 students (10.8%) considered the existence of association, and 61 students (28.6%) provided no answer.

Table 3. Frequency (and percent of) type of association perceived by the students for Items 1 through 4

 Type of association Item Independence Direct Inverse No answer 1 84(38.4)* 118(55.4) 1(0.5) 10(4.7) 2 65(30.5) 25(11.7) 108(50.7)* 15(7.0) 3 9(4.2) 194(91.1)* 3(1.4) 7(3.3) 4 219(8.9) 186(87.3)* 8(3.8)

The difficulty level was very low for items 3 (allergy) and 4 (exam), which involve direct association. Moreover, the nature of these items was such that either the strength of the association was high and students had no previous theory (items 3) or students' expectations about the type of association coincided with the contingency presented in the data (item 4).

Item 5 (laterality) was of moderate difficulty, although it is a 3x3 table and involves independence; despite the fact that a high number of students provided no answer. Compared with Item 1 (smoking), which is a 2x2 table and refers to independence, the difference in difficulty is noticeable. For item 1 most students considered the association as direct, because this is the expected type of association suggested by the context of the problem (smoking and cough).

As Crocker (1981) pointed out, judging the relationship between different events and evaluating environmental contingencies is not only a mathematical problem, it is a task in which our previous knowledge about the phenomena is used and so the context of the information exercises a notable influence. Furthermore, the case of independence in a 2x2 contingency table corresponds to comparing two probabilities in which the numbers of favorable and unfavorable cases are proportional. This type of task is, according to Piaget and Inhelder (1951), the most difficult one when comparing two probabilities because it requires level IIIb of probabilistic reasoning in order to be completed successfully.

Finally, on item 2 (drug), which was of moderate difficulty, a significant number of students interpreted the fact that taking the drug implies no digestive troubles as independence, rather than as an inverse association. This result confirms Jenkins and Ward's thesis (1965), that, in causal contexts, causes and effects are not perceived symmetrically by subjects. When there is a large number of cases in which cause is present but effect is not, subjects will conclude that there is no relationship between cause and effect.

We suggest that the students who consider as independence the fact that taking the drug implies no digestive troubles had an unidirectional conception of statistical association. They considered only the association between two statistical variables when the sign of association is direct and judged the inverse association as independence. This interpretation of the students' answers was reinforced by the qualitative analysis of the students' strategies. (This analysis is discussed in the next section.)

This unidirectional conception can also be identified in the historical development of the association concept. As Pearson (1920) pointed out, even Galton was not conscious of the possibility of an inverse association when he defined correlation in his speech to the Royal Society on December 5th, 1888. "Two variable organs are said to be correlated when the variation of the one is accompanied on the average by more or less variation of the other, and in the same direction" (Galton, quoted in Pearson, 1920, p. 199). Pearson also reported that the consideration of the meaning of a negative correlation coefficient was due to Weldon, who published an article related to this subject in 1892.

Classification of Students' Strategies

The correct judgment of association by the students does not allow one to evaluate their preconceptions concerning statistical association because, as many authors have noted, it is possible to get a correct judgment with an incorrect strategy. From the point of view of mathematics education, the ability to provide both a correct judgment and a correct strategy are needed for an adequate understanding of this concept. To evaluate this second aspect, we classified the students' strategies in two different ways. The aim of this section is to present the results of these two classifications.

Levels of Elaboration of Students' Strategies

First, students' strategies were classified according to the levels of difficulty proposed by Pérez Echeverría (1990) for 2x2 contingency tables, which we extended to general tables. The results of this classification are presented in Table 4. In the 2x2 tables we obtained a higher percentage of students in levels 1, 4 and 5 than Perez Echevarría, and a smaller percentage in levels 2 and 3. (This author obtained on average 3.7% in level 1, 30.5% in level 2, 15.7% in level 3, 37.2% in level 4, and 12% in level 5).

Table 4. Frequency (and percent of) students' strategies in the different items classified by level of elaboration

 Item Level of elaboration 1 (2x2) 2 (2x2) 3(2x2) 4(2x3) 5(3x3) 1 29(13.6) 26(13.6) 22(10.3) 6(2.8) 0 2 62(29.1) 40(18.8) 50(23.5) 100(46.9) 28(13.2) 3 1(0.5) 0 9 2(0.9) 22(10.4) 4 15(7.0) 74(34.7) 70(32.3) 66(31.0) 58(27.2) 5 45(21.1) 51(23.9) 45(21.1) 12(5.6) 14(8.9) Other 56(26.3) 18(8.5) 23(10.8) 21(9.9) 37(17.4) No argument 5(2.3) 4(1.9) 3(1.4) 6(2.8) 49(23.0)

These differences can be explained by the fact that, although we chose the same context for the problems as Pérez Echeverría used in her research for items 1, 2, and 3, we varied the format of the items and the frequencies of the cells for items 1 and 3. We must also consider that our students (pre-university students who had not previously studied this topic) were younger than those of Pérez Echevarría (students of psychology in their last year at university, who had taken a previous course in statistics during their course of study).

In general, we found good intuitive ability for judging association in 2x2 contingency tables in a significant percentage of our students. We also note the large number of students who were incapable of providing an argument for the 3x3 contingency table (33 cases out of 213) or who were unable to make a judgment of association in this table (49 additional cases). This confirms the greater difficulty of the task when the dimension of the table are increased. Aso, people do not maintain the same level of performance for the different types of tasks.

Classification of Students' Strategies from a Mathematical Point of View

In studying the strategy levels, we noticed that some students proved their correct intuitive conceptions as they solved the problem by comparing the different probabilities (level 5 strategies). But the concept of association is not simple and some students who used levels 1 to 4 may have had correct intuitive conceptions concerning some properties of association mixed with other incorrect conceptions. This led them to choose an incorrect or partially correct strategy. All these conceptions need to be identified in order to plan adequate instruction.

In order to identify which of the properties linked to the concept of association were intuitively used correctly or incorrectly by our students, we developed a classification of students' strategies according to the mathematical concepts involved and theorems in action implied in the different procedures used and the type of errors. Such a classification is, in Vegnaud's (1982) opinion, essential for didactical analysis. In the following paragraphs we describe this classification.

Correct strategies. Three completely correct strategies were found.

S1: Comparison of all conditional relative frequency distributions h(Bj|Ai) of every value Bj for two or more different values of Ai . For example, in item 1 (smoking), the proportion of bronchial disease in the smokers was compared to the proportion of bronchial disease in nonsmokers.

Students who employed this strategy were implicitly using the following theorem, T1: "The dependence of a variable B on another variable A implies the variation of the relative conditional frequencies h(Bj|Ai) when Ai varies." This involves the comparison of rows. The strategy of comparing columns is also included here. Some students implicitly used the following theorem, T2: "Statistical dependence of a variable B on another variable A implies also some statistical dependence of A on B."

S2: Comparison of only one conditional relative frequency h(Bj|Ai), for a fixed value j, for each possible value Ai with the marginal frequency h.j. For example, in item 1the proportion of bronchial disease in smokers was compared to the proportion of bronchial disease in the total sample.

Here, students implicitly used the theorem, T3: "Independence means the invariance of the distribution of B when conditioned with a value Ai " (rows and columns could be interchanged).

S3: Odds comparison. The students neither quantified nor compared the conditional frequencies, but they compared the frequencies of cases in favor of B and cases against B (or compared the ratio of these frequencies) for each value of A. For example, in item1students compared the ratio between people with bronchial disease and those without bronchial disease who smoke (90/60) with the same ratio for nonsmokers (60/40). In this strategy the theorem, T4, is implicit: "There is a univocal correspondence between the probability of an event and its odds ratio." It is this idea that serves as a base for the definition of the odds ratio as a measure of association.

Partially correct strategies. Eight strategies were identified that were considered partially correct.

S4: Comparison of two or more absolute conditional frequency distributions with the corresponding marginal absolute frequency. It is similar to strategy S1, in which students implicitly used a version of theorem T1, without using relative frequencies explicitly. Instead, they performed qualitative or additive comparisons of the absolute frequencies. Consequently, they did not properly quantify the difference between the probability of the values of one variable as a function of the other one.

S5: Comparison of only one absolute conditional frequency in each conditional distribution with the corresponding marginal frequency. This is similar to S4 but the students only used one of the absolute frequencies in each conditional distribution. This also involves the implicit theorem T5: "The variation of the conditional relative frequency of one of the values of A, when B varies, is sufficient to prove dependence". Nevertheless, these students incorrectly used additive comparisons to quantify the necessary probabilities.

S6: Comparison of the sum of the frequencies in the diagonals. For the 2x2 table, the main strategies included in this level were the strategy [(a+d) -(b+c)] and its mathematical equivalent strategy [(a-c)-(b-d)], which were found in Inhelder and Piaget (1955), Shaklee and Tucker (1980), Arkes and Harkness (1983), and Allan and Jenkins (1983). This strategy is valid for a table with equal marginal frequencies, but not for the general case. The students implicitly used the theorem T6: "Cells [a] and [d] in the 2x2 table contain the frequencies of cases favorable to a direct association; cells [b] and [c] represent the cases favorable to an inverse association." Some students used a similar strategy in 2x3 or 3x3 tables because they previously reduced these types of table to a 2x2 table, either using only two rows and two columns of the table or adding the frequencies in adjacent rows or columns.

Incorrect strategies.

S7: Use of only one cell, often the cell whose frequency is maximum. This strategy is related to the representativeness heuristic (Kahneman, Slovic & Tversky, 1982), because the students did not use all the relevant information to solve the problem, only the most salient. Students failed to realize the existence of other cells relevant to the association. For 2x2 tables the most common case is using only cell [a], a strategy which has also been described in previous research work, such as Inhelder and Piaget (1955), Smedslund (1963), Beyth-Marom (1982), and Jenkins and Ward (1965).

S8: Use of only one conditional distribution. These students did not understand that they had a problem for which a comparison of two probabilities was needed. They based their judgment on the frequencies in one row or one column of the table.

S9: Comparison of absolute frequencies in some cells with the total number of cases. The students did not realize the need to compare the probability of a value of A with the different values of B ( i.e., the need to use one of the theorems described). Instead, they compared the probability of some pairs (Ai, Bj) in a non-systematic way.

S10: Use of marginal frequencies. Some students considered the problem impossible to solve because of the difference in marginal frequencies in different rows or columns in the table.

S11: Other procedures. In this category we included: students who expressed their previous theories about the nature of the association between the variables, gave incomplete arguments, or tried to solve a system of equations.

In general, all the students who answered the question gave an argument that served to identify the strategy used for solving the problem. In the final category we grouped the students who did not provide a judgment for the association and those whose arguments were not sufficiently clear to be interpreted.

The frequencies of use of the different strategies are given in Table 5.

 Item Strategy 1(Smoking) 2 (Diet) 3 (Allergy) 4 (Exam) 5 (Laterality) S1 8 (3.8) 16 (7.5) 2 (0.9) 20 (9.4) 18 (8.4) S2 33 (15.5) 27 (12.6) 35 (16.4) 21 (9.9) 0 S3 13 (6.1) 39 (18.3) 34 (16.0) 62 (29.9) 5(2.3) S4 3 (1.4) 19 (8.9) 21 (8.9) 15 (7.0) 76 (35.6) S5 2 (0.9) 19 (8.9) 17 (8.0) 0 0 S6 5 (2.3) 11 (5.2) 18 (8.4) 7 (3.3) 4 (1.9) S7 29 (13.6) 26 (12.2) 22(10.3) 6 (2.8) 0 S8 56 (26.3) 28 (13.2) 39 (14.1) 47 (22.1) 24 (11.3) S9 1 (0.5) 6 (2.8) 7 (3.3) 2 (0.9) 0 S10 8 (3.8) 0 1 (0.5) 6 (2.8) 0 S11 13 (6.1) 2 (0.9) 3 (1.4) 7 (3.3) 4(1.9) No answer 42 (19.7) 20 (9.8) 23(10.8) 20 (9.4) 82 (38.5)

S1: Comparison of all conditional relative frequency distributions

S2: Comparison of only one conditional relative frequency

S3: Odds comparison

S4: Comparison of two or more absolute conditional frequency distributions with the corresponding marginal absolute frequency

S6: Comparison of the sum of the frequencies in the diagonals

S7: Use of only one cell

S8: Use of only one conditional distribution

S9: Comparison of absolute frequencies in some cells with the total number of cases

S10: Use of marginal frequencies

S11: Other procedures

The most popular strategy in our study was using only one conditional distribution (S8). This strategy, as well as that of using only one cell (S7), correspond to Piagetian level IIIa. A significant percentage of students were unaware of the necessity to compare the different samples in the problem in order to judge an association and they had not reached Piagetian level IIIb in their understanding of association.

There was a significant number of students who chose a correct or a partially correct strategy (strategies 1 to 5). In these cases, students implicitly used some of the theorems T1 to T7, which we have identified. In particular, some students not only compared conditional relative frequency distributions (S1), but also compared these distributions with the marginal distribution (S2); similarly, other students used comparison of odds in order to solve the problem (S3).

The classification of strategies we have made constitutes a refinement of the five levels of Pérez Echeverría (1990) and emphasizes the mathematical relations used by the students. The following equivalence can be established between the two classifications:

Level 1 and our strategy S7 are equivalent. Level 2 includes strategy S8 and those cases of strategy S9 in which only two or three cells are used. (We prefer to distinguish the use of a complete conditional distribution as a separate category in level 2.) Level 3 includes the other cases of strategy S9. Level 4 includes strategies S4, S5, and S6. Level 5 includes strategies S1, S2, and S3. Strategies S10 and S11, which are incorrect strategies, cannot be included in Pérez Echeverría's levels.

Correspondence analysis between Strategies and Items

Looking at Table 5, we can see that the strategies used by students vary for the different items and we suspect this is due to the different values of the tasks variables for each item. To describe this dependence, a correspondence analysis (Greenacre, 1984) of Table 5 was performed. This is a multivariate technique that provides a method for comparing row and column proportions in a two-way or multi-way table by decomposing the Chi-square measure of association of the table into components in a manner similar to principal components analysis for continuous data.

Task variables of the items were used as supplementary variables, with the aim of improving the interpretation. The Chi-square statistic value was highly significant (c 2 = 481.284, d.f. = 44, p=0.0009) and the representational quality of rows and columns was higher than 0.707.

The first factor, which explained 59.6% of inertia, differentiated the strategies used in the 3x3 table from the strategies used in the 2x3 and 2x2 tables. This was due to the fact that students intuitively solved the 2x2 tasks by using non-normative strategies and they were even capable of finding a correct strategy by themselves (in about 25 to 38.4% of the cases) for this type of task. Nevertheless, this percent went down to 10.7 percent for the 3x3 table, in which these subjects were incapable of generalizing this correct strategy, although they had succeeded in extending the strategy to the 2x3 table. As indicated by Nisbett and Ross (1980), subjects do not usually employ a normative strategy in judging association without being given specific instruction because these strategies are highly sophisticated.

With regards to the task variables, on the first factor independence was opposed to association, especially to the inverse association. So, for independence problems, a more elaborate strategy was needed to provide a correct judgment. This case requires the comparison of proportional absolute conditional frequency distributions, which makes it more difficult to compare probabilities (Piaget & Inhelder, 1951).

The second factor accounted for 21percent of the inertia. This second factor divided the incorrect and partially correct strategies that were used for the 2x2 tables into two groups. The first group contained the partially correct strategies that were preferred for the dependency cases (item 2, and item 3): comparing absolute frequencies with the total number of cases (S4), comparing one absolute frequency in each conditional distribution with the marginal frequency (S5), and using the frequencies in the table diagonals (S6). The second group contained the incorrect strategies: using only one conditional distribution (S8), using marginal frequencies (S10), and other strategies (S11). These incorrect strategies were preferred for the independence case (item 3).

Concerning the task variables, the most notable influence is that of having theories against the data as compared to theories in favor or no theory. Thus, it appears that some incorrect strategies were used as a means to justify previous theories about association when there was no agreement between students' preconceptions about the nature of the relationships that "ought" to exist and the contingency presented in the data. That is, when the students found an empirical association that contradicted their previous expectations concerning the relationship between the variables, they shifted their strategy, trying to confirm their "illusory correlation" (Chapman & Chapman, 1969).

Qualitative Study of the Interrelation between Strategy and the Type of Association Perceived by Students

In the previous sections we have analyzed students' judgments of association and solution strategies for the different items. The cross tabulation of these two variables revealed a certain lack of consistency between the judgment of association and the strategy used to make this judgment.

We propose two hypotheses to explain the lack of correspondence between strategy and judgment in some students:

1. Students who employ a correct or partially correct strategy but give an incorrect association judgment, do so because they have a misconception about specific aspects of statistical association, although they have other correct conceptions which led them to choose the correct strategy.

2. Students who employ an incorrect strategy but give a correct judgment, do so because there are specific factors in the item in which success can be obtained with an incorrect procedure.

In both cases a strong relationship can be observed between the students' conceptions and the situations in which these conceptions are shown. In order to identify these incorrect conceptions we present a summary of a qualitative analysis of the relationship between strategy and judgment, that was carried out with the pilot sample of 51 students. The analysis was based on the written responses to the questionnaire. There was one student who gave no argument for item 1, two students who did not give any explanation for item 4 and 17 students who did not provide any argument for item 5. Table 6 presents the results for the remainder of the students.

Table 6. Frequency of strategies and judgments

 Item Judgement Strategy 1 2 3 4 5 Correct Correct 1 7 8 1 1 Partially correct 2 6 16 26 10 Incorrect 12 11 21 18 15 Incorrect Correct 1 1 0 0 0 Partially correct 7 5 2 1 6 Incorrect 28 19 4 4 19

Correct Strategy and Incorrect Judgment

In item1one student who used a correct procedure made an incorrect judgment, which showed a lack of proportional reasoning, as can be seen in his response:

Yes, there is an influence. Although both the percentage of smokers with bronchial disease and the percentage of smokers without bronchial disease are 60% and both the percentage of non smokers with bronchial disease and the percentage of nonsmokers without bronchial disease are 40%, there are more smokers with bronchial disease than nonsmokers.

This is a typical answer to problems involving the comparison of two probabilities in which the number of favorable and unfavorable cases is proportional (Piaget & Inhelder, 1951; Green, 1981). Also, Garfield and Ahlgren (1988) suggest that the lack of proportional reasoning is one of the most frequent causes of students' difficulties with statistical concepts. Nevertheless, we have not found any description of this type of answer in research on judgment of association.

In item 2, a typical answer was to consider the fact that taking the drug lowers the odds of having digestive troubles as an indication of a lack of dependence. So, dependence of two dichotomous variables was mistakenly conceived by these students as a coincidence of the presence or absence of these two variables and the inverse association for this item was never considered. As we have reasoned in the analysis of students' judgments for this item, we consider this type of answer to be an indicator of an unidirectional conception of association in which students only admit direct association or independence. The following response illustrates the case of inverse association is interpreted as independence:

There is a minimal dependence, because the percentage of old people who had digestive troubles and did not take the drug is higher than those old people who had taken the drug and had digestive troubles. Also the percentage of old people who had taken the drug and didn't have digestive troubles is higher than the percentage of people who did not take the drug and did not have troubles.

Partially Correct Strategy and Incorrect Judgment

On item 1, some students showed a lack of proportional reasoning, providing arguments similar to first sample response. On item 2, all students who used a partially correct strategy and provided an incorrect judgment used similar arguments to those in the second response, manifesting what we have called an unidirectional conception of association.

On item 3, one student who used a partially correct strategy stated the independence of variables (incorrect judgment) because he considered that cell [d] in the tables was contrary to association, although this cell is normatively equivalent to cell [a]:

When you study the table you can observe that, although the number of people with allergy and a sedentary life style is high, so is the number of people without allergy and with a non-sedentary life style. I believe you can't say there is dependence.

This answer could be explained by suggesting that there is a tendency to give more relevancy to positive cases that confirm a given hypothesis than to negative cases that confirm the same hypothesis (Crocker, 1981; Kunda & Nisbett, 1982). Even when the negative confirmatory cases (absence, absence) have a formal equivalence with positive confirmatory cases (presence, presence) they do not have a pragmatic equivalence with them. Subjects do not weigh a negative event in the same way as a positive one, because negative events have a lower impact on the subjects' attention.

Incorrect Strategy and Correct Judgment

On item1students frequently considered that cells [b] and [c] in the table ought to be null in order to assume association. For these students, a given value of one of the variables must always correspond to a given value of the other variable. That is, the correspondence between the variables must be, from the mathematical point of view, a function. We have considered this type of argument to be an indicator of a deterministic conception of association, because these students expected that the presence (absence) of A must imply the presence (absence) of B, as indicated by the following reasoning:

There is an influence of smoking, but it is not very great, because there are smokers without bronchial disease and nonsmokers with the disease.

Another case was that of students who considered that the frequency in cell [d] ought to be greater than the frequency in cell [c] in order to admit the existence of association. They admitted only a direct type of association, and so they could also be classified as possessing the unidirectional conception of association that has been described earlier:

It does not depend so much, because there are fewer people who don't have troubles among non smokers (40) than in the cell corresponding to smokers (60). In case of dependence the number of people without troubles (in non smokers) must surpass smokers by a great percentage.

Students who compared the relative frequencies in only one conditional distribution showed a localist conception of association, which has been described before. Depending on the distribution chosen, they obtained a direct, inverse, or no association, as in the following example concerning item 1, in which the unidirectional conception of association can also be observed:

I personally believe that there is no dependence on smoking in having or not having bronchial disease, because if we observe the table, in people with bronchial disease there is a higher percentage of non smokers.

Similar cases were observed for item 2 and in the students who used only one cell in the table.

Finally, on item 1 some students used their previous theories about the dependence of illnesses on smoking, instead of using the data provided in the table. As Nisbett and Ross (1980) have pointed out, subjects introduce new elements into the data of problems that are suggested by their previous experiences. We do not usually find contingency tables in decision making, although we attribute an effect to a given cause. We maintain our beliefs concerning the way in which events are (co) related, even when these theories do not correspond to environmental contingencies.

CONCLUSIONS

In this article, we have presented an experimental study of students' strategies in judging association in 2x2, 2x3 and 3x3 contingency tables and we have compared our results with previous research concerning 2x2 contingency tables. The classification of these strategies, from a mathematical point of view, has allowed us to identify intuitively correct and partially correct strategies that are indicators of correct or partially correct conceptions concerning statistical association. These strategies are the following:

1. Comparison of the conditional relative frequency distribution of one variable, when conditioned by the different values of the other variable, or comparison between these conditional distributions and the marginal relative frequency distribution.

2. Comparison of frequencies of cases in favor of and against each value of the dependent variable B or comparison of ratio of these frequencies in each value of the independent variable A.

In the same way, incorrect strategies and judgments have been used to identify the following three incorrect conceptions:

1. Determinist conception of association. Some students do not admit exceptions to the existence of a relationship between the variables. They expect a correspondence which assigns only a value in the dependent variable for each value of the independent variable. So, these students argue that the cells [b] and [c] in the 2x2 contingency table need to have zero frequency. When this is not so, they consider there is no dependency between the variables.

2. Unidirectional conception of association. Sometimes students perceive the dependence only when the sign is direct, and so they consider an inverse association as independence. This was observed especially on item 2, which presents an inverse association, although some students also provided this type of argument for item 1.

3. Localist conception of association. Students often form their judgment using only part of the data provided in the contingency table. If this partial information serves to confirm a given type of association, they adopt this type of association in their answer. Often this partial information is reduced to only one conditional distribution or even only one cell, frequently the cell for which the frequency is maximum.

The correspondence analysis carried out has shown the effect of task variables on the strategies. In particular, when the dimensions of the table are increased and students' previous beliefs are associated with incorrect strategies. Finally, we have observed a lack of proportional reasoning in our students, which made it difficult for them to solve item 3, concerning independence.

All these findings show the complexity of a topic that is simple in appearance. Due to this complexity and the relevance of understanding the idea of association, we conclude that there is a need to extend this research and reinforce the teaching of this topic at the university level and in the final years of secondary school. This will require informing teachers about the mathematical, epistemological, psychological, and educational aspects of the topic, including information about students' preconceptions as set out in this article.

REFERENCES

Allan, L. G. & Jenkins, H. M. (1983). The effect of representations of binary variables on judgment of influence. Learning and Motivation, 14, 381-405.

Alloy, L. B. & Tabachnik, N. (1984). Assessment of covariation by humans and animals: The joint influence of prior expectations and current situational information. Psychological Review, 91, 112-149.

Arkes, H. R. & Harkness, A. R. (1983). Estimates of contingency between two dichotomous variables. Journal of Experimental Psychology: General, 112 (1), 117-135.

Beyth-Marom, R. (1982). Perception of correlation reexamined. Memory and Cognition, 10(6), 511-519.

Brennan, R. L. (1983). Elements of generalizability theory. Iowa: ACT Publications.

Chapman, L. J. & Chapman, J. P. (1969). Illusory correlation as an obstacle to the use of valid psychodiagnostic signs. Journal of Abnormal Psychology, 74, 271-280.

Confrey, J. (1990). A review of the research on students conceptions in Mathematics, science, and Programming. Review of Research in Education, 16, 3-55.

Crocker, J. (1981). Judgment of covariation by social perceivers. Psychological Bulletin, 90 (2), 272-292.

Estepa, A. (1994). Concepciones iniciales sobre la asociación estadística y su evolución como consecuencia de una enseñanza basada en el uso de ordenadores. [Preconceptions of statistical association and their evolution as result of computer-based teaching] (Doctoral dissertation, University of Granada). Barcelona: ETD Micropublicaciones.

Garfield, J. & Ahlgren, A. (1988). Difficulties in learning basic concepts in probability and statistics: Implications for research. Journal for research in Mathematics Education, 19 (1), 44-63.

Godino, J. D. & Batanero, M. C. (1994). Significado institucional y personal de los objetos matemáticos. [Institutional and personal meaning of mathematical objects]. Recherches en Didactiques des Mathématiques, 14 (3), 325-355.

Green, D. R. (1981). Probability concepts in school pupils aged 11-16 years. Unpublished doctoral dissertation. Loughborough University of Technology. Loughborough, U.K.

Greenacre, M. J. (1984). Theory and applications of correspondence analysis. London: Academic Press.

Inhelder, B. & Piaget, J. (1955). De la logique de l'enfant á la logique de l'adolescent [The grouth of logical thinking from childhood to adolescence]. Paris: Presses Universitaries de France.

Jenkins, H. M. & Ward, W. C. (1965). Judgment of contingency between responses and outcomes. Psychological Monographs, 79, 1-17.

Jennings, D. L., Amabile, M. T. & Ross, L. (1982). Informal covariation assessment: Data-based versus theory-based judgments. In D. Kahneman, P. Slovic & A. Tversky (Eds), Judgment under uncertainty: Heuristics and biases, (pp. 211-230). New York: Cambridge University Press.

Kahneman, D., Slovic, P. & Tversky, A. (Eds) (1982). Judgment under uncertainty: Heuristics and biases. New York: Cambridge University Press.

Kunda, Z. & Nisbett, R. (1982). The psychometrics of everyday life. Cognitive Psychology, 18, 195-224.

Nisbett, R. & Ross, L. (1980). Human inference: strategies and shortcoming of social judgments. New Jersey: Prentice Hall.

Pearson, K. (1920). Notes on the history of correlation. Biometrika, 13, 25-45. Reprinted in E. S. Pearson y M. Kendall (Eds.) (1970), Studies in the history of statistics and probability, v.1 (pp. 185-205). London: Charles Griffin.

Pérez Echevarría, M. P. (1990). Psicología del razonamiento probabilístico. [The psychology of probabilistic reasoning]. Madrid: Ediciones de la Universidad Autónoma de Madrid.

Piaget, J. & Inhelder, B. (1951). La genese de l'idee de hasard chez l'enfant [The origin of the idea of chance in the child]. Paris: Presses Universitaires de France.

Scholz, R. W. (1991). Psychological research in probabilistic understanding. In R. Kapadia and M. Borovnick (Eds), Chance Encounters: Probability in Education (pp. 213-249). Dordrecht: Kluwer Academic Publishers.

Shaklee, H. & Tucker, D. (1980). A rule analysis judgments of covariation between events. Memory and Cognition, 8, 459-467.

Smedslund, J. (1963). The concept of correlation in adults. Scandinavian Journal of Psychology, 4, 165-173.

Vergnaud, G. (1982). Cognitive and developmental psychology and research inMathematics Education: some theoretical and methodological issues. For the Learning of Mathematics, 3 (2), 31-41.

Wright, J. C. & Murphy, G. L. (1984). The utility of theories in intuitive statistics: the robustness of theory-based judgments. Journal of Experimental Psychology, 113(2), 301-322.

AUTHORS' NOTE: This research was supported by the DGICYT grants PS88-0104 and PS90-0246, M.E.C. (Spain).

APPENDIX: QUESTIONNAIRE

ITEM 1 (Smoking): In a medical center 250 people have been observed in order to determine whether the habit of smoking has some relationship with bronchial disease. The following results have been obtained:

 Bronchial disease No bronchial disease Total Smoke 90 60 150 Not smoke 60 40 100 Total 150 100 250

Using the information contained in this table, would you think that, for this sample of people, bronchial disease depends on smoking? Explain your answer.

ITEM 2 (Drug): We are interested in assessing if a certain drug produces digestive troubles in old people. For a sufficient period, 25 old people have been studied, and these results have been obtained:

 Digestive troubles No digestive troubles Total Drug taken 9 8 17 No drug 7 1 8 Total 16 9 25

Using the information contained in this table, would you think that for this sample of old people digestive troubles depends on taking the drug? Explain your answer.

ITEM 3 (Allergy): In order to investigate whether a sedentary life style has some relationship with skin allergy, 30 people have been studied and the following data have been obtained:

 Skin allergy No skin allergy Total Sedentary life style 13 3 16 Non sedentary life style 2 12 14 Total 15 15 30

Using the information contained in this table, would you think that for this sample of people skin allergy depends on leading a sedentary life? Explain your answer.

ITEM 4 (Exam): In the following table we show the number of students who passed or failed an examination and the number of hours that these students dedicated to the study of the topic.

 Number of hours studying the topic Less than 5 hours 5 to 10 hours More than 10 hours Total Pass 11 25 35 71 Fail 15 7 7 29 Total 26 32 42 100

Using the information contained in this table, would you think that, for this sample of students, the result of the examination depend on the number of hours they have dedicated to study of the topic? Explain your answer.

ITEM 5 (Laterality): In an experiment we obtained the following data concerning people's use of one or both eyes and hands:

 Left eye Both eyes Right eye Total Lefthanded 3 6 2 11 Ambidextrous 2 2 2 6 Righthanded 5 9 4 18 Total 10 17 8 35

Using the information contained in this table, would you think that, for this sample of people, the use of eyes depends on the use of hands? Explain your answer.