Chi Square Tests

Chi Square Tests

PhD Özgür Tosun

IMPORTANCE OF

EVIDENCE BASED

MEDICINE

The Study

 Objective: To determine the quality of health

recommendations and claims made on popular medical talk shows.

 Sources: Internationally syndicated medical television talk shows that air daily (The Dr Oz Show and The Doctors).

 Interventions: Investigators randomly selected 40

episodes of each of The Dr Oz Show and The Doctors from early 2013 and identified and evaluated all

recommendations made on each program. A group of experienced evidence reviewers independently searched for, and evaluated as a team, evidence to support 80

randomly selected recommendations from each show.

 Main outcomes measures: Percentage of

recommendations that are supported by evidence as

determined by a team of experienced evidence reviewers.

Results

 On average, The Dr Oz Show had 12 recommendations per episode and The Doctors 11.

 At least a case study or better evidence to support 54%

(95% confidence interval 47% to 62%) of the 160 recommendations (80 from each show).

 For recommendations in The Dr Oz Show, evidence

supported 46%, contradicted 15%, and was not found for 39%.

 For recommendations in The Doctors, evidence supported 63%, contradicted 14%, and was not found for 24%.

 The most common recommendation category on The Dr Oz Show was dietary advice (39%) and on The Doctors was to consult a healthcare provider (18%).

 The magnitude of benefit was described for 17% of the recommendations on The Dr Oz Show and 11% on The Doctors

Conclusions

 Recommendations made on medical talk shows often lack adequate information on specific benefits or the magnitude of the effects of these benefits.

 Approximately half of the

recommendations have either no

evidence or are contradicted by the best available evidence.

 The public should be skeptical about

recommendations made on medical talk shows.

A Fictional Answer for a Random Dr. Oz’s Recommendation

 Dr Oz:

◦"Saturated fat is solid at room

temperature, so that means it's solid inside your body."

 Patient:

◦Thanks, Dr. Oz. You give the best advice.

◦Carrots are very hard and dense, so they'll petrify (transform into stone) your body, turning you into an

orange statue. Am I doing it right?

Pull down that bread, kiddo!!!

CATEGORICAL

ONE SAMPLE TWO SAMPLES >2 SAMPLES

CATEGORICAL

ONE SAMPLE TWO SAMPLES >2 SAMPLES

Independent Paired Independent

CATEGORICAL

ONE SAMPLE TWO SAMPLES >2 SAMPLES

Independent Paired Independent

2 x 2 Chi Square Test

Mc Nemar Test

N x M Chi Square Test

One Sample Chi Square

Test Fisher’s

Exact Test

Nonparametric

One sample difference of proportions test

Parametric

Cross Table (Contingency Table)

• enables showing two or more variables simultaneously in table format

• a table of counts cross-classified according to categorical variables

• best way to include sub-group descriptive statistics

• simplest contingency table is a 2 x 2 table

• is good for demonstrating possible relationships among variables

Cross Table (Contingency Table)

• An r X c contingency table shows the

observed frequencies for two variables.

• The observed frequencies are arranged in r rows and c columns.

• The intersection of a row and a column is called a cell

Misreading the Table

• it is important to correctly read the information given in a table

• although the original data do not change at all, tables can be arranged in several different views

• looking at the table does not necessarily show the reader about possible relationships among variables

– in order to decide on the existence of relationship,

«statistical hypothesis testing» is required

Observed versus Expected

• In a cross tabulation, the actual numbers in the cells of the table are called the observed values

• Observed Frequencies are obtained empirically through direct observation

• Theoretical, or Expected Frequencies are developed on the basis of some hypothesis

Expected Frequencies

• Assuming the two variables are independent, you can use the contingency table to find the expected frequency for each cell.

Finding the Expected Frequency for Contingency Table Cells

The expected frequency for a cell E_r,c in a contingency table is

, (Sum of row ) (Sum of column )

Expected frequency .

Sample size

r c r c

E  

Example:

Find the expected frequency for each “Male” cell in the contingency table for the sample of 321 individuals.

Assume that the variables, age and gender, are independent.

105 6

10 21

33 22

13 Femal

e 16

10 61 and older

321 38

64 85

73 45

Total

216 28

43 52

51 32

Male

Total 51 –

60 41 –

50 31 –

40 21 –

30 16 –

20 Gende

r

Age

Expected Frequency

Example continued:

105 6

10 21

33 22

13 Female

16 10 61 and older

321 38

64 85

73 45

Total

216 28

43 52

51 32

Male

Total 51 – 60

41 – 50 31 – 40

21 – 30 16 – 20

Gender

Age

, (Sum of row ) (Sum of column ) Expected frequency

Sample size

r c r c

E  

1,2 216 73 49.12 E  321 

1,1 216 45 30.28

E  321  _1,3 216 85 57.20

E  321 

1,5 216 38 25.57 E  321 

1,4 216 64 43.07

E  321  _1,6 216 16 10.77

E  321 

Chi-Square Independence Test

• A chi-square independence test is used to test the independence of two variables.

• Using a chi-square test, you can determine whether the occurrence of one variable affects the probability of the occurrence of the other variable.

• For the chi-square independence test to be used, the following must be true.

1. The observed frequencies must be obtained by using a random sample.

2. Each expected frequency must be greater than or equal to 5.

Chi-Square Independence Test

• We are looking for significant differences

between the observed frequencies in a table (f_o) and those that would be expected by

random chance (f_e)

2 x 2 Chi Square

df = (r-1)(c-1)=1

+ -

1 O₁₁

2

Total N

First criteria

Total Second

Criteria

O₁₂ O₂₁ O₂₂ O_.1 O_.2

O₁. O_2.

E_ij should be greater than or equal to 5.



 ² 

1 i

2 1

j ij

2 ij 2 ij

E

) E χ (O

N O E_ij  O^{i. .j}

Is squint more common among children with a positive family history?

Is there an association between squint and family history of squint?

+ -

+ 20 30 50

- 15 55 70

Total 35 85 120

Squint (Şaşılık)

Total Family

History

²_(1,0.025)=5.024 > 4.869. Accept H₀.

There is no relation between squint and family history

14.58 35.42 20.42 49.58

4.869 χ² 

Attention

In 2 X 2 contingency tables,

if any expected frequencies are less than 5, then alternative procedure to called Fisher’s

Exact Test should be performed.

An Example

• A study was conducted to analyze the relation

between coronary heart disease (CHD) and smoking.

40 patients with CHD and 50 control subjects were randomly selected from the records and smoking habits of these subjects were examined. Observed values are as follows:

Observed and expected frequencies

+ -

Yes No

Total 90

Smoking

Total CHD

30

4 46

14 76

40 50

10 ^{6.2 33.8}

7.8 42.2

( ) ( ) ( ) ( )

95 . 4 2 =

. 42

2 . 42 –

+ 46 8

. 7

8 . 7 – + 4

8 . 33

8 . 33 – + 30

2 . 6

2 . 6 –

= 10

E

) E –

= (O χ

2 2

2 2

2

1

= i

2

1

=

j ij

2 ij

2

∑∑

df = (r-1)(c-1)=(2-1)(2-1)=1

²_(1,0.05)=3.841

Conclusion: There is a relation between CHD and smoking.

² =4. 95 > reject H₀

An Example for Fisher’s Exact Test

Research question: does positive BRCA1 gene

actually affects the occurrence of breast cancer?

Since the percentage of the cells which have expected count < 5 is 50%, Fisher’s exact test should be applied.

According to Fisher’s test, p value is 0.070 p>α

Fail to reject H₀

BRCA1 gene has no affect on breast cancer

McNemar Test

• 35 patients were evaluated for arrhythmia with two different medical devices. Is there any statistically significant difference between the diagnose of two devices?

Device I

Device II

Total Arrhythmia (+) Arrhythmia (-)

Arrhythmia (+) 10 3 13

Arrhythmia (-) 13 9 22

Total 23 12 35

The significance test for the difference between two dependent population / McNemar test

H₀: P₁=P₂ H_a: P₁ P₂

Critical z value is ±1.96 Reject H₀

25 .

13 2 3

1 – 13 –

3

1 – –

 



 

c b

c z b

McNemar test approach:

²_(1,0.05)=3.841<5.1 p<0.05; reject H₀.

c b

c b

  ²

2 ( – )

 b c

c b

 

2

2 ( – –1)



1 . 13 5

3

) 1 – 13 –

3 ( )

1 – –

( ^{2 2}

2 

 

 

c b

c

 b

Evaluation of arrhythmia patients using these two devices will provide significantly different results. Further research is required to understand which one is better for diagnosis.

A researcher wants to know whether the mothers age is affecting the probability of having congenital abnormality of neonatals or not. The collected data is given in the table:

Congenital abnormality

Total Present Absent

Age groups ≤25 3 22 25

26-35 8 34 42

>35 18 16 34

Total 29 72 101

N x M Chi Square

H₀: There is no relation between the age of mother and presence of congenital abnormality.

Under the assumption that null hypothesis is true:

(Expected count)

Congenital abnormality Present Absent Age groups ≤25 3 (7.2) 22 (17.8)

26-35 8 (12.1) 34 (29.9)

>35 18 (9.8) 16 (24.2)

Reject H₀

Congenital abnormality χ² Present Absent

Age

groups ≤25 3 (7.2) 22 (17.8) 3.44 26-35 8 (12.1) 34 (29.9) 1,95

>35 18 (9.8) 16 (24.2) 9,64 Omit the

>35 age group

Congenital abnormality Present Absent

groupsAge ≤25 3 22

26-35 8 34

H₀ isaccepted

At the end of the analysis, we should conclude that the risk of

having a baby with congenital abnormality is significantly higher for

>35 age group.

However, risk is not differing significantly between <= 25 age group and 26-35 age group

Attention

In N x M contingency tables,

if the proportion of cells those have expected frequencies less than 5 is above 20%, then it is

not possible to perform any statistical analysis

EXAMPLE: Researcher wants to know if there is any

significant difference among education groups in terms of their alcohol consumption rates

At the end of the analysis, since the proportion of cells which have

expected count <5 is 50%, we must conclude that this hypothesis cannot be tested under this circumstances. The samples size in the study is not high enough.

Calculated p value is not valid.