Confounding Effect in Clinical Research of Otolaryngology and Its Control

Yong-qiang Yu*, Dong-yan Huang, Susan Armijo Olivo, Huai-an Yang, Yagesh Bambanini, Lyn Sonnenberg, Brenda Clark, Gabriela Constantinescu, Jason Qian Yu, and Ming Zhang

1Department of Communication Sciences and Disorders,3Research Center,

5Department of Occupational Therapy, Faculty of Rehabilitation Medicine, University of Alberta, Edmonton, AB T6G 2G4, Canada

2Department of Otorhinolaryngology Head and Neck Surgery, Clinical Division of Surgery, General Hospital of Chinese

People’s Liberation Army, Beijing 100853, China

4Department of Otolaryngology, the First Affiliated Hospital, China Medical University, Shenyang 110001, China

6Department of Developmental Pediatrics,9Department of Audiology, Glenrose Rehabilitation Hospital, Edmonton, AB T5G 0B7, Canada

7Institute for Reconstructive Science in Medicine, Edmonton, AB T5R 4H5, Canada

8Alberta School of Business, University of Alberta, Edmonton, AB T6G 2R6, Canada

10Department of Surgery-Otolaryngology, Faculty of Medicine and Dentistry, University of Alberta, Edmonton, AB T6G 2B7, Canada

REVIEW

Confounding Effect in Clinical Research of Otolaryngology and Its Control

Yong-qiang Yu1†*, Dong-yan Huang2†, Susan Armijo Olivo3, Huai-an Yang4, Yagesh Bambanini5, Lyn Sonnenberg6, Brenda Clark6, Gabriela Constantinescu7, Jason Qian Yu8, and Ming Zhang1,9,10

1Department of Communication Sciences and Disorders,3Research Center,

5Department of Occupational Therapy, Faculty of Rehabilitation Medicine, University of Alberta, Edmonton, AB T6G 2G4, Canada

2Department of Otorhinolaryngology Head and Neck Surgery, Clinical Division of Surgery, General Hospital of Chinese

People’s Liberation Army, Beijing 100853, China

4Department of Otolaryngology, the First Affiliated Hospital, China Medical University, Shenyang 110001, China

6Department of Developmental Pediatrics,9Department of Audiology, Glenrose Rehabilitation Hospital, Edmonton, AB T5G 0B7, Canada

7Institute for Reconstructive Science in Medicine, Edmonton, AB T5R 4H5, Canada

8Alberta School of Business, University of Alberta, Edmonton, AB T6G 2R6, Canada

10Department of Surgery-Otolaryngology, Faculty of Medicine and Dentistry, University of Alberta, Edmonton, AB T6G 2B7, Canada

clinical research; otolaryngology; confounding effect

Confounding effect is a critical issue in clinical research of otolaryngology because it can distort the research’s conclusion. In this review, we introduce the definition of confounding effect, the methods of verifying and controlling the effect. Confounding effect can be prevented by research’s design, and adjusted by data analysis. Clinicians would be aware and cautious about confounding effect in their research. They would be able to set up a research’s design in which appropriate methods have been applied to prevent this effect.They would know how to adjust confounding effect after data collection. It is important to remember that sometimes it is impossible to eliminate confounding effect completely, and statistical method is not a master key. Solid research knowledge and critical thinking of our brain are the most important in controlling confounding effect.

Chin Med Sci J 2015; 30(2)：121-130

CONFOUNDING effect is a common issue in clinical research of otolaryngology. When clinicians do a research, they expect a true conclusion. But sometimes a statistical significance may not be true, and there might be confounding effects in the research. For example, the authors did a research before, in which they wanted to compare the postoperative recurrent rates of otitis media between two surgical groups, one group undergoing radical mastoidectomy and another group receiving modified mastoidectomy. The result showed the radical surgery had a higher rate of re-infection than the modified surgery did.

These findings might be correct, but it could be wrong as well. The difference in re-infection rate between the two groups, might be due to the different surgical approaches received by the patients, but also could be due to a confounding effect which have been overlooked. The potential confounder might be diabetes mellitus.

Diabetes mellitus is a generalized disease, diabetic patients have a lower immunity than their normal counterparts do, and a lower immune protection will increase the rate of re-infection. Suppose the incidences of diabetes mellitus were different between the two groups, and this imbalanced distribution of diabetes mellitus might have effect on the difference in the rates of re-infection between the two groups. Diabetes mellitus is a third factor besides exposure and outcome, which is defined as a confounder because it might distort the research’s conclusion.

A confounder can overestimate or underestimate an exposure’s effect or even change a research’s conclusion completely. In order to reach a real conclusion, confounding effect must be under control, and fail to control the confounding effect will undermine a research’s credibility and ruin its validity.1, 2

Up to now we have not found paper written to discuss the specific confounding effect in clinical research of otolaryngology, and information in this field is very limited. The aim of this review is to help otolaryngologists and junior clinical researchers to recognize and deal with confounders in their research. This article will introduce the definition of confounder, its common sources, and the methods of dealing with confounding effect.

DEFINITION OF CONFOUNDING FACTOR

Confounding effect refers to a situation where a spurious association is found between exposure and outcome due to a confounder. A confounder must not be an intermediate factor between exposure and outcome, it is an independent third factor besides exposure and outcome.3,4A confounder is usually associated with both exposure and outcome, and unevenly distributed between the comparing groups,5so the estimation of exposure effect can be contaminated, and the conclusion may not be true.1,2

IDENTIFICATION OF CONFOUNDING EFFECT

Compare odds ratio (OR) before and after stratification

In order to verify a suspicious confounder, researchers first need to prove it is associated with both exposure and outcome, then to compare the OR before and after adjusting the suspicious factor. An OR is a measure of association between an exposure and an outcome. The OR represents the odds that an outcome will occur given a particular exposure, compared to the odds of the outcome occurring in the absence of that exposure.6

The crude data of the author’s research are shown in Table 1. In the group of radical mastoidectomy (radical group), 62 out of 96 (64%) patients developed postoperative re-infection; in the group of modified mastoidectomy (modified group), 88 out of 202 (44%) developed re-infection. The crude OR was 2.36 (unadjusted OR), indicating the re-infection rate of the radical group was 2.36 times higher than that of the modified group. However, diabetes mellitus was a potential confounder, so the authors went through the following steps to verify the confounding effect.

Step 1: To prove if diabetes mellitus was associated with surgical selection (exposure). The crude data from Table 1 was stratified by diabetes mellitus, and shown inTable 2. In diabetic patients, 53% (55 out of 103) underwent radical mastoidectomy. In non-diabetic patients, 21% (41 out of 195) received radical surgery. An OR of 4.3 was calculated, which indicated diabetes mellitus was associated with surgical selection.

Table 1. Unadjusted odds ratio (OR) of postoperative recurrence of infection for radical versus modified mastoidectomy

Step 2: To verify if diabetes mellitus was associated with the rate of re-infection (outcome), and data were shown in Table 3. In diabetic patients, 72% (74 out of 103) developed postoperative re-infection. In non-diabetic patients, 39% (76 out of 195) had re-infection. An OR of 3.99 was calculated, which meant diabetes mellitus was related to an increased rate of re-infection. Therefore, diabetes mellitus was associated with both surgical selection and the rate of re-infection. This result is similar to other researches, in which diabetes mellitus is an unfavourable factor in the treatment of chronic otitis media.7-9

Step 3: The crude data were stratified into two strata, shown in Table 4. In the stratum with diabetes mellitus, the adjusted OR was 0.91. In the stratum without diabetes mellitus, the adjusted OR was 1.14 (Table 4). The adjusted OR of each stratum was very close to 1, which indicated that after stratification there is no association between surgery selection and the rate of re-infection.

The crude OR was 2.36, the pooled OR calculated by Mantel-Haenszel method was close to 1 (Table 4). And the incidence of diabetes mellitus in the radical group was 57% (55 out of 96), the incidence in the modified group was 24% (48 out 202). There was a confounding effect due to the imbalanced distribution of diabetes mellitus between two groups, so research conclusion was distorted. In order to prevent or eliminate confounding effect, researchers need to know where the effect comes from. Directed Acyclic Graphs (DAG) will help to identify the potential confounders.

Table 2. Prove if patients with diabetes mellitus are more likely to have radical surgery

Table 3. Prove if patients with diabetes mellitus are more likely to have recurrent infection

Table 4. Calculation of adjusted OR after stratification by diabetes mellitus

DAG—an aide to assess confounder

Sometimes it is difficult to judge if a suspicious factor is a confounder. Making a wrong adjustment for a false confounder can even produce a new confounding effect which does not exist before.10DAGs is a method to help to verify suspicious confounders. DAGs is a group of arrows drawn along time sequence which provide a graphic description of the relationship among exposure factors, potential confounders, and outcomes. So a quick visual inspection can be done to assess the potential confounders. Because in real life it is impossible to go back to the past, no cycle will be present in DAGs, it is acyclic, and the arrows are always pointing towards the outcome.11,12

For example, researchers want to investigate the effect of an ototoxic drug on tinnitus. A single-headed arrow from the drug to tinnitus represents a causal pathway between them (Fig. 1). It is known the drug is metabolized by kidney, kidney function can affect the drug’s serum level; and abnormal kidney function itself is also a risk factor of tinnitus.13An arrow is directed from kidney function to drug and tinnitus respectively.

In conventional statistics, kidney function can produce a confounding effect because it is associated with both the drug and tinnitus, and it can set up pathway relationship between them. In DAGs, the cofounding effect is termed as backdoor, which is constituted by the arrows from kidney function (confounder) to drug (exposure) and tinnitus (outcome). The graphic description makes it easy for quick visual inspection of confounding effect. Researchers can close the backdoor pathway by stratification of kidney function, and then they will be able to observe the true effect of the drug on tinnitus.

COMMON CONFOUNDING EFFECT IN CLINICAL RESEARCH OF OTOLARYNGOLOGY

Confounding effect is very common in clinical research of otolaryngology. It can be due to susceptibility bias, exposure selection bias, clinical indication bias, contamination bias, or any combination of these biases.

Figure 1. Effect of a drug on tinnitus illustrated with Directed Acyclic Graphs.

Confounding effect from susceptibility bias

In a clinical setting, a patient may develop a disease at first, then a second disease which is related to the first one might follow up, this is because the first disease can predispose the development of the second one. If researchers want to compare the incidence of the second disease between the group A and group B; and if the group A has the first disease, and the group B does not, then the susceptibilities of the second disease are different between the two groups, this susceptibility bias can cause confounding effect.14

For example, researchers try to compare the incidence of nasal polyp between two groups. One group has a family history of allergic rhinitis, the other does not. Because allergic rhinitis can predispose the development of nasal polyps, this will cause confounding effect on the incidence of nasal polyps between the two groups.

Confounding effect from selection bias

In a clinical study, researchers need to set up selection methods for the participants. If the methods are different between the two groups, it may cause confounding effect.15For example, when researchers try to investigate the relationship between air pollution and laryngeal cancer, they want to compare the incidences between two small towns, town A and B. In town A there are enough volunteers available to participate in the study. But in town B, volunteers are not enough, so non-volunteers are also recruited.

The volunteers might have more health consciousness than the non-volunteers do. The volunteers might pay more attention to nutrients, do regular exercise, etc., which can improve immunity and reduce the cancerigenic effect of air pollution. While in non-volunteer, the situation is different, so there is a confounding effect in the research due to selection bias.

Confounding effect from clinical indication bias

The treatment decision is based on clinical indication, such as the diagnosis, the severity of a disease, the anticipated prognosis, and the baseline morbidity, and patient’s indications are always different. In a research of comparing different treatments, the conscious allocation of patients to different treatment groups which is based upon indication, can result in confounding effect, for the simple reason that randomization has been violated. Clinical indication can be a confounder.16,17

For example, there are two chemotherapy agents available for nasopharyngeal cancer, drug A and B. Researchers want to compare the effect between A and B. Either A or Bwill be prescribed to the patients according to clinical indications, such as the tumor’s histological type, degree of differentiation, tumor size, and remote metastasis, etc. There is no randomization, and the indications between the two groups are different and incomparable, and the final drug effect is confounded by these indications, this is called confounding effect by indication bias.

Confounding effect from contamination bias

This may happen when the exposure effect is contaminated by other factors, which is particularly common in a cohort study, such as testing a newly developed treatment.15For example, surgeons are interested in developing a modified laryngectomy, patients who have accepted this new surgery, probably will receive more postoperative care than those who have undergone standard operation. The modified surgery may improve prognosis, but it is hard to determine whether the improved prognosis is due to the new surgery itself, or due to the increased postoperative care. Thus, the increased postoperative care is a confounding factor.

Once confounding effect is recognized to be present in a research, it can be prevented or controlled by study design and/or data analysis. In study design, the commonly used methods are restriction, matching, and randomization. In data analysis, stratification, analysis of covariance and multivariable regression are often used.5,18

CONTROL OF CONFOUNDING EFFECT IN STUDY DESIGN

Restriction

Restriction is a simple and effective method to eliminate, or at least, partially reduce confounding effect. By restriction, specific criteria will be set up for participants so that the selected subjects are restricted to the individuals with certain characteristics. For example, when researchers try to investigate the risk factors of laryngeal cancer other than smoking, they would select only non-smoking patients to remove the potential confounding effect of smoking. However, the number of eligible participant will be cut down, and the external validity of the study will be reduced as well, because it is no longer possible to assess the synergism effect of smoking on laryngeal cancer.

Therefore, restriction should be applied to real confounders only. The fewer factors are restricted, the more closer the sample is to the target population, and the more generalized the research result is.1,3

Matching

Matching can increase the similarity between comparing groups. In this way even if there are potential confounders in a study, they are matched between the comparing groups.16,19The common matching factors are age, sex, addictions, illness status, and comorbidity, etc.

Matching can be used in a cohort study or a case-control study. For example, in a case-control study, researchers try to compare the rates of wound infection between two surgical groups. One group receives radical neck dissection, another group undergoes modified neck dissection. The two groups should be matched on some potential confounders, such as age, nutrition status, and diabetes mellitus.

In a case-control study, when the case number is limited, small sample size can reduce statistic power. Matching has a solution to this problem. Each case can be matched with multiple controls, so that the sample size is increased, and statistic power will be enhanced accordingly.20

It may be difficult to achieve matching if several confounders need to be matched in a study. Even if all the confounders are matched after strenuous effort, an additional drawback may occur, called overmatching. Overmatching can occur when the controls are very similar to the experimental group, particularly when the matched factors are highly correlated with the exposure.21,22

Overmatching can make the exposure effect less apparent than it should be, and the exposure effect is diluted out. In other words, there is a new factitious confounding effect produced by overmatching. If overmatching exists in a study, researchers might fail to reveal a causal relationship between exposure and outcome.23The best way of preventing overmatching is to match the most significant confounders only, the less important confounders can be adjusted by data analysis.19,24

As in restriction, the matched variables can no longer be evaluated as risk factors for the outcome.25,26If one member of a matched pair withdraws, this pair has to be excluded from the analysis, so matching might lose data.27,28

Randomization

Matching cannot deal with unknown confounders, randomization can particularly reduce/eliminate unknown confounding effect which might exist in a research. Given sufficient subjects are recruited and randomized into both control and experimental groups, randomization can increase the comparability between the two groups because all variables are evenly distributed. Even if there might be unknown confounders in a research, but they are balancedbetween the two groups.16,18,25,29The larger the sample size is, the more balance will be achieved by randomization, so randomization may be inefficient when the sample sizes is small.29,30

Violations of randomization, such as loss of follow-up, may disturb the balanced distribution of variable between the groups, and the comparability is lost. A solution for the loss of follow-up, is to use the analysis of intention to treat in which comparison is carried out by the treatment assigned to all the subjects rather than the treatment received by the subjects.16

Randomization is an optimal method to control confounding effect. Its advantage is to balance both known and unknown confounders between comparing groups. However, randomization is not always feasible in a clinical trial. For example patients who want to participate in a new drug trial, probably have failed all other treatments, so it would be unethical if they are randomized to a control group treated by a conventional drug which has been proved to be no benefit for them. This ethical concern does not allow randomization of participants. And randomization is not always possible in an observational study.31

CONTROL OF CONFOUNDER IN DATA ANALYSIS

If confounding effects are not preventable in study design, they should be controlled at the step of data analysis. Stratification, analysis of covariance, and multivariable regression analysis maybe employed.

Stratification

After data collection a crude OR can be calculated. Then the data can be divided into a few strata based on suspicious confounders, and stratum specific ORs will be compared with the crude OR. The comparison may be easy if the stratum specific ORs are equal or very close to each other.

However, the exposure effects in each stratum can be different, and stratum specific ORs are different too. In this case, a pooled ORs can be calculated by Mantel–Haenszel method,18,27,32which can be used to compare with crude OR. If a minimum change of 10% is observed, it indicates the exposure effect has changed significantly before and after stratification, then the suspicious factor will be considered as true confounder.3

The calculation of pooled OR is shown in Table 4. The crude OR was 2.36, the stratum specific ORs were 0.91 and 1.14 respectively, the pooled OR was 1.03, and a change of 56% was observed between the crude and pooled OR (2.36-1.03/2.36 = 56%), suggesting that diabetes mellitus was a confounder.

Stratification is an effective method to deal with dichotomous confounder because the data can be easily divided into two strata. For continuous confounder, such as age, tumor size and burden, arbitrary strata can be created. However, arbitrary stratification cannot remove confounding effect completely, the fewer arbitrary strata created, the more residual confounding effects left in each stratum; increasing the number of stratum requires large sample size.5Stratification becomes difficult if multiple confounders are present in a research, a huge sample size will be needed for stratification.5,18

Effect modification in stratification A pooled OR may become invalid if there is effect modification in stratification. In statistical analysis, an interaction may arise when the simultaneous effect of two or more exposure factors on an outcome is not additive, but multiplicative, this interaction is called effect modification; even though an individual variable may not produce individual confounding effect, but when two or more variables are associated together, there might be an integrated confounding effect, or effect modification.33

Effect modification can be found out by a quick visual inspection. For example, data were stratified based upon a suspicious confounder, if all the stratum specific ORs are very close to each other, there is no effect modification. If the stratum specific ORs vary considerably, there is an effect modification.

Collider in stratification The judgment of a suspicious confounder (true or false) is based upon biological knowledge about the causal relationship between exposures and outcome. The wrong stratification of a false confounder will produce a new confounding effect which does not exist before.10,34,35

For example, the authors found out a drug was an effective treatment for tinnitus, but after the data were stratified on gender, there was no treatment effect in either male or female group (Tables 5, 6); this was because gender was a false confounder. The explanation is that, according to biomedical information, gender had no effect either on theexposure (drug/placebo) or on the outcome. But in each stratum gender was mistakenly connected with the exposure and the outcome because there was an imbalanced distribution of gender in the two strata. In the male stratum more participants were in the drug group, while in the female stratum more participants were in the placebo group.

Table 5. Overall analysis of the effect of drug for tinnitus improvement

Table 6. Analysis of the effect of drug for tinnitus improvement after stratification by gender

A false confounder is called a collider DAG (Fig. 2). When the arrows from two distinct variables meet at a third variable, they are said to collide and the third variable is called a collider.36A quick judgement of suspicious confounder is to see the arrow direction. If gender has effect on the exposure and outcome, it is a true confounder, and the arrow direction is from gender towards both exposure and outcome. In Figure 2 the arrow direction is the other way around, pointing towards a false confounder.

Analysis of covariance

As discussed before, it would be hard to stratify a continuous confounder. In this case other statistical method maybe helpful, such as analysis of covariance (ANCOVA).5,18ANCOVA is another option of dealing with confounder. It is a general linear model which combines the analysis of variance and regression. ANCOVA is suitable for a research which has a dependent variable (DV), a categorical independent variable (IV), and other continuous covariates (CVs, quantitative or scaled). IV is the experimental factor. CVs are not of primary interest, but have effect on the DV, so CV can be confounders.37-39

By statistical adjustment, the DV is corrected to what it should be when the potential confounding effect of the CV is under control. The real effect of the experimental factor (IV) on DV can be provided and called experimental effect, which is estimated by comparing the amount of variability in the data that can be explained by the experimental factor, against the variability that it cannot be explained by the factor.

Figure 2. A collider in Directed Acyclic Graph.

ANCOVA uses F test, and F = MSbetween/MSwithin(MS is mean’s square). MSbetweenis the explainable variance from experimental factor, and MSwithinis the unexplainable variance which comes from random error and the effect of covariate (potential confounding effect). When F value is larger than a critical value, the experimental effect is considered to be significantly different between comparing groups.38,39

When the effect of covariate is under control, the denominator is smaller, F becomes larger, thereby ANCOVA can increase statistical power, increase the possibility to find a significant difference between comparing groups given the difference really exists.40

For example, the authors tried to compare the anti-tumor effect among three kinds of Chinese herbal medications in mouse model of nasopharyngeal cancer (Table 7). The authors wanted to compare the changes of tumor size before and 5 weeks after medication, and this change was the dependent variable Y. In the study there were two potential confounders, the initial tumor size before taking medication and the amount of drug eaten by animal (X). Initial tumor size can be stratified, and ANCOVA can be applied to adjust the covariate X.

In Table 7, 72 mice were stratified into 8 strata according to the initial tumor size, 9 mice in each stratum. Then the 9 mice were randomized into 3 drug groups, and each drug group had 3 mice. The averages of X and Y were calculated respectively. ANCOVA was used to compare the Y among the three drug groups, while X was taken as a covariate. After analysis by SPSS 15, the F value was 10.805, P = 0.001<0.05 (Table 8). So after adjusting the amount ofdrug eaten by mouse, the changes of tumor size were still significant among the three drug groups.

Table 7. Adjustment of covariate by analysis of covariance (n=3)

Table 8. SPSS analysis result of Table 7

Sometimes it may be hard for ANCOVA to deal with several covariates, because the analysis result would be complicated and difficult to interpret.41Multiple regression is suitable for the analysis of two or more covariates.

Multivariable regression analysis

General introduction of multivariable regression analysis Multivariable regression analysis is a mathematical model that can estimate the association between one dependent variable (outcome) and many independent factors (exposures), or evaluate the individual effect of any single exposure while other factors are under control.42,43The regression analysis is a powerful, flexible but complicated method which can be used to control many confounders, which can be continuous or dichotomous.44

In this regression analysis, it is the value of coefficient that is considered important, but not the statistical significance (P value). P value can indicate if the research null hypothesis is acceptable. A null hypothesis usually states that the coefficient of a variable is zero. P < 0.05 means there is only less than 5% probability that coefficient is zero, in other words, it is very likely that coefficient is not zero which indicates the variable has effect on the outcome. The bigger the coefficient, the higher the effect intensity. But P value itself has nothing to do the effect intensity. It is very possible to have a highly significant result (very small P-value) for a small effect intensity. Only the effect intensity can tell the extent of association, and how importance of a variable is in predicting an outcome.45

There is a requirement of sample size for regression analysis, the ratio is minimum 10 subjects for one independent variable. If the sample size is insufficient, the analysis result may be inaccurate or deviated.42,46

DAGs can help to minimize confounders When many potential confounders exist in a research, DAG can help to identify the key confounders and minimize the numbers of confounders. In Figure 3, this is an example about the effect of a drug on tinnitus. There are many risk factors related to tinnitus, some of them are potential confounders, the situation is complicated. However, a DAG can be drawn according to biomedical knowledge, which can provide a piece of information about how to analyze data.

After study the DAGs, only two confounders need to be adjusted, kidney function and endocrine imbalance. After controlling the two factors (remove the factors), all the backdoor pathways will be closed, and the real effect of the drug on tinnitus can be obtained. With the assistance of DAGs, researchers can identify and control the key confounders so that they can analyze complicated data with great efficiency.

Figure 3. Scheme of how to minimize confounders in a research with Directed Acyclic Graph.

CONCLUSION

Confounding effect is very common in clinical research of otolaryngology. It occurs when a risk factor is associated with both the exposure and the outcome. When confounding effect exists, the research validity is distorted. So it is very important to understand what a confounder is, and how to control it.

Confounding effects can be controlled at two stages, study design and data analysis. In study design, the commonly used methods are restriction, matching, randomization; in data analysis, stratification, analysis of covariance, and multivariable regression are often applied. The optimal stage of controlling confounding effect is at the study design. Randomization is would be an effective method, successful randomization can minimize confounding effect by balancing measurable variables as well as unmeasurable factors between comparing groups. If randomization is or not applicable, other methods such as restriction, matching, and stratification can be adopted. Stratification can also help to identify effect modification which is an important issue in confounding effect.

With the aid of statistical software, researchers feel more confident in dealing with confounders at the stage of data analysis. However, sometimes it is impossible to get rid of confounding effect completely, because there are always unmeasurable biases in a research. Elaborate study design and advanced analysis method cannot guarantee no confounding effect. The most important is that we cannot depend upon statistical method completely.

For example, DAGs is drawn according to biomedical rationale, if different DAGs can be drawn, researchers can reach completely different conclusions depending upon which DAGs they believe to be true. In Figure 4, researchers tried to investigate the causal relationship among three variables, acute laryngopharyngitis, acute otitis media, and serum level of C-reactive protein (CRP, an inflammatory marker). The data showed that there was a significant relationship between every two variables.

Two DAGs could be drawn which might lead to two different conclusions. In Figure 4A acute laryngopharyngitis could cause acute otitis media which is mediated by CRP. In Figure 4B, there were no real relationship between acute laryngopharyngitis and acute otitis media, the statistical association is established by a backdoor pathway of CRP. So statistical method is not a master key. Solid research knowledge and critical thinking of our brain are the most important in controlling confounding effect.

Figure 4. Two different DAG explanations for one set of data.A. According to this DAG, acute laryngopharyngitis can cause otitis media with a mediator CRP.B. In the DAG, there is no true association between acute laryngopharyngitis and acute otitis media. The statisticalrelationship is established by the backdoor pathway CRP.CRP: C-reactive protein.

REFERENCES

1. Hennekens CH, Buring JE. Epidemiology in Medicine. Philadelphia: Lippincott Williams & Wilkins; 1987.

2. Bailey L, Vardulaki K, Langham J, et al. Introduction to Epidemiology. Berkshire: Open University Press; 2005.

3. Kestenbaum B. Methods to control for confounding. In: Epidemiology and Biostatistics. An introduction to clinical research. New York: Springer; 2009.

4. Wunsch G. Confounding and control. Demographic Res 2007; 16:97-120.

5. Petrie A, Sabin C. Bias and confounding. Medical statistics at a glance. 3rd ed. Oxford: Wiley-Blackwell; 2009.

6. Szumilas M. Explaining Odds Ratios. J Can Acad Child Adolesc Psychiatry 2010; 19:227-9.

7. Choi SY, Cho YS, Lee NJ, et al. Factors associated with quality of life after ear surgery in patients with chronic otitis media. Arch Otolaryngol Head Neck Surg 2012; 138:840-5.

8. Avetisyan N, Lautermann J. Ear diseases and diabetes mellitus. J HNO 2014; 62:823-32.

9. Orji FT, Dike BO, Oji O. Determinants of non-healing ear discharge in chronic suppurative otitis media in a developing country. Eur Arch Otorhinolaryngol 2014;[Epub ahead of print].

10. Schisterman EF, Cole SR, Plattc RW. Overadjustment bias and unnecessary adjustment in epidemiologic studies. Epidemiology 2009; 20:488-95

11. Weinberg CR. Can DAGs clarify effect modification? Epidemiology 2007; 18:569-72.

12. Vanderweele TJ, Robins JM. Directed Acyclic Graphs, sufficient causes, and the properties of conditioning on a common effect. Am J Epid 2007; 166:1096-104.

13. Nowak K, Banaszewski J, Dabrowski P, et al. Tinnitus in systemic diseases. Otolaryngol Pol 2002; 56:213-6.

14. Horwitz RI, McFarlane MJ, Brennan TA, et al. The role of susceptibility bias in epidemiologic research. Arch Intern Med 1985; 145:909-12.

15. Kramer MS. Clinical epidemiology and biostatistics. New York: Springer; 1988. p. 54-6.

16. Greenland S, Morgenstern H. Confounding in health research. Ann Rev Public Health 2001; 22:189-212.

17. Lobo FS, Wagner S, Gross CR, et al. Addressing the issue of handling bias in observational studies with propensity scores analysis. Res Social Adm Pharm 2006; 2:143-51.

18. Jager KJ, Zoccali C, Macleod A, et al. Confounding: What it is and how to deal with it. Kidney Int 2008; 73:256-60.

19. Rothman JJ. Modern epidemiology. 6th ed. Boston: Little Brown and Company; 1986. p. 84-94.

20. Jaeschke R, Sackett DL. Research methods for obtaining primary evidence. Int J Technol Assess Health Care 1989; 5:503-19.

21. Bland JM, Altman DG. Statistics notes: Matching. BMJ 1994; 309:1128.

22. Bland JM, Altman DG. Matching in case-control studies. BMJ 1995; 310:329-30.

23. Marsh JL, Hutton JL, Binks K. Removal of radiation dose response effects: An example of over-matching. BMJ 2002; 325:327-30.

24. Hajian-Tilaki KO. Evaluation of the rule of matching in case-control and cohort studies. Proceeding of the 6th International Statistics Conference. Tehran: Tarbiat Modarres University; 2002. p. 32.

25. Oesterling JE, Cooner WH, Jaconsen SJ, et al. Influence of patient age on serum PSA concentration. An important clinical observation. Urol Clin North Am 1993; 20:671-80.

26. Brookhart MA, Sturmer T, Glynn RJ, et al. Confounding control in healthcare database research: Challenges and potential approaches. Med Care 2010; 48:114-20.

27. Gordis L. More on causal inferences: Bias, confounding and interaction. 4th ed. Philadelphia: Saunders Elsevier; 2009. p. 247-65.

28. van Stralen KJ, Dekker FW, Zoccali C. Confounding. Nephron Clin Pract 2010; 116:c143-7.

29. Farrokhyar F, Bajammal S, Kahnamoui K, et al. Practical tips for surgical research. Ensuring balanced groups in surgical trials. Can J Surg 2010a; 53:418-23.

30. Farrokhyar F, Karanicolas PJ, Thoma A, et al. Randomized controlled trials of surgical interventions. Ann Surg 2010b; 251:409-16.

31. Egger M, Schneider M, Davey SG. Spurious precision? Meta-analysis of observational studies. BMJ 1998; 316:140-4.

32. Tripepi G, Jager KJ, Dekker FW. Stratification for confounding—part 1: The Mantel-Haenszel formula. Nephron Clin Pract 2010; 116:c317-21.

33. Fisher L, Patil K. Matching and unrelatedness. Am J Epidemol 1974; 100:347.

34. Simpson EH. The interpretation of interaction in contingency tables. J R Stat Soc Series 1951; B13:238-41.

35. Wagner CH. Simpson's paradox in real life. Am Stat 1982; 36:46-8.

36. Greenland S, Pearl J, Robins JM. Causal diagrams for epidemiologic research. Epidemiology 1999; 10:37-48.

37. Keppel G. Design and analysis: A researcher's handbook. 3rd ed. Englewood Cliffs: Prentice-Hall Inc.; 1991.

38. Howell DC. Statistical methods for psychology. 7th ed. Belmont: Cengage Wadsworth; 2009.

39. McNamee R. Regression modelling and other methods to control confounding. Occup Environ Med 2005; 62:500-6.

40. Tabachnick BG, Fidell LS. Using multivariate statistics. 5th ed. Boston: Allyn and Bacon; 2007.

41. Kutner MH, Nachtsheim CJ, Neter J, et al. Applied Linear Statistical Models.5th ed. New York: McGraw-Hill/Irwin; 2005.

42. Tripepi G, Jager KJ, Dekker FW. Linear and logistic regression analysis. Kidney Int 2008; 73:806-10.

43. Blackstone EH. Comparing apples and oranges. J Thorac Cardiovasc Surg 2002; 123:8-15.

44. Morshed S, Tornetta P Ⅲ, Bhandari M. Analysis of observational studies: A guide to understanding statistical methods. J Bone Joint Surg Am 2009; 91:50-60.

45. Good PI, Hardin JW. Common Errors in Statistics (And How to Avoid Them). 3rd ed. New Jersey: Wiley; 2009.

46. Peduzzi P, Concato J, Kemper E. A simulation study of the number of events per variable in logistic regression analysis. J Clin Epidemiol 1996; 49:1373-9.

for publication October 21, 2014.

†These authors contributed equally to this work.*

E-mail: yyu2@ualberta.ca