Please note : the data presented in all course material for the statistical module are
generated by computers to demonstrate the methodologies, and should not be confused with
actual clinical information
Introduction
Exercises
Exercise 4 contains exercises for statistics for metaanalysis. As metaanalysis can be used for all effect size and their Standard Error, the theoretical contents and calculations that are used to produce effect size are all of the contents of the module other than those for predictions. Likewise, the calculations to be carried out use most of the StatPgm algorithms. Therefore no specific ones will be linked from this panel, and it is assumed that, by this stage, students will have no difficulty finding the contents and the algorithms on this site.
The Microsoft Office package of Word, Excel, and Powerpoint, or similar software, should be activated during the exercise. Excel is a useful tool to manipulate data, Powerpoint is useful to edit graphics, and the results should be copied to and edited in a Word file.
Please note : The data in the exercises on this page were generated by computers to deliberately having problems with heterogeneity and publication bias, so that the students are encouraged to work their way through the problem and gain some skills in addition to just running the computer programs.
It is envisaged that some students would be initially confused or even intimidated by the complexity involved. Students are however encouraged to try and seek help if necessary, and not get to worried about initial difficulties.
Metaanalysis
Questions 1_1 : Metaanalysis of iron therapy : click to show contents
 Grp 1 : Iron Therapy  Grp 2 : Control 
Country  n  mean  SD  n  mean  SD 
Australia  150  11.6  4.2  145  11.8  4.3 
Bangladesh  180  11.5  4.8  150  8.3  5.2 
Buranda  195  12.1  3.6  200  9.2  4.5 
Canada  309  12.3  3.8  300  12.4  3.5 
England  180  12.0  4.0  180  11.3  4.1 
Nigeria  130  11.5  3.8  120  9.1  3.8 
USA  150  12.3  3.8  160  12.4  3.5 
Zimbabwe  145  11.2  4.3  150  8.3  4.8 
We wish to review the effectiveness of iron therapy to increase Haemoglobin level in pregnant women.
We found 8 controlled trials, listed in the table to the right
 Create a table of Effect Sizes and their Standard Errors. Each row contains results from each trial, and there should be 5 columns
 Column 1 is the country where the trial was carried out
 Column 2 is the difference in Haemoglobin concentration between the treatment and the control group
 Column 3 is the Standard Error of the difference
 Column 4 and 5 are the 95% confidence interval of the difference (two tail)
 Create a Forest plot to show the 95% confidence intervals of the studies
 Evaluate the heterogeneity of the data set
 Produce a x/y scatter plot, where the x axis is the mean haemoglobin level of the control group, and the y value is the
effect size
 Evaluate the cause of heterogeneity, and make decisions as to how to combine the data
 Combine the data to produce one or more summary effects
 Draw conclusions from the analysis, produce Forest plots to support these conclusions
Answers 1_1 : click to show contents
Country  Difference  SE  95% CI 
Australia  0.2  0.4949  1.1700  0.7700 
Bangladesh  3.2  0.5512  2.1196  4.2804 
Buranda  2.9  0.4107  2.0950  3.7050 
Canada  0.1  0.2963  0.6807  0.4807 
England  0.7  0.4269  0.1367  1.5367 
Nigeria  2.4  0.4811  1.4570  3.3430 
USA  0.1  0.4146  0.9126  0.7126 
Zimbabwe  2.9  0.5312  1.8588  3.9412 
 Q Test p=0.0001, I2= 92% of variance attributable to differences between study.
Severe and significant level of heterogeneity exists
 The heterogeneity exists because the countries involved are from two distinct clusters
 Bangladesh, Buranda, Nigeria, and Zimbabwe is a cluster where the control group have low Haemoglobin levels, and
iron therapy appears to be effective in raising the levels.
Heterogeneity of this cluster : Q Test p=0.73 (n.s.), I2=0%. No evidence of heterogeneity
 Australia, Canada, England, and USA is the other cluster where the control groups already have high Haemoglobin levels,
and iron therapy have little effect in raising the levels further
Heterogeneity of this cluster : Q Test p=0.4 (n.s.), I2=0%. No evidence of heterogeneity
 The conclusion is that first world countries, where nutrition levels are high and iron therapy irrelevant, should not be
analysed together with third world countries, where nutritions are poor and iron therapy is beneficial.
The two clusters of countries should be separately analysed.
 Using the Random Effect model
 The combined summary effect for Bangladesh, Buranda, Nigeria, Zimbabwe cluster is : Difference (TmtCont) = 2.8,
Standard Error = 2.419, 95% CI = 2.4 to 3.3
 The combined summary effect for Australia, Canada, England, USA cluster is : Difference (TmtCont) = 0.05,
Standard Error = 0.1932, 95% CI = 0.3 to 0.4
 Final Conclusions
 It was an error to include trials of iron therapy from very different environmental background in the first place.
 In the third world countries where nutrition is poor and women have numerous pregnancies, iron deficiency is likely,
and iron therapy can be expected to work
 In the first world countries where nutrition is good and women have few pregnancies, iron deficiency is uncommon,
and iron therapy cannot increase Haemoglobin levels above what is already optimal.
 The test for heterogeneity is therefore useful in identifying this error
 When the two clusters of countries are analysed separately
 no heterogeneity is detected in either cluster
 For the third world cluster, the improvement in Haemoglobin averaged 2.8, 95% confidence interval 2.4 to 3.3. As this
does not overlap the null position (0), the improvement is statistically significant, and the hypothesis that
iron therapy is beneficial can be supported
 For the first world cluster, the improvement in Haemoglobin averaged 0.05, 95% confidence interval 0.3 to +0.4.
As this overlaps the null position (0), the improvement is not statistically significant, and the hypothesis that
iron therapy is beneficial cannot be supported
Questions 1_2 : Metaanalysis of hormonal support in early pregnancy : click to show contents
 Treated with Hormone  Placebo 
Historical order  Total  Aborted  Live Birth  Total  Aborted  Live Birth 
1  15  1  14  16  6  10 
2  50  3  47  60  10  50 
3  30  2  28  40  10  30 
4  80  10  70  60  10  50 
5  150  12  138  180  20  160 
6  218  35  183  240  34  206 
7  500  78  422  550  83  467 
8  420  65  355  450  68  382 
A theory exists that miscarriages (spontaneous abortions) can be caused by hormonal deficiency, and that women with a history of repeated abortions may benefit from hormonal support in early pregnancy to prevent abortion.
We wish to clarify whether treating women in early pregnancy can reduce abortion rates, and obtained 8 controlled trials for a metaanalysis. The data is as presented in the table to the right.
 Create a table of effect sizes from the data, each row contains results of a study, and should have 8 columns
 Column 1 is the order of the study
 Column 2 is the total sample size from both groups
 Column 3 and 4 are the risk of abortion in the hormonal and placebo groups
 Column 5 is the Effect Size, the Risk Difference (Risk_{Hormone}Risk_{Placebo})
 Column 6 is the Standard Error of the Risk Difference
 Column 7 and 8 the 95% confidence interval (2 tail) of the Risk Difference
 Estimate whether Publication Bias exists
 Report and comment the results of the Rank Correlation Test
 Create a x/y scatter plot with the x axis the historical order and y axis the Risk Difference
 Create a x/y scatter plot with the x axis the total sample size and y axis the Risk Difference
 Comment on the possible patterns that may explain the publication bias
 Combine the studies to create summary Effect Size and its 95% confidence interval. Tabulate the results and create one or more
Forest plot to demonstrate the 95% confidence intervals of the analysis
 Combine studies where total sample size is less than 100
 Combine studies where total sample size is 100 or more
 Combine all studies
 Comment and draw conclusions from results of the analysis
Answers 1_2 : click to show contents
Historical Order  Total Sample Size  Risk Hormone  Risk Placebo  Risk Difference  Standard Error  95%CI 
1  15  0.067  0.375  0.308  0.137  0.577  0.04 
2  50  0.06  0.167  0.107  0.059  0.222  0.008 
3  30  0.067  0.25  0.183  0.082  0.345  0.022 
4  80  0.125  0.167  0.042  0.061  0.161  0.077 
5  150  0.08  0.111  0.031  0.032  0.094  0.032 
6  218  0.161  0.142  0.019  0.034  0.047  0.085 
7  500  0.156  0.151  0.005  0.022  0.039  0.049 
8  420  0.155  0.151  0.004  0.024  0.044  0.052 
 Analysis for Publication Bias
 Rank Correlation Test p=0.03, significant publication bias exists
 Combining data to produce Summary Effect Size
 The first 4 studies where sample size <100 : Risk Difference=0.121, Standard Error=0.044,
95% confidence interval=0.208 to 0.034
 The last 4 studies where sample size >=100 : Risk Difference=0.000, Standard Error=0.013,
95% confidence interval=0.226 to +0.027
 All 8 studies combined : Risk Difference=0.029, Standard Error=0.020,
95% confidence interval=0.069 to +0.012
 Comments
 The early studies (first 4) were small sample size ones (sample size <100), and they show significant or near significant
risk difference. If conclusions were to be drawn from these 4 studies, the hypothesis that hormonal support in early
pregnancy reduces spontaneous abortion would have been supported.
 The latter studies (last 4) were large studies (sample size >=100), and they show effect sizes that were not significant
and close to zero. If conclusions were to be drawn from these 4 studies, the hypothesis that hormonal support in early
pregnancy reduces spontaneous abortion would not have been supported.
 Taking all 8 studies together, the combined effect size is not statistically significant, and the hypothesis that
hormonal support in early pregnancy reduces spontaneous abortion would not have been supported.
 The results demonstrate a classical situation of Publication Bias, where early small studies that show significant results
were offered for publication while nonsignificant results were shelved and not published. Only when large definitive
studies were conducted did the true picture emerged, that hormonal support in early pregnancy makes no difference to
spontaneous abortion rate
 From the technical point of view, many would argue that all studies should be combined, as the large definitive
studies would dominate the data, as in this study, and a nonsignificant conclusion will be drawn
Others, however, would argue that the smaller studies should be deleted and analysis should be confined to the
larger studies
Some would argue that all data should be included, but statistical adjustment can be made to reduce the
Publication Bias. Statistical methods for doing so are advanced topics not covered in the module. For those
interested, they can read a discussion of it in the metaanalysis discussion page on the department's StatTools site
