Center for Earth and Environmental Science
Indiana University ~ Purdue University, Indianapolis

The Experiment

There are three commonly used methods for floodplain or bottomland restoration in the Midwest. One of the questions we're interested in evaluating is which method, if any, works best.  At the ARBOR site, each method was used in two 1-acre plots; we have also kept unplanted control plots for comparison. In both the north and south sections, four 1-acre areas were planted as follows (numbers = total trees planted in the north & south areas):

1. Control Plots - were mowed and left unplanted.
2. Containerized Plants - 420 trees, each in 3-gallon containers, were planted every 12 feet after an initial mowing and the application of spot herbicide.
3. Bare Root Seedlings - 480 seedlings planted in a random pattern after mowing and the application of spot herbicide.
4. Bare Root Seedlings - 432 seedlings were planted in rows, after mowing and treatment with herbicide, and fitted with a weed inhibitor mat. Native wild rye grass was planted between rows to control competition.

Each treatment method was planted in both the northern and southern sections. Plots were randomly placed in each section and individual trees were planted randomly within each treatment plot. Additionally, equal numbers of trees and an even distribution of tree species were planted in each treatment plot. Trees included 12 native species that naturally occupy the edges of rivers and streams (= riparian) and species adapted to the environmental conditions of the Midwest, generally, and to the Tipton Till Plain, specifically. They exclude extremely rare or habitat-restricted species, including rock elm and blue ash, and the American elm. The latter was formerly an important canopy species on many Indiana floodplains but Dutch elm disease now routinely kills them before they reach canopy height. Therefore, it was also excluded from the planting list. 

How Well Are the Trees Surviving? 

Restoring a section of the White River’s floodplain to its natural state is one goal of the ARBOR Project but we're also attempting to determine the best method of accomplishing the  restoration. One way to evaluate the success or failure of our efforts is to examine how the trees are 1) surviving as time progresses within a single treatment plot (strategy) and, 2) whether or not different planting strategies enhance a species’ chances of survival.

We want to statistically evaluate tree survival (= quantitative assessment) and not just look at raw numbers (or percentages) of surviving trees and make a value judgment (= qualitative assessment). How should this be accomplished?

So, How do we do this?

The chi-square test:  This type of data (counts) is ideally suited for a statistical test called the chi-square test. The results of the test let you know the degree of confidence you can have in accepting or rejecting a hypothesis. Typically, the hypothesis being tested with chi square is whether or not two separate samples are different enough, in some characteristic or aspect of their behavior, that we can consider them to be different statistically. Here, for example, we’re interested in the survival of species of trees within each planting area as time passes and in survival patterns related to different planting strategies. Stating this another way, do particular trees tend to survive better than others, and do one or more methods of planting increase a species chances for survival over other methods or practices?

How does chi square do this? Basically, the chi square test of statistical significance is a series of simple mathematical formulas that compare the observed frequencies of some phenomenon (in our sample) with the frequencies we would expect if there were no relationship at all between the two variables in the larger (sampled) population. This hypothesis of no difference is called the null hypothesis. Chi square tests our actual observations against the null hypothesis and assesses whether the actual results are different enough to overcome a certain probability that they could be due to sampling error. For example, our null hypotheses are that 1)  there is no difference in the ability of trees to survive as time passes, and 2) differences in tree survivorship between areas or plotting strategies do not exist.

An example of calculating Chi²

 Let’s run a chi square test on data taken from Sokal and Rohlf (1987 p. 307) where we will test the effect of an antiserum on bacteria infecting populations of mice. The setup of the test will be similar to that for our tree data. 

1.      A sample of 111 mice was administered a dose of bacteria and was then divided into 2 groups:  57 mice that also received a dose of antiserum and a control group of 54 that did not.

2.      After the disease had run its course, 38 dead mice & 73 survivors were counted. Thirteen of the dead mice had received the antiserum while 25 received only bacteria.

3.      The question of interest is did the antiserum protect the mice so that there were proportionately more survivors in that group than in the control group?

Observed Frequencies:  It is convenient to display the observed data in a two-way or contingency table:

Two-way or Contingency table: 
Marginal Totals are in bold  
Observed: 

 Within the table, you can see the number of dead & surviving mice while the sums, called marginal totals, indicate the total number of mice that display any one property. For example, 57 mice were treated with antiserum while 38 mice did not survive the experiment.

Calculating expected frequencies:

Next, we need to determine the expected numbers of living and dead mice based on the null hypothesis that the proportions do not differ between treated and untreated mice. To do this, we use the marginal totals from the table of observed frequencies of living & dead mice. Let’s calculate them and then display those values in a table similar to the one above:

1.      Bacteria + antiserum (Dead)
(38 x 57)/111 = 20

2.      Bacteria + antiserum (Alive)
(73 x 57)/111 = 37

3.      Bacteria only  (Dead)
(38 x 54)/111 = 18

4.      Bacteria only  (Alive)
(73 x 54)/111 = 36

Expected:

 We now can compare the observed results against the results expected if the null hypothesis were true. However, we need some way to determine whether the null hypothesis can be rejected and, if it can be rejected, what level of confidence we have that we're not making a mistake.

Calculating Chi²

To do this we must measure the size of the difference between each pair of observed and expected frequencies for cells in our contingency table. To accomplish this, we calculate the difference between the observed (O) and expected (E) frequency in each cell, square that difference, and then divide that product by the difference itself. The formula for chi square is:

Χ² = (O - E)²/E

Squaring each difference eliminates negative values because some will be larger and others smaller than the expected value. If we didn't do that, the positive and negative differences across the entire table would add up to zero. Dividing the squared difference by the expected frequency removes the expected frequency from the equation, so that the remaining measures of observed-expected difference are comparable across all cells.

For our immunology example, the individual chi square calculations are:

Bacteria + antiserum  (Dead)

Χ² = (O - E)²/E = (13 - 20)²/20 = 2.45

Bacteria + antiserum  (Alive)

Χ² = (O - E)²/E = (44 - 37)²/37 = 1.32

Bacteria only  (Dead)

Χ² = (O - E)²/E = (25 - 18)²/18 = 2.72

Bacteria only  (Alive)

Χ² = (O - E)²/E = (29 - 36)²/36 = 1.36

The Total Chi Square Value

χ² = å (O - E)²/E = 7.85

Calculation of Chi² (χ²):

Is this value significant, statistically?

We need to know how much larger than Zero (the absolute chi square value of the null hypothesis) our calculated chi square value must be before we can confidently reject the null hypothesis. If we do this manually, we’d have to look at a table of chi square values, choose a level of significance, and look up the appropriate value for a test with 1 degree of freedom (for a 2 x 2 table).

If we did that, we’d discover that the tabled value of chi square for:

• 1 degree of freedom and
• a significance level of 0.05
  (we’d be willing to be wrong 1 time in 20 or correct 95% of the time)
• the value would be = 3.841.
• Our calculated value of 7.85 exceeds the tabled value by a large margin.

This means that we cannot accept the null hypothesis of no difference in the survival of mice. The antiserum DOES make a difference!

 The statistical programs that are currently available for personal computers do all of this for you: 

• they automatically calculate the expected frequencies for your table and determine the probability of getting the chi square value that was calculated.
• This means that you don’t have to worry about degrees of freedom or looking values up in a table for comparison!
• If you’re interested in the details of the assumptions for chi square tests, making calculations, the concept behind the number of degrees of freedom, and confidence levels, you can go to Dr. Jeff Connor-Linton’s web site at Georgetown University to read about them and examine his examples.

 OK, What About the Trees at the ARBOR Site?

The Data:  Click here to show the mortality data for the trees planted at the ARBOR Research site. Use the tabs at bottom of spreadsheet to alternate between Mortality-Fall 2001, Mortality-Spring 2002 and Mortality-Spring 2003. Notice that we’ve recorded the number of trees that were planted in each area (column titled Planted).  We also have been counting, at regular time intervals, the number of trees that are still Alive and the number that have died (Dead).

The Tools:  Click here to view Dr. Connor-Linton’s Web Chi Square Calculator. Using this web site, you can calculate chi square values interactively by simply entering frequencies of living and dead trees into an electronic contingency table.

Tree Survival within an single planting area:

First, let’s take a look at the survival pattern of Red Maples from Area 1 (containerized trees) that were planted in October of 2000.

Examine the two tables below.

 Notice 

1. In October, 2001, 16 of the 21 Red Maples that had been planted were still alive.
2. In March 2003, the number of Red Maples that were still alive had decreased to 9.

We want to determine if the number of dead maples in March 2003 is significantly different than the number of dead maples in October 2001

Performing the Chi2 test using Web-based Chi2 Calculators: Exercise 1

1. Begin by opening Dr. Connor-Linton’s Web Chi Square Calculator. 
2. The relevant rows from the previous two tables are copied below

3. First enter the dimensions of the table, which are 2 rows x 2 columns, and press the Generate Table button.
4. Double click in the Table Name box & enter "Red Maple Survival – Area 1".
5. Press the Tab key and enter Alive in the Col 1 box.
6. Press the Tab Key and enter Dead in Col 2.
7. Press the Tab Key and enter October 2001 in the Row 1 Box
8. Press the Tab Key and enter the number of trees alive in October 2001 (see table) 
9. Press the Tab Key and enter the number of trees dead in October 2001 (see table) 
10. Press the Tab Key and enter March 2003 in the Row 1 Box
11. Press the Tab Key and enter the number of trees alive in March 2003 (see table) 
12. Press the Tab Key and enter the number of trees dead in March 2003 (see table) 
13. Press the Calculate Chi Square Key to perform the calculations.
    (NOTE: The normal radio button is the default and produces the contingency table, degrees of freedom, chi square value, the probability that you’d get a value of that magnitude assuming no difference between categories, and whether or not that value is significant.  If you choose the verbose radio button, you’ll receive all of the output for the normal button plus calculations of the expected frequencies for each cell in the table and calculations of probabilities.)
14. What can you conclude from these results?

Now let's do a similar calculation for Chinkapin oaks in Area 2

Chinkapin Oaks in Area 2

Next, perform a similar test for chinkapin oaks in area 2 (bare-root, random planting).

Click to see all tree mortality data again

Dr. Connor-Linton’s Web Chi Square Calculator 

1. The relevant rows from the previous two tables are copied above
2. Enter the dimensions of the table (2 X 2); press the Generate Table button.
3. In the Table Name enter "Chinkapin Oak Survival – Area 1".
4. Enter Alive in the Col 1 box.
5. Enter Dead in Col 2.
6. Enter October 2001 in the Row 1 Box
7. Enter the number of chinkapin oaks alive in October 2001
8. Enter the number of chinkapin oakss dead in October 2001
9. Enter March 2003 in the Row 1 Box
10. Enter the number of chinkapin oaks alive in March 2003
11. Enter the number of chinkapin oaks dead in March 2003
12. Press the Calculate Chi Square Key to perform the calculations.
13. What can you conclude from these results?
14. Are Chinkapin oaks surviving as well as the Red Maples?

Comparing Differences in Survivorship Between Areas - Different Planting Schemes

Let’s begin by comparing different planting schemes within the same general area of study – the northern area.  This time, we’ll use data for Honey Locust

1. The March 2003 survey for Area 1  (containerized trees) indicated that all 15 of the Honey Locust planted in October of 2000 were still alive.

2. The March 2003 survey for Area 4 (bare-root, planted in rows, with fertilizer and  protective mats around trees) had 9 of its 13 Honey Locust trees surviving in March 2003.

3. Run a chi square test for these data.  

Dr. Connor-Linton’s Web Chi Square Calculator 

4. Answer the following:

1. Does Honey Locust survivorship differ between these two areas? 
2. What can you conclude, specifically, about survivorship in these two areas.

Comparing Differences in Survivorship Between Areas - Same Planting Schemes, Different Location

Next, let’s compare the same planting scheme in a northern and a southern area, again using data for Honey Locust from the March 2003 survey.

1.                  Area 4 (bare-root, planted in rows, protective mats around trees, fertilized) is at the southern edge of the northern area and had 9 of 13 trees surviving.

2.                  Area 6 is in the middle of the southern planting area, used the same planting protocol as Area 4, and had 10 of 13 trees surviving.

3.                  Perform a Chi square analysis and analyze the results.

Dr. Connor-Linton’s Web Chi Square Calculator 

 Click here to return to top of page.

Center for Earth and Environmental Science
 School of Science
 Indiana University~Purdue University, Indianapolis
 723 West Michigan Street, SL118
 Indianapolis, IN 46202
 www.cees.iupui.edu
 cees@iupui.edu