Adam, thanks for the additional detail. Regarding your last question in the P.S., no, I do not believe that will provide you with a defensible conclusion. That approach would tell you whether the intervals described by the margins of error of the two samples overlap or touch, but that doesn't really answer the question about whether the means could have resulted from two survey groups drawn from the same population. The more robust way to draw this type of conclusion is to perform a hypothesis test where P1 and P2 are the mean population proportions (intended to be inferred from the sample means p1 (or A) and p2 (or B) in your sheet):

**Describe the sample statistic, create the hypotheses, and establish the level of significance**
In this case, I think the question being asked involves whether the two samples could have been drawn from the same population, a population whose mean characteristic (whatever the survey is trying to assess) remained about the same over the years covered by the surveys. The sample statistic would represent the difference of the population means, where we treat each sample as though it was drawn from a population associated only with it...so a sample mean of p1 from the first survey is intended to represent a random selection of respondents drawn from a population whose mean is P1, and similarly, p2 from the second survey would be representative of a population whose mean is P2. Then the hypotheses focused on the population means would be stated like this:

Null hypothesis: H0: P1-P2=0 or put differently, P1 = P2, that is, we will accept or reject this depending on the outcome of the test statistic below.

Alternate hypothesis: H1: P1-P2 <> 0 or put differently P1 <> P2

Other assumptions that need to be reviewed:

- Are the survey samples considered large (perhaps >= 30), which goes toward the type of test statistic to be used (probably either Z or t).
- Are the two survey samples independent and unpaired (also said to be "unmatched", meaning that the same people were not used in both samples, except for the occasional one who may have been chosen randomly for both surveys).
- What is the nature of the survey response variable? Is the response variable dichotomous or continuous? An example of the former: "Do you favor term limits for federal judges? Yes or No", and for the latter, "How many miles do you drive annually?" The answer to this question changes the form of the equation used to estimate the standard deviation of the sampling distribution of the difference. This estimate is often needed because the standard deviation of the populations is typically not known.
- Are the population variances believed to be equal (i.e., the variance of the population represented by the first survey is approximately the same as the variance of the population represented by the second survey). This assumption is typically assessed by examining the ratio of the two sample standard deviations to confirm that it falls somewhere between 0.5 and 2. If so, then the assumption of approximately equal population variances is typically accepted as reasonable.
- Evaluate at a level of significance of alpha = 0.05, a typical value associated with statements that carry a fairly high degree of confidence.

A quick back-of-the envelope assessment, making some assumptions (so please assess whether these are valid):

- The sample sizes are large (to be confirmed next), so the assumption of normality is reasonable and the Z test is an appropriate test statistic. The response variable is dichotomous, so the margin of error (MOE) for some confidence level is a function of the mean and the sample size only. Let's say the hypothesis is to be tested at the level of significance of alpha =0.05 (i.e., the test rejects H0 at alpha=0.05) and the distribution is two-tailed, so the critical Z value, Z_c=1.96 (two-tailed at 95 % confidence). Then the form of the MOE equation is: MOE = Z_c * sqrt( p * (1-p) ) / sqrt ( n )

The MOE equation is solved twice...one time for each survey. We know the sample proportion means p1=0.10 and p2=0.15. This suggests that survey 1 had a sample size n1=384, and survey 2 had a sample size n2=1225. Both of these are considered large, so the assumption of normality is also justified.

**Selection of the test statistic:**
Since the sample sizes are large, we look at the mean proportions (p1=0.10 and p2=0.15). These are toward the edges of the distribution, but still probably central enough for a valid test. Then the form of the Z test is given by:

z = (p1 - p2) / sqrt ( p * (1-p) * ( 1/n1 + 1/n2 ) )

where we've already described p1 and p2, but we see that we have a new variable, p, representing the overall proportion of responses that are consistent with p1 and p2, and which is essentially a weighted average. First we consider that the proportion of respondents in the first survey responding one way is p1, which is given by p1 = x1 / n1, where x1 is the number of respondents giving that particular response and n1 is the total number of respondents in the first survey. So x1 = p1 * n1 = 0.10 * 384 = 38. Similarly, x2 = p2 * n2 = 0.15 * 1225 = 184. Turning now to the computation of p:

p = ( x1 + x2 ) / ( n1 + n2 ) = ( 38 + 184 ) / (384 + 1225 ) = 0.1380

**State the Decision Rule:**
Reject H0 if Z<=-1.960 or if Z>=1.960.

**Compute the test statistic:**
Substituting into the equation for z...

z = (p1 - p2) / sqrt ( p * (1-p) * ( 1/n1 + 1/n2 ) ) = z = (0.10 - 0.15) / sqrt ( 0.138 * (1-0.138) * ( 1/384 + 1/1225 ) ) = -2.479

**Conclusion:**
We reject the null hypothesis H0 since -2.479 <= -1.96. There is statistically significant evidence at alpha = 0.05 to show that there is a difference in the means of the two surveys, even though the sample margins of error result in the two intervals touching each other.

You might want to investigate some online resources...all of these are good:

Enroll today at Penn State World Campus to earn an accredited degree or certificate in Statistics.

online.stat.psu.edu

How to construct a confidence interval for the difference between two sample means. Step-by-step instructions, including two sample problems with solutions.

stattrek.com

If you have a number of these results to evaluate, you could set this up in Excel following this or a similar approach for each evaluation.