Learn more about analytics and research best practices, as well as real world examples and solutions for nonprofits.

RECENT POSTS

Jan 29, 2019

Are My Response Rates Statistically Significant?

I’m taking a different approach in today’s blog to discuss a problem I continue to run into when I am working with clients. And that is testing whether the response rates in a direct mail test are meaningful.

Now response rates are relatively uncomplicated to test. Anyone can do it. That’s because there are only two possible outcomes: Response or nonresponse.

Here’s an online testing tool for you to use.

After you click on the link, you will need to enter your data. The “base sizes” are the number of pieces mailed in each of your tests. The “proportions” are the number of responses for each test divided by the number of mailed. You can choose your confidence level (we recommend 90%), and then click calculate.

Voila!

I suggest using a 90% confidence level because it is a high standard. That means if you have this difference in 100 tests, in 90 of those tests the difference will be real. The “Results” box will state either “Significant” or “NOT Significant.” You don’t need to understand statistics to use this tool!

Another helpful metric to know is your average gift size. Now, testing average gift size is more work to calculate, because the values you are testing can range from $0.01 to $1 million or more. That means you can’t use summary statistics (averages) to calculate significance. Instead, you need the entire gift distribution. Plus, you will need a more robust software tool than Excel to do this. We use SPSS.

We suggest that you talk with your analytics people to have them help you through testing average gift size differences.

Thank you for joining me for today’s stats class.

Measuring Speedy Gratitude

On the first day of November, I sent out 49 small “white mail” gifts. After 3-weeks, here is what I received: Two of the gifts were returned because the address was undeliverable. Of the 47 gifts that (I think) were delivered, I have received 14 acknowledgments (30%...

read more

Grateful

Here it is Thanksgiving week. If you are like most people in the fundraising sector, the lead-up to Thanksgiving can be a hectic period, with the most important cultivation asks either in the mail or ready to be deployed, new donor acquisition campaigns in-home, and...

read more

Giving Tuesday: Is it still working for you?

In the past, some of us at Analytical Ones have been a little “Grinch-y” about Giving Tuesday. Being blitzed by emails during the workday can be overwhelming. However, we won’t knock it if it’s working! In September, Classy (the online fundraising platform utilized by...

read more

ARCHIVES

© 2020 Analytical Ones