Are My Response Rates Statistically Significant?

I’m taking a different approach in today’s blog to discuss a problem I continue to run into when I am working with clients. And that is testing whether the response rates in a direct mail test are meaningful. Now response rates are relatively uncomplicated to test. Anyone can do it. That’s because there are only two possible outcomes: Response or nonresponse. Here’s an online testing tool for you to use. After you click on the link, you will need to enter your data. The “base sizes” are the number of pieces mailed in each of your tests. The “proportions” are the number of responses for each test divided by the number of mailed. You can choose your confidence level (we recommend 90%), and then click calculate. Voila! I suggest using a 90% confidence level because it is a high standard. That means if you have this difference in 100 tests, in 90 of those tests the difference will be real. The “Results” box will state either “Significant” or “NOT Significant.” You don’t need to understand statistics to use this tool! Another helpful metric to know is your average gift size. Now, testing average gift size is more work to calculate, because the values you are testing can range from $0.01 to $1 million or more. That means you can’t use summary statistics (averages) to calculate significance. Instead, you need the entire gift distribution. Plus, you will need a more robust software tool than Excel to do this. We use SPSS. We suggest that you talk with your analytics people to have them help you through testing average gift size...

People First, CRM Second

A festering trend of the past decade is many of our clients have migrated from micro-computer based fundraising databases to CRM (Constituent Relationship Management) cloud-based databases. And for 10-years we continue to see too many of these organizations’ ability to leverage their data for insights decline. Rather than the CRM serving the nonprofit, it seems like the nonprofit now must serve the CRM. I still find this so ironic and frustrating. Databases don’t solve problems – people solve problems. I know it’s not any one CRM or any one client. I think most every organization underestimates the commitment of undertaking migrating to a CRM. They also underestimate the cultural change required to harness the power of CRM – which includes adding highly trained (and highly salaried) fundraising professionals to run it. Most organizations understand the need for CRM and are willing to swallow the expensive of the database infrastructure, but in my experience, too few organizations have been willing to cover the cost to train their people adequately or add CRM professionals who know fundraising to their payrolls. It’s like building a fancy new library, filling the library with books, magazines and periodicals, and then not hiring any librarians. A lot of clients come to us asking our opinion about what database they should choose. And the first question I ask is how are they leveraging the current data they already have? The better investment may be to make sure you have the right people who can use data to solve your...

Tax Reform 2018: All Eyes on Mids

Several groups have done research to address the question of how changes in the tax code for 2018 might impact charitable giving. Much of this research has focused on the fact that only 30% of Americans itemize their deductions. Some have gone further to include demographic splits by wealth or region. At Analytical Ones, we believe the most predictive measure of future donor behavior is how they have given in the past. This perspective informs all our statistical models, analytics reporting, and strategy recommendations. A national survey co-sponsored by  Donlon Agency and Analytical Ones found that in the case of 2018 tax reform, segmenting donors  by their past giving levels reveals an important piece of the story. While less than a third of Americans itemize their deductions, our most valuable charitable donors itemize at a much higher rate. As many as 75% of donors who give $1,000 or more to charity each year itemize their charitable deductions. General donors (Under $1k annual) and Major donors ($10k+) are mostly immune to the changes to the tax code for 2018.  However, nine percent of Mid-level donors ($1,000 to $9,999 annual) report that they will have a significant decrease in giving next year in response to the change. Because this group typically makes up a large portion of revenue for non-profits, it is more important than ever to have a specific strategy to cultivate and upgrade mid-level donors. See the infographic below for more...

Don’t Just Be Smart – Be Emotionally Intelligent

This is the final in a series of blogs on 2018 trends. You can read about the first four trends here. The last trend for 2018 that Forbes highlights is: Don’t just be smart. Be emotionally intelligent. As someone who has managed a number of analysts, I totally understand this one. Throughout my career I have worked with some super smart analysts. Brilliant analysts. Analysts who could write code in circles around me 24/7. However, what I found is that “smart” is only minor variable in the algorithm of success. If an analyst didn’t possess a high co-efficient of emotional intelligence, they were usually a disappointment. I’m sure you’ve seen the same thing at the places where you have worked. Sometimes the “best” candidate fails because they can’t connect with others or don’t fit the organization’s culture. I like the way Peter Drucker talked about this, “Culture eats strategy for breakfast.” Ultimately, we are all in the people business. And it’s best for you to fill your organization with people that have emotional intelligence. Because reading between the lines is a critical component of your organization’s...

Success Stories

This is the first post in a 3-part guest series by Steven Screen of The Better Fundraising Company. If you’ve ever had Analytical Ones analyze your nonprofit’s donor data, you’ve heard them say this: “Every donor data file has a story to tell.” I always thought that was a powerful idea: that your donors as a group, through their giving and their lack of giving, are telling your organization a story. And as a fundraiser, I know that the story your donors are telling is mostly a response to the story that your organization is telling. Right? Your organization sends out your donor communications – the tactics and content that make up your organization’s story – and your donors either give or they don’t. The better your story, the more money you raise. So I got to wondering; was there a certain kind of story that lead to fundraising success? For the donor files that Analytical Ones showed were doing really well, were there any commonalities in the “stories” those organizations were telling their donors? And I work mostly with small- to medium- nonprofits, so I was specifically curious about the ‘success stories’ for organizations with three characteristics: 1. Organizations without a built-in advantage. For instance, if you’re the national organization that works on a common type of cancer, thousands of people are diagnosed each day. Those new patients, and their friends and families, are immediately potential donors for your organization. That’s a built-in advantage that most nonprofits don’t have. 2. Organizations that weren’t too big. I didn’t precisely define a number, but we all know that there are...

Response Rate Testing and Statistical Significance

One of the more straightforward analyses we often do at Analytical Ones is to compare the results of a direct mail test to identify whether the differences are statistically significant. Though this is a straightforward analysis, there are a lot to these tests. So, let me try to clarify a couple of things. Generally, we are testing to determine whether the differences in either the response rates or average gift sizes are statically significant. Each of these tests are very different and require different data sets and tests. Let’s take the easy one first, testing the statistical difference in response rates. Because there are just two outcomes for response – yes, the donor responded, denoted by a value of “1”, or no, the donor did not respond, denoted by a value of “0” – you can use summary statistics (averages) for this test. We recommend using a Z test. All you need for testing response rates are: 1. Number of mailed in the control 2. Number of responses in the control 3. Number of mailed in the test 4. Number of responses in the test You also need to know the level of test confidence you are comfortable. Confidence levels are standards. Let me try to explain this. Typically, in Z tests, analysts use one of three confidence levels: 1. 99% 2. 95% 3. 90% If your results are significant at the 99% confidence level, it means that if you repeated the test 100 times, the result will be statistically significant 99 in 100 times. That’s a very high level of confidence. Conversely, if your results are significant at...