Are My Response Rates Statistically Significant?

Are My Response Rates Statistically Significant?

I’m taking a different approach in today’s blog to discuss a problem I continue to run into when I am working with clients. And that is testing whether the response rates in a direct mail test are meaningful. Now response rates are relatively uncomplicated to test. Anyone can do it. That’s because there are only two possible outcomes: Response or nonresponse. Here’s an online testing tool for you to use. After you click on the link, you will need to enter your data. The “base sizes” are the number of pieces mailed in each of your tests. The “proportions” are the number of responses for each test divided by the number of mailed. You can choose your confidence level (we recommend 90%), and then click calculate. Voila! I suggest using a 90% confidence level because it is a high standard. That means if you have this difference in 100 tests, in 90 of those tests the difference will be real. The “Results” box will state either “Significant” or “NOT Significant.” You don’t need to understand statistics to use this tool! Another helpful metric to know is your average gift size. Now, testing average gift size is more work to calculate, because the values you are testing can range from $0.01 to $1 million or more. That means you can’t use summary statistics (averages) to calculate significance. Instead, you need the entire gift distribution. Plus, you will need a more robust software tool than Excel to do this. We use SPSS. We suggest that you talk with your analytics people to have them help you through testing average gift size...
People First, CRM Second

People First, CRM Second

A festering trend of the past decade is many of our clients have migrated from micro-computer based fundraising databases to CRM (Constituent Relationship Management) cloud-based databases. And for 10-years we continue to see too many of these organizations’ ability to leverage their data for insights decline. Rather than the CRM serving the nonprofit, it seems like the nonprofit now must serve the CRM. I still find this so ironic and frustrating. Databases don’t solve problems – people solve problems. I know it’s not any one CRM or any one client. I think most every organization underestimates the commitment of undertaking migrating to a CRM. They also underestimate the cultural change required to harness the power of CRM – which includes adding highly trained (and highly salaried) fundraising professionals to run it. Most organizations understand the need for CRM and are willing to swallow the expensive of the database infrastructure, but in my experience, too few organizations have been willing to cover the cost to train their people adequately or add CRM professionals who know fundraising to their payrolls. It’s like building a fancy new library, filling the library with books, magazines and periodicals, and then not hiring any librarians. A lot of clients come to us asking our opinion about what database they should choose. And the first question I ask is how are they leveraging the current data they already have? The better investment may be to make sure you have the right people who can use data to solve your...
Retiring the Term “Donor Engagement”

Retiring the Term “Donor Engagement”

I woke up this morning with the need to cease using the term donor engagement. It needs to be laid to rest. This is a serious personal drag because it now means I need to remove all those blogs we have written on the topic. But nonetheless, I am retiring the word from my vocabulary. I think you should too. Here’s why:  It’s a top down word. You’d never here a donor say: “I’m going to engage with my favorite nonprofit organization today.” Donor engagement is an institutional term. And if we are truly to be donor-central, then we must retire the words and ways that don’t support that goal. The term is a euphemism. When development people say donor engagement they really mean is “let’s engage the donor with the hope that they give us more money.” Let’s be honest: donor engagement has become synonymous with donor cultivation. Language is important. I believe donors want their nonprofits to sound like real people who are working in great movements to change the world for the better. I think we need a new term, and it needs to be honest. What do you...
Postage Going Up

Postage Going Up

On January 27, first class postage is going up to 55 cents, a 5-cent increase. This is the largest increase in almost 30-years. As this article states: “The Postal Service lost $3.9 billion in 2018, attributing the losses to drops in mail volume and the costs of pensions and health care. It marked the 12th year in a row the agency reported a loss despite growth in package shipping.” $3.9 billion. That’s a big number. To put this in context the USPS lost almost a half million dollars every hour, 24-hours a day. That’s just not sustainable. We have been fortunate in this country to have had such a great postal system. It has been fast, accurate and affordable. Nonprofits have been doubly blessed by the US Postal Services’ postal subsidy for nonprofit organizations. But that is changing before our eyes. Delivery of nonprofit mail is getting slower and slower and impossible to predict. Nonprofit mail is now delivered in batches. That means when your appeal does finally get delivered, it’s hitting your donors’ mailbox at the same time as every other nonprofit’s mail. This fall I received 8 appeals on the same day. So now not only are your appeals unpredictably slow on delivery, they are facing greater competition. I can’t help but think that the nonprofit postal subsidies will likely end soon. Here’s what I recommend while there is still time: Analyze the ROI of your donor segments this fall and test first class postage in top performing segments. With access to reliable nonprofit postage, first class postage is an extravagance. However, you are paying a lot...
Improving the Acquisition Experience

Improving the Acquisition Experience

Allegories for Unintegrated Direct Mail Acquisition Testing Activities: • Tailor fitting a funeral suit • Climbing a tall ladder on a building that’s on fire • Planting orange trees above the freeze line Don’t get me wrong. I love direct mail. But sometimes, context matters. Spending hours and hours coming up with new ideas on how to beat your direct mail acquisition control kit is probably not the best use of your time right now – especially if you aren’t integrating your direct mail with other media campaigns. Realistically, moving your direct mail acquisition response rate from 0.36% to 0.56% isn’t going to solve all your problems. New donor acquisition is a long-term problem, and you need an integrated solution. I suggest, this time, you stick with your tried and true acquisition kit. Reinvest the time and resources you’d otherwise spend in direct mail testing protocols into brainstorming ways to engage a new audience that’s completely different. I know from a career standpoint that seems bold. That everyone knows you can’t get fired for doing the “safe” thing. But as you can tell by looking at our stock market, there are no safe things right now. These new layers to an old problem require innovative solutions. So be bold. Test something that’s never been tried – and then tell us about...