David Shepard Associates Blog

In our “The New Direct Marketing” course, there’s a slide with the following text:
 
The Database Marketing Business Model 

“Acquire new customers, almost always at a loss, in the expectation that the present value of their future contributions will exceed their acquisition costs, and, in addition, manage new customer acquisition expenditures consistent with fiscal sales and net income objectives.”
 
And, until the internet bubble burst, we would joke that the eCommerce Business Model looked like this.
 
The eCommerce Business Model 

“Acquire new customers at a loss, always, in the exception that no one cares, provided you spend enough on new customer acquisition, so that the possibility of making a profit, by accident, is minimized.”
 
A long time ago, this business model made sense, at least from a stock valuation perceptive.  The more you spent on customer acquisition, the more customers you would have — no argument there — so that when the big shakeout came, your company would be left standing – while the competition slipped away into bankruptcy.

Remember when they said that if you were showing a profit, you were not investing enough in the business? 

Well, those days are long gone. The question every dot-com company President is asked on CNN these days is when will your company turn a profit, and an answer of more than another four quarters is frowned upon.
 
So now all dot.com e-tailors are direct marketers or retailers without a store, which is another way to define a database marketer.  The implication for these new players, let’s call them edirect marketers is that if they are to survive they need to become fairly sophisticated direct marketers, and they need to do so fairly quickly. Which means exactly what? 

For starters, it means that they must learn to measure (or at least estimate) new customer acquisition costs by acquisition source, just like any other direct marketer must do in order to grow their business.
 
Now, admittedly, in a multi-media environment, when a company is running Web advertising, TV, radio, print and direct mail simultaneously in multiple markets, measuring the effectiveness of each is easier said then done. But, there’s no choice. Unless you understand the effectiveness of alternative acquisition sources, or acquisition strategies, you don’t have the ability to intelligently manage the allocation of your acquisition budget. 

What’s more in addition to measuring cost per new customer, it’s equally important to be able to measure the value of the customers you are acquiring. And, as suggested above unless the average present value of the average customer exceeds the cost of acquiring the average customer, the business is inherently unprofitable.
 
And here’ the rub, for new edirect marketers it’s exceptionally hard to measure and project the value of customers by acquisition source, in fact it’s exceptionally hard just to measure and project the value of the average customer –period—even without regard to source. 

Why is it so hard for e-direct marketers to measure customer value, or stated more accurately why is it especially hard for edirect marketers to measure and project customer value? Or, stated still another way, why is it harder for them, then for a traditional direct marketer? 

If you’ve been reading carefully you may have noticed that I’ve always linked measurement with prediction. Why?

Because in order to use the notion of lifetime value you need to be able to predict future behavior based on measurable behavior to date. In truth, it should be no harder for an edirect marketer to measure customer behavior, measured at any level, average, by source code, by campaign, or by enrollment group, then it is for a traditional direct marketer.

They just don’t know, as far as I can tell from talking to the internet marketers that attend our seminars, that unless you’re tracking customer behavior, multiple ways, you have no way of knowing the real profitability of the business you’re managing.  Nor, do you have any way testing and measuring the effects of alternative marketing strategies. 

What’s even harder and truly different for edirect marketers is the inherent lack of loyalty of the e-customer. Amazon.com aside, customer loyalty among Internet users is an oxymoron. Part of the fun of being an Internet shopper is finding that new site, it’s about surfing the Web and just because somebody stops by and buys something from you once, or even every once in while, does not make that buyer a customer, or a customer with a significant predictable lifetime value. 

It’s the traditional catalog one-time buyer problem, but magnified multiple times. 

Because of the inherent disloyalty of the Internet shopper it’s particularly important that the edirect marketer go out of his or her way to develop the beginnings of customer loyalty, or in even more traditional dm terms repeat purchase. 

What does this mean, in concrete actionable steps?

It means that a nice welcome back on your site, is mandatory but certainly not sufficient. It’s expected, therefore like an expected increase in interest rates, the market has probably discounted it. It means that routine e-mail messages, Facebook updates and Tweets probably won’t be sufficient to maintain customer loyalty, because everyone does it, and does it about the same way.

It means that customer cultivation will require using a combination of new and old direct marketing tools including direct mail, contests, events, reward programs, and customized Web sites for repeat customers.  It certainly, most emphatically means, that you just can’t sit back and wait for that “one time visitor/buyer” to come back on their own.
 

If you’re a marketer who uses or commissions regression models you need to understand the topic of non-linearity, what is it, why is it important, how it could improve your models, and why it doesn’t happen automatically. This article will address all of these issues.If you’ve built or used regression models to predict response or sales you know that a regression equation looks like this: 

Y = a +b1*X1 + b2*X2 + b3*X3…bn*Xn 

In this equation Y is the “thing” you’re trying to predict (the dependent variable)  and the X’s represent the “things” (independent variables) you know about your customers or prospects that allow you to make the predictions. Typical independent variables include performance indicators such as recency, frequency and dollar sales; demographics such as age, and income, and promotion history, such as the number of times called, etc. 

The” b’s” are called regression coefficients and you can think of them as weights assigned to each variable in the model, the assignment is generated by a regression program. The bn*Xn notation simply means that there could be up to some number (n) of variables in the model. The “a” is a constant that we can skip over for now.
 
The job of the statistician, working with a particular dataset, such as the results of a past promotion, is to discover which independent variables have a significant effect on the dependent variable and then feed this information to the regression program which will produce the regression equation.
 
One of the keys to a “good” long lasting model is to find the right set of predictive variables given the hundreds if not thousands of potential predictors from which to select.
But, in addition to finding the right variables it’s important to determine if the relationship between a predictor variable such as AGE and the a dependent variable such as SALES is best described by a simple straight line relationship, or whether some other “non-linear” relationship makes for a better, more accurate prediction.
 
When a non-linear relationship exists, it’s the job of the modeler to try different transformations of the data to determine the best fit. You as a user can tell if this has been done in one of your models if you see something like this: 

Sales = a +b1*Log of Recency +b2*Square Root of Prior Sales  

What this equation tells you is that the modeler determined that that relationship between Sales and Recency is best described by replacing Recency (number of months since the last purchase) by the log of Recency, and that the relationship between Sales and Prior Sales is best described by replacing Prior Sales by the Square Root of Prior Sales. 

Exhibits 1 and 2 show how the log transformation works to straighten the relationship between Sales and Recency.. Exhibit 1 is a plot of Sales against Months Since Last Purchase, Exhibit 2 is a plot of Sales against the Log of the Months Since Last Purchase. The Log transformations straightens the data and results in a better fit as indicated by the R Squared value of 1 versus an R Squared value of .86 for the original or untransformed data.
 
Exhibit 1

Exhibit 2

If nothing else, the above equation (with transformations) certainly looks more impressive than the equation below, without the data transformations. 

Sales = a +b1*Recency + b2*Prior Sales 

But apart from looking impressive, the real question is: does finding the right shape of a relationship, correcting for non-linearity, or straightening, three different ways to say the same thing, really make a difference?
 
To answer this question we created two data sets. Each data set has 400 observations representing 400 customers, each of whom responded to a mailing and purchased some amount of product. As is customary, the first data set will be used to build the model the second to test or validate the model.
 
But, to make sure that we could prove our point we cheated.  Instead of searching for variables that had a non-linear relationship with sales, and developing an equation, we started with the correct model! 

In the Correct Model each customer’s sales is determined by this formula 
Sales = 75 –30 times the log of the  number of days since last purchase + 5 times the square root of Prior Orders +.5 times the exponential value of Prior Sales/million + 6  if age is greater than 45 + a random error that ranges between –50 and +50.
 
To determine the effect of correcting for non-linearity we simply ran the data through an Excel spreadsheet and had the program calculate a regression model, using the four variables (Recency, Orders, Prior Sales and Age) but with no attempt to incorporate their known non-linear relationships.
 
The program produced the following equation.
 
Sales = 64 – .58*Recency + .20*Orders +2.61*Prior Sales + .085*Age 

The Model had an R Squared of 33%. (In other words the simple model explained 33% of the variation in Sales.
 
Then we ran the data through the program again, this time substituting the correct form of the relationship for the original uncorrected data.
 
The same program produced the following equation. 

Sales = 84 –29.29*the log of the number of days since last purchase + 4.39*square root of Prior Orders +.48*the exponential value of Prior Sales/million + 4.05 if age is greater than 45 

The Model’s R Squared was 79%. (Even though we knew the correct form of the only four variables affecting the model, the model was not perfect because of the random error. 
So, it would appear that knowing the correct shape of the relationship between independent variables and the dependent variable makes a huge difference—at least to a statistician, but how about the difference it makes to a direct marketer. 

To answer this question we applied both models to our second data set of 400 different customers and produced the two decile analyses shown in Tables 1 and 2. 

As you can see by comparing Tables 1 and 2, the Correct Model results in a greater spread and a closer fit and is therefore the better model. But don’t draw the wrong conclusions from this example. In the real world the search for the correct relationship is not done just to get a better fit. In fact that is a relatively weak reason for going through all the work that it takes to find and correct for non-linearity. In the real world, many relationships are so non-linear that these important variables will not appear in a regression model at all… unless their non-linearity is first identified and then corrected for. 

Why is that? Because the regression programs are expecting linear relationships and a relationship that is in fact very strong, but very non-linear may be missed entirely by an analyst just running data through a regression program. (And, most importantly, the regression programs don’t do this automatically by themselves, this work has to be done by an analyst working with the data.) 

So, how does the analyst discover these non-linear relationships? By using a number of graphical techniques and/or CHAID.  The lesson for the direct marketer is that these non-linear relationships exist. We find one or two in nearly every model we do. If you don’t see them in yours, that does not mean they are not there, they just may have been overlooked and your models could be significantly improved. 

One last note, correcting for non-linearity is a central part of what statisticians call Exploratory Data Analysis (EDA). This practice is recommended even when the modeling technique does not assume that the relationships it’s being asked to analyze are linear. For example, artificial neural net solutions do not assume linear relationships.

Nevertheless, straightening complicated non-linear relationships prior to submission of data to the neural net is a commonly recommended procedure. It makes it easier for the Net to arrive at a reliable solution, and there’s nothing wrong with that.   

One of the ways in which you can improve your modeling results is to look for segments within your customer database that have different relationships to potentially predictive variables such as  Recency, Frequency, Monetary Value and Products purchased.

The trick is to determine if the strength of the relationship is equally strong across all segments, or whether the strength of the relationship differs from segment to segment.

For example, lets suppose you believe that your sales are correlated with two variables, will call them variables X1 and X2. What you might do is ask your statistician to draw a sample of data, create a Scatter Diagram so that you can see the relationship and calculate the Correlation Coefficient so that you can quantify the relationship as well as visualize it. We did that for a dataset we created for this article.

So far so good. Your hunch was correct your sales (Y) are positively correlated with X1 and also with X2. And while the correlation statistics are not great (.7 to .9) they are not weak (.1 to .3) either. They are moderate, .45 and .64. (The absolute value of a correlation coefficient can not be less than 0 or more than 1.)

Now that you’ve discovered two variables that are related to sales you would want to build a two variable regression model of the form Y = A +b1X1 + b2X2.  Using the same data set that produced the above results you have your statistician run the data through the a Regression procedure and produce the following results.

Y = 31.5 + 9.2*X1 + 6.7*X2 with an R-Squared of 59%.

Not Bad. Our simple two variable example produced an equation or a model which explains 59% of the difference we see among our customers’ behavior.

Suppose it now dawned upon you that while sales of your customers were correlated with variables X1 and X2, your customer file was really made up of three distinct segments: that you call: Young, Middle and Old and that you suspect that the relationship between sales and X1 and X2 might not be the same for each segment.

What could you do?
Since you’ve identified three segments you could use this information in your model. How? Have your statistician create two new “Dummy Variables” and code your young customers DY and your middle aged customers DM. You don’t need to code your old customers DO, because if they are not Young (Coded DY) or Middle (coded DM) then they must be in the segment called Old. Your statistician runs the data through the regression program again and arrives at the following equation:

Y = 428 + 8.4*X1 + 7.6*X2 – 539.5*DY – 804.4*DM and R-Squared goes to 86%.

Your hunch was correct each segment has a different relationship with X1 and X2. Your statistician now suggests that the results could be improved even more if we looked for the interaction between the segment identifiers and the individual variables themselves. You have no idea what this means but it sounds good so you try it and this is what you come up with.

Y = 4 + 7*X1 + 13*X2 –1*DY +1*DM -2*DY*X1 –5*DY*X2 +4*DM*X1 -10*DM*X2 and R-Squared =100%

What happened? What happened is that we discovered, in our made up example, that each segment behaves differently with regard to variables X1 and X2. And, that by understanding the relationship between X1 and X2 and sales in each segment we were able to build, in this artificial case, a perfect model! Of course in real life you will never be able to build anything close to a perfect model.

But the lesson to be learned is that if you suspect that different demographic or lifestyle or attitudinal segments might display different relationships with regard to your key performance variables, try building separate models for each segment.

Building separate models, as opposed to building one equation with all dummy and interaction variables, as we did above, is a simpler solution and one that is more likely to be understood and less prone to implementation errors.

If you’ve been following our running commentary over the last year or so you know that I’ve become somewhat obsessed with the issue multiple models. For those that haven’t been paying close attention, meaning just about anyone with a life, here’s the problem. You want to build a model to predict some outcome, a response to a cross-sell mailing, attrition, lifetime value…whatever.

You’d like to come away with one simple to use equation that can be applied to your entire customer file. But you intuitively know that this might not be possible or at least easy. For example, Tenure, how long someone has been your customer, is certainly an important variable, but how does it relate to the other variables in you model?  Consider this, demographics may be important predictors, for customers that have been on your file for just a few months, but will they be important predictors, or as important predictors, for customers that have been on the file for years, and about whom you have lots of transaction information/variables?

Read the rest of this entry »

Let’s talk about Lifetime Value, why not, everyone else does.  Lifetime Value even made it into a recent publication where someone replayed the “old saw” about the Manager who looked at his customer leaving the store and “envisioned him or her (I don’t know why he couldn’t tell the difference) walking out with $50,000 worth of groceries, which is what [he] expected the customer to buy during the life of the relationship”. The author used the above example to help describe “what the term CRM really means.”

Nice idea, great concept, now what? Was that customer really worth $50,000, and if so, what should the store manager be doing about it…perhaps he should build a database so that the surly teenager at the checkout counter doesn’t completely ignore him after he passes his loyalty card through a reader that has kept track of his lifetime-purchases-to-date and has predicted that unless he comes down with mad cow disease, from the meat he just purchased, he will come back on Saturday for a six pack of beer and some toilet paper…which helps explains why the beer and the toilet paper are kept in close proximity. (Readers not familiar with the groundbreaking beer & toilet paper study (or urban myth), from the 1990’s by market researches using three-dimensional cubes, should look it up.) Read the rest of this entry »

Want to make your head hurt?  Think about this.  You have X million customers, N number of products and Y dollars to spend over some promotion period, say three to six months, and at least three contact channels: direct mail, telemarketing and e-mail.

The smart reply is that why would you want to make your head hurt. So, skip to another article or go for coffee.  That’s kind of like what a lot of direct marketers do. They try to keep it simple. This month, they say, I’m going to promote product A, next month I’ll promote product B and so on; I’m going to use Direct Mail or telemarketing; my fiscal plan calls for so many pieces; I’ll sort my customer file on the basis of RFM or some regression or artificial intelligence model, and I’ll mail/call as deeply as my budget calls for.  That’s it. Next month I’ll do it all over again.

Read the rest of this entry »

I recently saw a pie chart that indicated companies were spending 50% of their direct marketing dollars on Customer Retention and the 50% on Customer Acquisition. Last year,  Customer Acquisition accounted for 60% of spending and only 40% on Retention.

Assuming that these numbers were true, in the sense that they were an accurate barometer of what’s happening in the direct marketing world (whatever that means) the implication would be very disturbing. While there is nothing wrong per se about spending on customer retention, you can’t grow a business that way – certainly not in terms of the number of customers, and almost certainly not in terms of revenues and eventually not in terms of bottom line profits.

A related finding was that 60% of the respondents reported that they would be spending more (across all channels) on direct marketing activities next year while the balance would be spending about the same.

Read the rest of this entry »

In our last publication on this topic we argued that segmentations based on a simultaneous combination of survey generated demographic, behavior and attitudinal data tend to produce a small number of interesting, fun to name but often fuzzy segments. Fuzzy in the sense that while the segments differ with respect to the average value of important segment defining variables, the spread around the averages can be significant.

For example, Segment 1 may include older married couples, who are frequent buyers, and who prefer to shop at retail; as opposed to members of an adjacent segment, Segment 2, who are also, on average, older married couples, also frequent shoppers, but who prefer to shop by mail.

Now, the problem is that included in Segment 1 will be some older singles and some younger married couples, who got into that segment because of their shopping patterns and their channel preferences closely matched those of members of segment 1.  This result is what we mean by a fuzzy segment.

So, what’s the problem with fuzzy segments?

Read the rest of this entry »