David Shepard Associates Blog

In a previous post I suggested that modelers could improve their results by splitting
their datasets according to some critically important variable, such as Tenure (the length of time a customer has been on the file) and then build separate models for each major segment.

The argument being that it is intuitive that the usual set of modeling suspects (Recency, Frequency, Monetary Value, Products Purchased, Source and the whole set of Demographic Variables) will display different relationships with Response or Sales, depending upon the Tenure Segment, and that just adding Tenure as a variable, without taking interactions into account, isn’t sufficient to capture the full effect of this variable.

As if this isn’t complicated enough, I came across an article that questioned fundamental direct marketing beliefs, including the belief that there is a strong positive relationship between customer lifetime and profitability in a non-contractual relationship. In other words, they think that direct marketers think that customers that kind of hang around a long time, buying every once in a while, are profitable and every effort should be made to enhance the relationship between buyer and seller.

Of course, direct marketers who have looked closely at the data know that the costs of servicing infrequent buyers may indeed exceed the margins they yield; and the authors discovered for themselves that the simple relationship between lifetime months on file and lifetime profits is relatively weak (r = about .2 for the two groups studied).

What I did find interesting and potentially actionable was that they could divide a
significant number of customers into four meaningful groups:(Some 9000 households were studied over a three-year period. The households were correctly split into two cohort groups, January and February starters.)

Segment 1. Those that had relatively Long Active Lives and High Lifetime Revenue

Segment 2. Those that had relatively Long Active Lives and Low Lifetime Revenue

Segment 3. Those that had relatively Short Active Lives and High Lifetime Revenue.

Segment 4. Those that had relatively Short Active Lives and Low Lifetime Revenue.

The Graph below indicates that customers in Segments 1 and 3 kind of look alike, behave in a similar fashion, over their first 12 months and then begin to separate over time. No doubt that this is true, the operable question is can this disparity be predicted, and predicted early enough in customer’s life so that corrective action taken be taken.

The argument is that simple RFM analyses will miss this phenomena, and that database marketers, as a consequence of their not understanding that their database consists of these segments, will overspend on the Short Life-High Revenue segment, before traditional RFM analysis will depress mailings to this segment.

So, the key question for marketers is, if this effect is widespread — if there really are customers that come in for a short while, buy a lot and then leave — can they be detected? Will modeling Tenure Segments, as suggest above, and in last month’s article capture this effect.  Probably not, at least not by itself. What might work is a Principal Component Analysis of the available purchase behavior data over the last six months.

This approach might discern either a trend in dollars spent, or a trend in the particular products purchased that would indicate that the customer was displaying a pattern associated with customers that buy heavily for a short while and then switch to someone else – for reasons we can only speculate about.

If you’re a marketer who uses or commissions regression models you need to understand the topic of non-linearity, what is it, why is it important, how it could improve your models, and why it doesn’t happen automatically. This article will address all of these issues.If you’ve built or used regression models to predict response or sales you know that a regression equation looks like this: 

Y = a +b1*X1 + b2*X2 + b3*X3…bn*Xn 

In this equation Y is the “thing” you’re trying to predict (the dependent variable)  and the X’s represent the “things” (independent variables) you know about your customers or prospects that allow you to make the predictions. Typical independent variables include performance indicators such as recency, frequency and dollar sales; demographics such as age, and income, and promotion history, such as the number of times called, etc. 

The” b’s” are called regression coefficients and you can think of them as weights assigned to each variable in the model, the assignment is generated by a regression program. The bn*Xn notation simply means that there could be up to some number (n) of variables in the model. The “a” is a constant that we can skip over for now.
 
The job of the statistician, working with a particular dataset, such as the results of a past promotion, is to discover which independent variables have a significant effect on the dependent variable and then feed this information to the regression program which will produce the regression equation.
 
One of the keys to a “good” long lasting model is to find the right set of predictive variables given the hundreds if not thousands of potential predictors from which to select.
But, in addition to finding the right variables it’s important to determine if the relationship between a predictor variable such as AGE and the a dependent variable such as SALES is best described by a simple straight line relationship, or whether some other “non-linear” relationship makes for a better, more accurate prediction.
 
When a non-linear relationship exists, it’s the job of the modeler to try different transformations of the data to determine the best fit. You as a user can tell if this has been done in one of your models if you see something like this: 

Sales = a +b1*Log of Recency +b2*Square Root of Prior Sales  

What this equation tells you is that the modeler determined that that relationship between Sales and Recency is best described by replacing Recency (number of months since the last purchase) by the log of Recency, and that the relationship between Sales and Prior Sales is best described by replacing Prior Sales by the Square Root of Prior Sales. 

Exhibits 1 and 2 show how the log transformation works to straighten the relationship between Sales and Recency.. Exhibit 1 is a plot of Sales against Months Since Last Purchase, Exhibit 2 is a plot of Sales against the Log of the Months Since Last Purchase. The Log transformations straightens the data and results in a better fit as indicated by the R Squared value of 1 versus an R Squared value of .86 for the original or untransformed data.
 
Exhibit 1

Exhibit 2

If nothing else, the above equation (with transformations) certainly looks more impressive than the equation below, without the data transformations. 

Sales = a +b1*Recency + b2*Prior Sales 

But apart from looking impressive, the real question is: does finding the right shape of a relationship, correcting for non-linearity, or straightening, three different ways to say the same thing, really make a difference?
 
To answer this question we created two data sets. Each data set has 400 observations representing 400 customers, each of whom responded to a mailing and purchased some amount of product. As is customary, the first data set will be used to build the model the second to test or validate the model.
 
But, to make sure that we could prove our point we cheated.  Instead of searching for variables that had a non-linear relationship with sales, and developing an equation, we started with the correct model! 

In the Correct Model each customer’s sales is determined by this formula 
Sales = 75 –30 times the log of the  number of days since last purchase + 5 times the square root of Prior Orders +.5 times the exponential value of Prior Sales/million + 6  if age is greater than 45 + a random error that ranges between –50 and +50.
 
To determine the effect of correcting for non-linearity we simply ran the data through an Excel spreadsheet and had the program calculate a regression model, using the four variables (Recency, Orders, Prior Sales and Age) but with no attempt to incorporate their known non-linear relationships.
 
The program produced the following equation.
 
Sales = 64 – .58*Recency + .20*Orders +2.61*Prior Sales + .085*Age 

The Model had an R Squared of 33%. (In other words the simple model explained 33% of the variation in Sales.
 
Then we ran the data through the program again, this time substituting the correct form of the relationship for the original uncorrected data.
 
The same program produced the following equation. 

Sales = 84 –29.29*the log of the number of days since last purchase + 4.39*square root of Prior Orders +.48*the exponential value of Prior Sales/million + 4.05 if age is greater than 45 

The Model’s R Squared was 79%. (Even though we knew the correct form of the only four variables affecting the model, the model was not perfect because of the random error. 
So, it would appear that knowing the correct shape of the relationship between independent variables and the dependent variable makes a huge difference—at least to a statistician, but how about the difference it makes to a direct marketer. 

To answer this question we applied both models to our second data set of 400 different customers and produced the two decile analyses shown in Tables 1 and 2. 

As you can see by comparing Tables 1 and 2, the Correct Model results in a greater spread and a closer fit and is therefore the better model. But don’t draw the wrong conclusions from this example. In the real world the search for the correct relationship is not done just to get a better fit. In fact that is a relatively weak reason for going through all the work that it takes to find and correct for non-linearity. In the real world, many relationships are so non-linear that these important variables will not appear in a regression model at all… unless their non-linearity is first identified and then corrected for. 

Why is that? Because the regression programs are expecting linear relationships and a relationship that is in fact very strong, but very non-linear may be missed entirely by an analyst just running data through a regression program. (And, most importantly, the regression programs don’t do this automatically by themselves, this work has to be done by an analyst working with the data.) 

So, how does the analyst discover these non-linear relationships? By using a number of graphical techniques and/or CHAID.  The lesson for the direct marketer is that these non-linear relationships exist. We find one or two in nearly every model we do. If you don’t see them in yours, that does not mean they are not there, they just may have been overlooked and your models could be significantly improved. 

One last note, correcting for non-linearity is a central part of what statisticians call Exploratory Data Analysis (EDA). This practice is recommended even when the modeling technique does not assume that the relationships it’s being asked to analyze are linear. For example, artificial neural net solutions do not assume linear relationships.

Nevertheless, straightening complicated non-linear relationships prior to submission of data to the neural net is a commonly recommended procedure. It makes it easier for the Net to arrive at a reliable solution, and there’s nothing wrong with that.   

One of the ways in which you can improve your modeling results is to look for segments within your customer database that have different relationships to potentially predictive variables such as  Recency, Frequency, Monetary Value and Products purchased.

The trick is to determine if the strength of the relationship is equally strong across all segments, or whether the strength of the relationship differs from segment to segment.

For example, lets suppose you believe that your sales are correlated with two variables, will call them variables X1 and X2. What you might do is ask your statistician to draw a sample of data, create a Scatter Diagram so that you can see the relationship and calculate the Correlation Coefficient so that you can quantify the relationship as well as visualize it. We did that for a dataset we created for this article.

So far so good. Your hunch was correct your sales (Y) are positively correlated with X1 and also with X2. And while the correlation statistics are not great (.7 to .9) they are not weak (.1 to .3) either. They are moderate, .45 and .64. (The absolute value of a correlation coefficient can not be less than 0 or more than 1.)

Now that you’ve discovered two variables that are related to sales you would want to build a two variable regression model of the form Y = A +b1X1 + b2X2.  Using the same data set that produced the above results you have your statistician run the data through the a Regression procedure and produce the following results.

Y = 31.5 + 9.2*X1 + 6.7*X2 with an R-Squared of 59%.

Not Bad. Our simple two variable example produced an equation or a model which explains 59% of the difference we see among our customers’ behavior.

Suppose it now dawned upon you that while sales of your customers were correlated with variables X1 and X2, your customer file was really made up of three distinct segments: that you call: Young, Middle and Old and that you suspect that the relationship between sales and X1 and X2 might not be the same for each segment.

What could you do?
Since you’ve identified three segments you could use this information in your model. How? Have your statistician create two new “Dummy Variables” and code your young customers DY and your middle aged customers DM. You don’t need to code your old customers DO, because if they are not Young (Coded DY) or Middle (coded DM) then they must be in the segment called Old. Your statistician runs the data through the regression program again and arrives at the following equation:

Y = 428 + 8.4*X1 + 7.6*X2 – 539.5*DY – 804.4*DM and R-Squared goes to 86%.

Your hunch was correct each segment has a different relationship with X1 and X2. Your statistician now suggests that the results could be improved even more if we looked for the interaction between the segment identifiers and the individual variables themselves. You have no idea what this means but it sounds good so you try it and this is what you come up with.

Y = 4 + 7*X1 + 13*X2 –1*DY +1*DM -2*DY*X1 –5*DY*X2 +4*DM*X1 -10*DM*X2 and R-Squared =100%

What happened? What happened is that we discovered, in our made up example, that each segment behaves differently with regard to variables X1 and X2. And, that by understanding the relationship between X1 and X2 and sales in each segment we were able to build, in this artificial case, a perfect model! Of course in real life you will never be able to build anything close to a perfect model.

But the lesson to be learned is that if you suspect that different demographic or lifestyle or attitudinal segments might display different relationships with regard to your key performance variables, try building separate models for each segment.

Building separate models, as opposed to building one equation with all dummy and interaction variables, as we did above, is a simpler solution and one that is more likely to be understood and less prone to implementation errors.

If you’ve been following our running commentary over the last year or so you know that I’ve become somewhat obsessed with the issue multiple models. For those that haven’t been paying close attention, meaning just about anyone with a life, here’s the problem. You want to build a model to predict some outcome, a response to a cross-sell mailing, attrition, lifetime value…whatever.

You’d like to come away with one simple to use equation that can be applied to your entire customer file. But you intuitively know that this might not be possible or at least easy. For example, Tenure, how long someone has been your customer, is certainly an important variable, but how does it relate to the other variables in you model?  Consider this, demographics may be important predictors, for customers that have been on your file for just a few months, but will they be important predictors, or as important predictors, for customers that have been on the file for years, and about whom you have lots of transaction information/variables?

(more…)

Want to make your head hurt?  Think about this.  You have X million customers, N number of products and Y dollars to spend over some promotion period, say three to six months, and at least three contact channels: direct mail, telemarketing and e-mail.

The smart reply is that why would you want to make your head hurt. So, skip to another article or go for coffee.  That’s kind of like what a lot of direct marketers do. They try to keep it simple. This month, they say, I’m going to promote product A, next month I’ll promote product B and so on; I’m going to use Direct Mail or telemarketing; my fiscal plan calls for so many pieces; I’ll sort my customer file on the basis of RFM or some regression or artificial intelligence model, and I’ll mail/call as deeply as my budget calls for.  That’s it. Next month I’ll do it all over again.

(more…)

I recently saw a pie chart that indicated companies were spending 50% of their direct marketing dollars on Customer Retention and the 50% on Customer Acquisition. Last year,  Customer Acquisition accounted for 60% of spending and only 40% on Retention.

Assuming that these numbers were true, in the sense that they were an accurate barometer of what’s happening in the direct marketing world (whatever that means) the implication would be very disturbing. While there is nothing wrong per se about spending on customer retention, you can’t grow a business that way – certainly not in terms of the number of customers, and almost certainly not in terms of revenues and eventually not in terms of bottom line profits.

A related finding was that 60% of the respondents reported that they would be spending more (across all channels) on direct marketing activities next year while the balance would be spending about the same.

(more…)

In our last publication on this topic we argued that segmentations based on a simultaneous combination of survey generated demographic, behavior and attitudinal data tend to produce a small number of interesting, fun to name but often fuzzy segments. Fuzzy in the sense that while the segments differ with respect to the average value of important segment defining variables, the spread around the averages can be significant.

For example, Segment 1 may include older married couples, who are frequent buyers, and who prefer to shop at retail; as opposed to members of an adjacent segment, Segment 2, who are also, on average, older married couples, also frequent shoppers, but who prefer to shop by mail.

Now, the problem is that included in Segment 1 will be some older singles and some younger married couples, who got into that segment because of their shopping patterns and their channel preferences closely matched those of members of segment 1.  This result is what we mean by a fuzzy segment.

So, what’s the problem with fuzzy segments?

(more…)

Part 4 of a Multi-Part Series

In our last article we discussed the issues relating to the choice of variables that go into a Cluster Analysis. The key points were: (1) choose  only those variables you want to segment around, i.e., don’t include variables that you think are irrelevant to your marketing strategy (2) standardize the variables so that scale (the size of the variables) does not become an unintended issue, $25,000 is not the same as $25m (3) consider giving more weight to some variables than others depending on your marketing objectives, and (4) use Principal Components Analysis to: (a) reduce the number of variables that will go into the solution, and(b) eliminate multi-co-linearity, the undesirable, from a modeling perspective, condition that arises because so many behavior and demographic variables are correlated with each other.

Now go run a Cluster Analysis on the data.

Not so fast.

More decisions have to be made and there’s no one right answer. 
(more…)