Why Proponents Of Klout Are Missing the Big Picture

Jay Baer wrote a blog post titled Why Critics of Klout Are Missing the Big Picture in which he argues that  “influence measures help business create order from chaos.” Baer goes on to write:

“What’s important is to recognize that more and more and more and more of our behaviors occur online and often with the social media realm. And if companies are going to succeed in a chaotic, real-time environment, they need some mechanism – even a flawed one – to triage promotion and reaction. So yeah, Klout isn’t perfect. But instead of rehashing the same old “look how screwed up their formula is” argument, let’s focus instead on how advanced metrics will enable companies to deliver highly specific interactions with customers based on perceived influence.”

My take: Baer makes a good and valid point. But I think Baer and I might disagree on what the “big picture” is.

Baer’s definition of big picture  seems to be “making sense of chaos.” My notion of the big picture is “making the right decisions.”

And, using my definition, what I see are marketers making questionable business decisions based on people’s Klout score.

The best example I can give you to demonstrate this is the bank that’s reserving the best spots in its parking lots for its customers with a high Klout score.

Let me state this is no uncertain terms, and aim it directly at the bank with which I do business:

If you reserve the best spots in your parking lot for some pimply-faced 25 year old (who spends too much time on Facebook and Twitter and has somehow managed to get himself a high Klout score) instead of for me, then I’m pulling my millions out of your bank.

If you think I’m kidding, try me. And I’ll also pull my kids’ accounts (they’re Gen Yers, btw — not little kids), too.  THEN you’ll learn who has INFLUENCE. And when dear-old Mom and Dad (who turns 80 this year!), ask me to take over the day-to-day management of their finances, their money is getting pulled out of your bank, as well. THEN AGAIN you’ll learn who has INFLUENCE.

All because you made the bad decision to reward one group of customers over another.

Bottom line: The purpose of a business metric isn’t just making sense out of chaos — it’s taking action. And unless your customer base is made up of just heavy social media users, then making decisions on what to do based on Klout scores may lead to sub-optimal decisions. 


More Likely To Purchase: Quantipulation In Action

How many times this week have you heard about some research study that found that one consumer segment is XX% more likely to purchase your products than another segment?

These studies and claims come out every day. And every one of them is a shining example of Quantipulation: The art and act of using unverifiable math and statistics to convince people of what you believe to be true.

The problem with these “more likely to purchase” claims is that they’re leading you to make bad marketing decisions.

For example, it’s popular these days to claim that Facebook fans are an important segment of your customer base because they’re “more likely to purchase” than other customers are. DDB (a very reputable advertising and marketing services firm) conducted a study last year and found that:

“Facebook users who like a brand’s page on the site are thirty-three percent more likely to buy a product, and 92 percent more likely to recommend a product to others. “Fan status is indicative of high purchase intent, especially when compared to any traditional form of advertising, and is an even greater predictor of advocacy with over 90% noting that being a fan has a positive impact on recommending a brand to friends,” said Catherine Lautier, Director of Business Intelligence at DDB.”

The implication of this is that: 1) If marketers can drive up their brands’ Facebook fan count, then more customers will become more likely to buy, and 2) Marketers should focus their marketing efforts on Facebook fans because of higher purchase likelihood.

But there are a few problems here:

1. What does “more likely to purchase” mean? If in a survey Customer A (Facebook fan) says he’s “very likely to purchase” and Customer B (non-Facebook fan) says he’s “somewhat likely to purchase”, what does this really tell you? How much more likely is “very likely” than “somewhat likely”? Isn’t timeframe important? Is that very likely to buy in the next 2 weeks or very likely to buy at some point in the future? Even if Customer B says “not likely”, does that mean we should give up on marketing to him? Really? People don’t change their opinions? After all, he’s already a customer — and isn’t the cost of acquisition 5x higher than the cost of retention?

2. The absolute numbers might not be compelling. In the DDB study, only 36% of Facebook fans said that they were very likely to purchase. Which means that 27% of non-Facebook fans were very likely to purchase (you do the math). Assume that your company has 10 million customers, of which 1 million are Facebook fans. That means you’ve got 360,00 Facebook fans who are very likely to purchase, and 2, 430,000 non-Facebook fans that are very likely to purchase. Which group do you want to market to?

3. Causation versus correlation. Do Facebook fans become “more likely to purchase” after becoming Facebook fans, or did the fact that they were already “more likely to purchase” lead them to become Facebook fans? Granted, their act of becoming a Facebook fan helps marketers better identify them out of the pack. But if — as the numbers above indicate — the differences in likelihood to purchase aren’t that compelling, then it’s simply not a very helpful segmentation tool.

Bottom line: Don’t be quantipulated into believing these “more likely to purchase” claims.

Quantipulation: ROI Versus Success

[This is a follow-up post to Quantipulation. I thought I could get away with just floating a few ideas out there, but some comments I’ve seen suggest that there’s a lot more to quantipulation than I wrote about, and those comments are correct.]

Quantipulation — the art and act of using unverifiable math and statistics to convince people of what you believe to be true — is commonplace in the marketing world, but perhaps nowhere more so than in the social media environment. Especially when it comes to everyone’s favorite topic: Social media ROI.

Whenever I use the term ROI in my reports, the editor where I work asks me to spell it out. As she rightly says, there may be people who aren’t familiar with the term. I don’t tell her this, but if you don’t know what ROI is, I don’t want you reading my reports.

There’s another reason why she’s right: There may be people who define ROI differently than I do. I won’t tell her this, either, but those people don’t deserve to read my reports.

ROI = return on investment. It doesn’t mean return on influence or any other “I” word you can dream up. And despite what some quantipulators would have us believe there’s only one formula for ROI: Financial return divided by financial investment. The only “variable” piece to the formula is the timeframe you use to quantify these variables.

That won’t stop some people from trying to redefine the formula, however.

The most egregious example comes from a firm called Digital Royalty. I won’t besmirch my blog by linking to the offending post. Instead, I’ll point you to Anna O’Brien’s brilliant (and very funny) critique of it.

Here’s another example of ROI quantipulation:

My bet is that tthe firm that put this chart together wanted to include other ROI components, but since it would have messed up their inverted hour glass figure, they decided to leave them out.

Then there’s attempt at redefining social media ROI:

This guy has decided that the ROI unit of measure should be “conversation”. He goes on to tell us that we can measure the “value” of conversation by looking at participation, engagement, influence, imagination, energy, and stickiness. But not increased revenue or decreased cost. Sweet.

There are (at least) two things going on with these attempts to redefine ROI. One is bad, the other is good. 

The bad: An annoying attempt to demonstrate thought leadership. Ugh. Not the way to do it. Anna O’Brien said it best in her blog post: “Random metric names and symbols is not an equation.” (Maybe she didn’t say it best, because it should be “are not an equation”).

There is a good aspect to what the ROI quantipulators are doing, however. They’re raising the very valid point that there are other measures of success beyond ROI. 

There’s a formula for that, too. The one I like is from Pat LaPointe who writes a blog called Marketing NPV. Pat’s formula says that success can be measured by dividing the value added by the resources used. And as this formula implies, “value” can take on the form of many of those measures that those other people wanted to use to calculate ROI.

But this isn’t the whole formula.

Pat added something on to this formula that, as far as I’m concerned, qualifies Pat as a marketing genius. Pat’s formula for calculating success is:

(Value Added/Resources Used) * Perception

What Pat recognized was that what you might consider to be “value” might not be viewed as valuable by other people. Other people like, say, your CEO or CFO.

We’re living in an ROI culture. Suggest that your company do something, and somebody will ask “what’s the ROI on that?” If you want to get up in front of your management team and suggest that your company do something because you “feel” it’s the best thing for the company to do, go for it. Just don’t send me your resume when you’re on the street. 

That doesn’t make your feeling wrong. But being right doesn’t make you successful. Persuading others to do the right thing does. 

This is why quantipulation is so important:  Quantipulation is an attempt to influence perception. To be a successful leader, innovator, or change agent, you have to shape, change, and confirm people’s perceptions.

There’s a reason I call quantipulation an art. Successful quantipulators know that it’s about more than just the data – it’s about logic and emotion. And there’s no formula or recipe for figuring out how much logic and emotion to mix in with the data.

The examples of ROI quantipulation shown above fail not because they’re wrong, but because they fail to influence perception. Those formulas simply confirm for the social media believers what they already believe. That’s easy. Converting the heathen is hard.

Had those social media ROI formulas made any attempt to link social media results to the conventional definition of ROI — financial return — they might have been more persuasive.

Last thought: Quantipulation is not inherently bad or evil. Yes, it’s a play on the word manipulative, which doesn’t have positive connotations. But I prefer to take a more realistic view: It is what it is. And it’s a necessary skill for today’s business world.

Twitter Vs. Facebook: Which Is Better For Driving Purchase Activity?

Compete recently published a blog post called Four Things You Might Not Know About Twitter. Based on its consumer data, Compete concluded that:

“Twitter is more effective at driving purchase activity than Facebook. 56% of those who follow a brand on Twitter indicated they are “more likely” to make a purchase of that brand’s products compared to a 47% lift for those who “Like” a brand on Facebook. This is further evidence that marketers can drive ROI with Twitter by engaging followers through compelling content.”

My take: Nonsense.

Compete is off-base concluding that Twitter “drove” purchase behavior simply because a larger percentage of Twitter users are “more likely” to purchase from a brand than Facebook followers do. The only way to conclude that a source is a more effective driver is by comparing actual purchase activity resulting from specific messages or offers.

In addition, without a measure of what consumers’ purchase intention was before following a brand on Twitter or liking it on Facebook, it’s impossible to determine if Twitter or Facebook is having any impact on the customer relationship (Compete’s use of the term “lift” is inappropriate in the context it was used in).

Even if Compete had that benchmark, a change in purchase intention could not be attributed to Twitter or Facebook unless the messages, content, and offers were identical.

Bottom line: This is just one example of many that claim the “superiority” of one social media platform over another. Sadly, all of them are based on flawed data and assumptions, and misses the important point:

Different platforms are better suited for different types of messages/interactions.

It’s blindingly obvious how Facebook and Twitter differ in terms of the types of messages, interactions, and content each are suited to. As a result, the only way to determine which is more “effective” is in terms of an individual company’s objectives and needs regarding engaging with customers and prospects. And that means that “effectiveness” is based on the message or content — not the platform.

In other words, neither Twitter nor Facebook is “better” for driving purchase activity.

p.s. Note to bloggers/researchers/consultants/pundits: When publishing data that purports to claim that one social network is superior to another for driving purchase activity, ROI, or whatever metric you’re talking about, it would be very helpful if you talked about WHY one platform is better than another. I don’t think I’m asking for too much, here.

Web Analytics And Analytical Maturity

I attended the Web Analytic Association’s symposium in Boston not long ago. For a good overview of the event, see Dean Westervelt’s recap on the Metrics Marketing blog. My motivation for attending was two-fold: 1) Picking up some nuggets to add to the Online Marketing Maturity model that I’ve developed, and 2) Seeing Tom Davenport speak.

Tom is the author of Competing on Analytics and a leading light in a number of management innovations over the past 20 years. I first heard of Tom in the early 90s when he wrote a Harvard Business Review article about business process redesign. He later wrote about knowledge management, and now champions the use of analytics for management decision-making. When Tom writes, I read. I can’t say that about a lot of folks in the business world.

The gist of Tom’s presentation is that we’re in a “new quantitative” era. Businesses need new decision approaches, approaches that will be driven by enterprise analytics (which are comprised by web analytics, marketing analytics, supply chain/OR analytics, HR analytics, predictive analytics, etc.).

Tom told the web analytics folks that they need new skills. They need to fix a problem — not just identify it. They need to tell a story with data, help frame decisions, and stand firm when necessary. According to Tom:

“Analytics without plans for decision-making is a waste of time.”

I couldn’t agree more.

So it struck me as a bit off-message when Tom displayed a graphic (shown below) that purported to show analytical intelligence and maturity (other citations I’ve seen of this model label the X axis as “degree of intelligence”).


My take: This model isn’t correct.

A little context on why: I often talk about a “new” competency that marketers need: A sense-and-respond competency.

Specifically, marketers needs to sense where customers and prospects are in the buying cycle and to respond with the most appropriate message.

The problem with my idea is that it’s not a new competency. It’s always what marketing has done. Tried to sense what consumer needs and wants are, and to respond with messages, offers, and actions.

What is different about marketing in the 2000s versus 30 or 40 years ago is that the amount of information available to fuel sensing activities has proliferated. Instead of relying on demographic and (internal) purchase data, the amount of data available to marketers has become overwhelming.

The respond side of the model is also very different. Not only are there many more channels or touchpoints with which a marketer can respond in, but the timing of those messages, offers, and actions can be made much more rapidly than in the past.

So why is Davenport’s model wrong?

Because it ignores the respond side of the sense-and-respond construct. Simply developing optimization or predictive models does not make you a mature marketer if you haven’t figured out how to take the output of those models and respond effectively and in a timely manner.

Ironically, based on the comment I cited above, Tom knows this very well.

Tom also commented, when presenting this model, that web analytics was mostly focused on the bottom half of the chart. I joke-tweeted at that point that Tom had pretty much ensured that he wouldn’t get invited back to next year’s symposium. Another attendee responded that I was wrong, and that web analytics agreed with Tom, and they were “getting there” in terms of moving up the model. 

That’s all well and fine. But it might be the absolutely wrong thing to do.

Analytical maturity isn’t a function of how “intelligent” your analytical approaches and models are. It’s a function of:

1. Alignment. Specifically, the alignment of analytical approach with the type of decision that needs to be made or the issue or problem that the firm is facing. Not every business problem requires an optimization or predictive model .Would you want to take a less-than-intelligent approach to solving a business problem? Of course not. That’s why drill-downs and alerts are not less intelligent than a predictive model. It all depends on what problem is being addressed.

2. Sense-and-respond ability. Queries, drill downs, and alerts might not be high on Davenport’s intelligence scale. But they can often be accomplished more quickly than a firm can develop, test, and implement a predictive or optimization model. Often, it’s the fastest response that wins, not the most elegant.

When I first thought about the sense-and-respond construct, I thought there was a third component: Assessment. I thought “first we sense, then we respond, then we evaluate or assess our actions.” But upon further thought, I realized that wasn’t quite right. Assessment/evaluation is just another form of sensing. It’s a continuous loop — we sense, respond, sense, respond, etc. So a contributor to how mature your analytics capability is depends not just on whether you respond timely, but how well you assess how effective your analytics are, and can recalibrate and adjust your models and approaches.

3. Culture. I know of some large financial services firms with teams of statisticians who develop (and implement) predictive and optimization models, who look down on web analytics efforts as somehow being inferior to their highly statistical efforts. An analytically mature organization doesn’t have this problem.

4. Data usability. In the list of factors determining analytical maturity, this is certainly not the last one that should be mentioned, but the immature use of data often stems from a cultural immaturity. As I mentioned above. plenty of firms hare developed predictive and optimization models to drive their marketing efforts. But many still use a relatively narrow set of data to power those models. Specifically, they often do not incorporate web-based data. An analytically mature marketing department uses a range of data sources, and gathers data for their analytical efforts more effectively and efficiently than a less-mature firm.

Bottom line: I really can’t speak to analytics efforts outside of the financial services world, but among financial institutions, there is an analytics gap. The gap is that highly sophisticated statistical approaches to analytics are being undertaken, but often in a very untimely manner and with a limited set of data. On the other hand, while web analytics efforts are often more timely, and incorporate a more efficiently gathered set of data, these efforts haven’t grown beyond relatively simplistic analytic approaches.

I don’t think the web analytics folks can bridge this gap by themselves.

When my fellow Symposium attendee tweeted that web analysts were “getting there” in moving up the (so-called) intelligence scale, I couldn’t help but wonder: 1) How is that going to happen? and 2) Why hasn’t it happened before?

Here’s the problem, as I see it: Unless you’re a web analyst with a good understanding of statistical approaches AND a good understanding of business problems, decision making, marketing, and organizational politics, then I’m not sure you’re well equipped to move web analytics up the maturity scale.

I’m not saying that aren’t web analysts out there who meet this criteria, but looking at this from the other end of the spectrum, I can’t say I’ve met a lot of statisticians with a good understanding of web analytics AND a good understanding of business problems, etc.

So who’s going to bridge the gap?

My bet (hope?) is on the vendor community. When I look at what IBM is doing with its acquisition of Unica, SPSS, and Coremetrics, or an Adobe with its acquisition of Omniture (who had previously acquired Offermatica and Touch Clarity), I see the pieces coming together. The part of the maturity equation that I’m not sure the vendor community can address is the cultural component.

Whatever happens, it’s an interesting time to be involved in marketing analytics.