Customer Satisfaction of American Airline Companies

Flying on US domestic airlines is a nightmare. The customer service is pathetic, the staff unfreindly, the airlines charge for every small thing…the list goes on and on.

University of Michigan carries out surveys of American customers and publishes the average scores annually as American Customer Satisfaction Index (ACSI). You can check the scores for several industries by visiting their website. For airlines, the chart looks like this:

Screen Shot 2016-04-03 at 5.20.51 PM

American Customer Satisfaction Index for American Airline Companies

The airlines appear in descending order of the 2015 ACSI scores, which range from 81 for JetBlue to 54 for Spirit.

ACSI is published for a given brand only once a year. But companies are interested in knowing about customer satisfaction round the clock. So I decided to use Twitter sentiment as a measure for customer satisfaction. This is a very rough exerise to see whether we get any results that have face validity. My students will realize that, for airlines, Twitter is one of the key social networks for addressing customer complaints. Therefore, Twitter will likely capture customer satisfaction in real time. So the validity is actually about ACSI and not about Twitter sentiment. However, there is a commonly discussed issue about Twitter — it’s not representative of the general population. Still, we must keep in mind that ACSI may not be a good representative of American flyers sentiment either.

I decided to focus on the 9 airlines for which the ACSI scores for 2015 are available – JetBlue, Southwest, Alaska, Delta, American Air, Allegiant, United, Frontier, and Spirit. The graph looks as follows:

ACSI

ACSI Scores for Nine American Airlines

The average score for these 9 airlines is 68.11. As the maximum possible ACSI score is 100,  68 is not a great score. However, I am amazed at how thoroughly the epxectations of American flyers have gone down. I am sure that if the survey respondents were from Asia, you would get an average of less than 50. But that’s a story for another post where I will compare the sentiment about the best airlines including Singapore Airlines, Emirates, Qatar, etc.

Next, I went on Twitter and downloaded tweets that were directed at these airlines. My condition was simply that the Twitter handle of the airline should appear in the tweet. For example, a tweet mentioning @JetBlue would indicate that this is a tweet targeted towards JetBlue and therefore should be included for the analysis. I carried out this data collection on 2 April 2016 from Singapore. Following this, I categorized the tweets as either positive, negative, or neutral. To compare to ACSI, I created a metric similar to Net Promoter Score (NPS). The formula for that is given as follows:

\displaystyle \mbox{Net Sentiment Score} = \frac{\mbox{(Total Positive Tweets - Total Negative Tweets)}}{\mbox{Total Tweets}}

 

Here is the graph when I plotted net sentiment scores of all the 9 airlines:

Twitter sentiment

Net Sentiment Scores for 9 American Airlines

The score is bounded between -1 and 1. If all the tweets are negative then the score will be -1 and if all the tweets are positive then the score will be 1.

The average score is 0.19, which is around 60% of the scale range (1.19/2.0). We see that similar to ACSI graph, 4 airlines–JetBlue, Alaska, Southwest, and Delta–are above the mean while remaining 5 are below the mean. Interestingly, these are the same 4 airlines which have above average scores on ACSI. The ordering is a bit off though. In order to better compare the two graphs, I decided to plot them in the same space. However, for that I need to have the same scale. For the sake of convenience I decided to use Z-scores.¹

Z-scores.png

ACSI and Twitter Net Sentiment Score Correlation

I find that the correlation is high at 0.77. It’s also statistically significant with a p-value equal to 0.016. However, notice that we have only 9 observations, which means that the standard error is likely to be high. Actually the 95% confidence interval for the correlation coefficient is pretty wide [0.21, 0.95] but the lower level is still comfortably far from 0.

I think that ACSI is doing a fair job of capturing customer satisfaction of American air travellers. It corresponds to the Twitter sentiment quite well. It’s worth noting that I am comparing the survey results, which were collected over 1-2 months period in 2015 with tweets that were sent on or slightly before 2 April 2016. It would be worth studying how Twitter sentiment fluctuates over a period of time. This is my next assignment once I am done with the sentiment analysis of top ranked airlines.

In case you are interested in individual airlines sentiment charts, you can view them here:

¹ Z-score of any variable has 0 average and 1 standard deviation.

Marketing Analytics – Summary of Session 1

We started the second trimester today at ESSEC’s Singapore campus. I am teaching Marketing Analytics (Engineering) to two sections of 50 students each. In my first lecture I introduced the fundamental problem in front of marketers – how to justify their decisions to others who control the budget. Gone are the days when people could simply use experience, gut feel, intuition, etc. as valid criteria for selecting marketing strategies. Now nobody wants to bet even $1 on speculative marketing managers. Data driven marketing is the new norm. My course is an introduction to this new reality. In any marketing course, ‘brand positioning’, ‘segmentation’, ‘targeting’, ‘media planning’, etc. are common terminologies. Professors and students know what these concepts mean. Yet, given a real life business situation how many students will be able to actually come up with a strategic solution? Very few indeed.

Our Course Text – Principles of Marketing Engineering

Over the next five weeks, we will take a two-step approach. First, we will clarify a certain marketing concept, for e.g., positioning. We will then understand what type of information needs to be collected to plot a perceptual map showing the brand positioning on 2 or 3 dimensional space. Next we will use SPSS to do the data analysis using statistical techniques such as factor analysis. Finally, based on the perceptual maps, students will recommend actions. There will be hard numbers involved. For example, when the students suggest launching a new brand to exploit potential gap in the market, they will need to justify that by projecting the changes in the market shares. They will have to account for cannibalization of any existing brands from the same company that is supposed to launch a new brand. This will be a complex but fun exercise!

The other topics include decisions on segmentation using probability models, salesforce allocation, and conjoint analysis. As we started working with SPSS today, I used a dataset consisting of accounting information on several US firms over 2010 and 2011. The students’ first task is to build a sales response model and test it using the data. To what extent do the sales respond to advertising? The response model will not be very complicated yet we may end up using a logit-type curve (ADBUDG model), who knows?

I believe that modeling the data is not the most important thing. It’s just a small component of decision making. The critical parts are to read the analysis, interpret it, and then recommend a decision path. I don’t like blind data mining of millions of data points to come up with patterns that everyone believes are true. Unfortunately this is exactly what’s happening in the analytics area. Data mining coupled with intelligent experiments is the way to go. (More on this later). Bringing intuition to this party is like inviting Michael Lohan to speak at a conference on responsible parenting!

How Fashion Brands Should Use Facebook

This post doesn’t list my opinions. Instead check out the embedded video below where Tracy Yaverbaun, Director of Retail, Luxury & Fashion Partnerships at Facebook shares her experiences. Start watching from 4 minutes mark.

Update: Watch it from 18 minutes mark for more relevant discussion on luxury and fashion

Targeting Eligible “Indians” – A Poll

In the screenshot below, you will see an ad for Indian Dating website appearing next to an article about native Americans, i.e., the mislabeled “Indians”! Please take the poll at the end of this post. The poll asks who you think is responsible for this misplacement of the advertisement.

Indian dating website’s ad was shown next to an article about native Americans

IBM CMO Study

Last year IBM conducted a study interviewing 1,700 CMOs across the globe. Some key finding from the report –

“Our interviews reveal that CMOs see four challenges as pervasive,universal game-changers: the data explosion, social media, proliferation of channels and devices, and shifting consumer demographics. But CMOs from outperforming organizations address these challenges differently from other CMOs”

I was curious to know what the successful CMOs do different. And lo and behold, there are three things that we keep on teaching to our students all the time and still many never seem to appreciate them –

1. The most proactive CMOs are trying to understand individuals as well as markets

2. CMOs in the most successful enterprises are focusing on relationships, not just transactions

3. The outperformers are committed to developing a clear “corporate character”

Finally, something my marketing analytics students will appreciate –

“Our research shows the measures used to evaluate marketing are changing. Nearly two-thirds of CMOs think return on marketing investment will be the primary measure of their effectiveness by 2015. But proving that value is difficult. Even among the most successful enterprises, half of all CMOs feel insufficiently prepared to provide hard numbers.”

Read the full report here –

iPad 2 Prices Across the World

I was a bit bored yesterday. So I put together these slides comparing iPad 2 prices in 35 different countries. You may find this on other websites and perhaps with more details. But I am including the screenshots of the Apple Store webpages. Note that all the prices are for the base iPad 2 model.

Application of Facebook to Local Marketing

I found this video on this blog. It is interesting because Dennis Yu works in the local application of Facebook marketing. I think there is a tremendous need for even workshops for such things. I have designed one, in fact. But I don’t think my b-school will ever offer it! Essec is a premium brand. Anyways, enjoy!