Customer Satisfaction of American Airline Companies

Flying on US domestic airlines is a nightmare. The customer service is pathetic, the staff unfreindly, the airlines charge for every small thing…the list goes on and on.

University of Michigan carries out surveys of American customers and publishes the average scores annually as American Customer Satisfaction Index (ACSI). You can check the scores for several industries by visiting their website. For airlines, the chart looks like this:

Screen Shot 2016-04-03 at 5.20.51 PM

American Customer Satisfaction Index for American Airline Companies

The airlines appear in descending order of the 2015 ACSI scores, which range from 81 for JetBlue to 54 for Spirit.

ACSI is published for a given brand only once a year. But companies are interested in knowing about customer satisfaction round the clock. So I decided to use Twitter sentiment as a measure for customer satisfaction. This is a very rough exerise to see whether we get any results that have face validity. My students will realize that, for airlines, Twitter is one of the key social networks for addressing customer complaints. Therefore, Twitter will likely capture customer satisfaction in real time. So the validity is actually about ACSI and not about Twitter sentiment. However, there is a commonly discussed issue about Twitter — it’s not representative of the general population. Still, we must keep in mind that ACSI may not be a good representative of American flyers sentiment either.

I decided to focus on the 9 airlines for which the ACSI scores for 2015 are available – JetBlue, Southwest, Alaska, Delta, American Air, Allegiant, United, Frontier, and Spirit. The graph looks as follows:


ACSI Scores for Nine American Airlines

The average score for these 9 airlines is 68.11. As the maximum possible ACSI score is 100,  68 is not a great score. However, I am amazed at how thoroughly the epxectations of American flyers have gone down. I am sure that if the survey respondents were from Asia, you would get an average of less than 50. But that’s a story for another post where I will compare the sentiment about the best airlines including Singapore Airlines, Emirates, Qatar, etc.

Next, I went on Twitter and downloaded tweets that were directed at these airlines. My condition was simply that the Twitter handle of the airline should appear in the tweet. For example, a tweet mentioning @JetBlue would indicate that this is a tweet targeted towards JetBlue and therefore should be included for the analysis. I carried out this data collection on 2 April 2016 from Singapore. Following this, I categorized the tweets as either positive, negative, or neutral. To compare to ACSI, I created a metric similar to Net Promoter Score (NPS). The formula for that is given as follows:

\displaystyle \mbox{Net Sentiment Score} = \frac{\mbox{(Total Positive Tweets - Total Negative Tweets)}}{\mbox{Total Tweets}}


Here is the graph when I plotted net sentiment scores of all the 9 airlines:

Twitter sentiment

Net Sentiment Scores for 9 American Airlines

The score is bounded between -1 and 1. If all the tweets are negative then the score will be -1 and if all the tweets are positive then the score will be 1.

The average score is 0.19, which is around 60% of the scale range (1.19/2.0). We see that similar to ACSI graph, 4 airlines–JetBlue, Alaska, Southwest, and Delta–are above the mean while remaining 5 are below the mean. Interestingly, these are the same 4 airlines which have above average scores on ACSI. The ordering is a bit off though. In order to better compare the two graphs, I decided to plot them in the same space. However, for that I need to have the same scale. For the sake of convenience I decided to use Z-scores.¹


ACSI and Twitter Net Sentiment Score Correlation

I find that the correlation is high at 0.77. It’s also statistically significant with a p-value equal to 0.016. However, notice that we have only 9 observations, which means that the standard error is likely to be high. Actually the 95% confidence interval for the correlation coefficient is pretty wide [0.21, 0.95] but the lower level is still comfortably far from 0.

I think that ACSI is doing a fair job of capturing customer satisfaction of American air travellers. It corresponds to the Twitter sentiment quite well. It’s worth noting that I am comparing the survey results, which were collected over 1-2 months period in 2015 with tweets that were sent on or slightly before 2 April 2016. It would be worth studying how Twitter sentiment fluctuates over a period of time. This is my next assignment once I am done with the sentiment analysis of top ranked airlines.

In case you are interested in individual airlines sentiment charts, you can view them here:

¹ Z-score of any variable has 0 average and 1 standard deviation.


Marketing Quotes 2011

Valeria Maltoni created this presentation with some of the nice quotes about marketing made in 2011 so far. When you see the slides, an ad might show up at the bottom. Just close it and watch the presentation in piece!

Social Media Metrics

On Tuesday we are going to start learning social media metrics. Measurement of any marketing strategy is tough because we are rarely able to “control” the experiment as scientists do. For example, when a firm launches its advertising campaign, there can be several other things going on at the same time. The competitors tweaking their ad campaigns, changing their prices, introducing new products, customer behavior might be changing, the economy may start tanking, government may change the regulation…any of these things and many more might be going on at the same time. In that case how does an advertiser isolate the effectiveness of the ad campaign? It is really difficult.[1] The same is true for social media marketing. Therefore, understanding measurement of the effectiveness of a social media campaign is critical.

Read the articles below. They give a fair idea about social media metrics. I am going to keep on updating this post over the next few days by adding more links below.

Book – Web Analytics 2.0 by Avinash Kaushik

CLV Calculator

Net Promoter Score

How to Track Social Media Metrics Like A Rockstar

Top 10 social media dashboard tools

Why Your Friends Have More Friends Than You Do

Ever wondered why your friends seem so much more popular than you are?

The 10 Social Media Metrics Your Company Should Monitor

Social Media Metrics – Chris Brogan

Archive for the ‘Social Media Measurement’ Category – Jeremiah Owyang

[1] Check this article, however.

Social Media Career Options in Small Businesses

As my class is moving closer to the end of the semester, many of the students are actively seeking jobs. I have little idea about how the career opportunities in social media marketing domain are. Therefore, we had Lisa DiVirgilio as our guest speaker yesterday. She gave a presentation entitled “Social Media in the Real World,” which is embedded below. Here I summarize some of the key points from the presentation. Note that we are talking about the jobs in the small businesses in local areas.

1. Career options: There are several levels at which social media specialists can work. For the entry level, however, the students are most likely expected to join as social media assistants or junior strategists. The salaries are relatively lower in the news media, local TV channels, etc.

2. Flexibility: The jobs have lot of flexibility because the social media strategists need not be tied to a chair and table in the office the whole day!

3. Entrepreneurship: Social media strategists in most cases have the liberty to design the firm’s social media efforts from the scratch. This is like running your own small business.

4. Performance metrics: Although measurement of performance is tough, it is still one of the most critical aspects of social media marketing.

Enjoy the presentation!

Where is Blogging Going?

Darren Rowse, who runs a photography blog talked about blogging, Flickr, and Facebook pages in the video embedded below.


1. Bloggers are realizing declining revenues from advertisements. The future revenue might come from original content.

2. Facebook pages are a great way to generate ideas for blog posts.

Vodpod videos no longer available.

Dragonfly Effect

Prof. Jennifer Aaker, one of the most respected consumer researcher, wrote a book entitled “Dragonfly Effect” about social media with her husband Andy Smith. I have added a couple of chapters from this book in my syllabus. I will highly recommend this book to you. It is not specifically about social media marketing per se, but it talks about the use of social media in various nonbusiness settings.

Here is a video of the interview of Andy Smith where he is discussing use of social media in Obama’s presidential campaign.

Vodpod videos no longer available.

Social Business Forecast 2011

Earlier I had shared a report by Altimeter’s Jeremiah Owyang. Below is the presentation from the same report. It will help you in understanding the key points if you don’t have time to read through the entire report. My students should look at slides 19 and 20 carefully where the priorities in social media domain are depicted. Not surprisingly ROI calculation is the most critical issue.

After the spring break, we are going to spend time in understanding social media and web analytics. The students will be setting up dashboards to monitor and evaluate the performance of their social media campaigns. Personally I am quite excited about this because performance metrics is my research area!