Customer Satisfaction of American Airline Companies

Flying on US domestic airlines is a nightmare. The customer service is pathetic, the staff unfreindly, the airlines charge for every small thing…the list goes on and on.

University of Michigan carries out surveys of American customers and publishes the average scores annually as American Customer Satisfaction Index (ACSI). You can check the scores for several industries by visiting their website. For airlines, the chart looks like this:

Screen Shot 2016-04-03 at 5.20.51 PM

American Customer Satisfaction Index for American Airline Companies

The airlines appear in descending order of the 2015 ACSI scores, which range from 81 for JetBlue to 54 for Spirit.

ACSI is published for a given brand only once a year. But companies are interested in knowing about customer satisfaction round the clock. So I decided to use Twitter sentiment as a measure for customer satisfaction. This is a very rough exerise to see whether we get any results that have face validity. My students will realize that, for airlines, Twitter is one of the key social networks for addressing customer complaints. Therefore, Twitter will likely capture customer satisfaction in real time. So the validity is actually about ACSI and not about Twitter sentiment. However, there is a commonly discussed issue about Twitter — it’s not representative of the general population. Still, we must keep in mind that ACSI may not be a good representative of American flyers sentiment either.

I decided to focus on the 9 airlines for which the ACSI scores for 2015 are available – JetBlue, Southwest, Alaska, Delta, American Air, Allegiant, United, Frontier, and Spirit. The graph looks as follows:


ACSI Scores for Nine American Airlines

The average score for these 9 airlines is 68.11. As the maximum possible ACSI score is 100,  68 is not a great score. However, I am amazed at how thoroughly the epxectations of American flyers have gone down. I am sure that if the survey respondents were from Asia, you would get an average of less than 50. But that’s a story for another post where I will compare the sentiment about the best airlines including Singapore Airlines, Emirates, Qatar, etc.

Next, I went on Twitter and downloaded tweets that were directed at these airlines. My condition was simply that the Twitter handle of the airline should appear in the tweet. For example, a tweet mentioning @JetBlue would indicate that this is a tweet targeted towards JetBlue and therefore should be included for the analysis. I carried out this data collection on 2 April 2016 from Singapore. Following this, I categorized the tweets as either positive, negative, or neutral. To compare to ACSI, I created a metric similar to Net Promoter Score (NPS). The formula for that is given as follows:

\displaystyle \mbox{Net Sentiment Score} = \frac{\mbox{(Total Positive Tweets - Total Negative Tweets)}}{\mbox{Total Tweets}}


Here is the graph when I plotted net sentiment scores of all the 9 airlines:

Twitter sentiment

Net Sentiment Scores for 9 American Airlines

The score is bounded between -1 and 1. If all the tweets are negative then the score will be -1 and if all the tweets are positive then the score will be 1.

The average score is 0.19, which is around 60% of the scale range (1.19/2.0). We see that similar to ACSI graph, 4 airlines–JetBlue, Alaska, Southwest, and Delta–are above the mean while remaining 5 are below the mean. Interestingly, these are the same 4 airlines which have above average scores on ACSI. The ordering is a bit off though. In order to better compare the two graphs, I decided to plot them in the same space. However, for that I need to have the same scale. For the sake of convenience I decided to use Z-scores.¹


ACSI and Twitter Net Sentiment Score Correlation

I find that the correlation is high at 0.77. It’s also statistically significant with a p-value equal to 0.016. However, notice that we have only 9 observations, which means that the standard error is likely to be high. Actually the 95% confidence interval for the correlation coefficient is pretty wide [0.21, 0.95] but the lower level is still comfortably far from 0.

I think that ACSI is doing a fair job of capturing customer satisfaction of American air travellers. It corresponds to the Twitter sentiment quite well. It’s worth noting that I am comparing the survey results, which were collected over 1-2 months period in 2015 with tweets that were sent on or slightly before 2 April 2016. It would be worth studying how Twitter sentiment fluctuates over a period of time. This is my next assignment once I am done with the sentiment analysis of top ranked airlines.

In case you are interested in individual airlines sentiment charts, you can view them here:

¹ Z-score of any variable has 0 average and 1 standard deviation.


Making Twitter Collaborative Using CoTweet

Today we learned how to use Twitter and various related cites. Since my students work in groups, I wanted them to create a Twitter account that can be used by all the group members. Now, it turns out that it is not straightforward! In, one group member could  create a blog and then add the group mates with administrative rights. This way, they can collaborate on blogs. However, Twitter has not yet publicly introduced a multiple-user Twitter account.

I found that is a web-based Twitter client that lets you add multiple users to one Twitter account! Here is a screenshot of how you do it.

Once you add your group members, they can access the Twitter account on their own.


How will you know who has been tweeting? Luckily Cotweet has thought about that. You can create cotags for each group members. The tweet will then contain that cotag which will identify the group member. For example, my cotag can be ^AM, just 3 character long, which will be added to the tweet. Here is a screenshot–

Tweet Scheduling

The coolest feature of Cotweet is that it lets you schedule your tweets! For businesses it is a very important feature because many promotions are planned in advance and you may want to schedule those tweets beforehand. Besides that, Cotweet provides you with a URL shortener too. Here is a screenshot

Overall, I think that CoTweet is a very useful Twitter client. Try it and let me know what you think!

Are We in a Social Media Bubble?

Many of us who have experienced the Internet bubble at the turn of the century are concerned about billions of dollars of current valuation of the social media companies like Facebook, Twitter, and LinkedIn. Yesterday WSJ reported that JP Morgan is planning a fund to invest in social media companies. Although JP Morgan has not yet provided any details, blogosphere is abuzz with analysis of the situation. Many reputed financial bloggers have equated social media valuation with the Internet bubble. See for example, here and here.

I think that there are some important differences between the valuation of the Internet firms and the social media firms. For example, unlike social media firms, most of the dotcom firms didn’t have any business model or revenues. The average life of these firms was less than 3 years (citation needed). Even then I am not ruling out overvaluation of social media companies outright. The question is “Is the over valuation of social media firms comparable to the firms?” It seems that many agree on a “No.” Here is a nice post about the Facebook revenue model and the way they are expanding. It is critical to realize that Facebook is a platform for so many things such as advertising, ecommerce, etc. Interestingly, Felix Salmon jumped into the fray to defend the valuation of Twitter!

It seems that the people who are criticizing the valuation don’t even know these businesses well. For example, Duff McDonald argues that there is no reason for people to pay for ad-free Pandora because they never have to see the ads. However, as one commenter pointed out, this is not the case! The commenter says –

Ads on Pandora are mostly audio ads. They may have some on their site, but you’ll hear audio ads in between songs in the standard version. There’s definitely a case to be made for paying a few extra bucks to avoid those ads.

Felix Salmon also points that fixating on value to revenue ratio is wrong since the revenue is currently being generated only from one part of the business. It is possible that the growth of revenue may not be huge in that part but as more revenue streams are tapped, the valuation might be justified.

It is possible that the businesses that are going to generate all the money only from advertising (a la Google) may find the growth flattening. Even though the US social media ad spending is increasing fast, it is still tiny at around $2-3 billion per year. It is therefore difficult to believe that only the ad revenue will lead to these huge valuations.