Advance Customer Analytics (Day 2)
This article is a continuation of ACA:(Day 1). We will be covering Recommendation System, Text Mining and Sentiment Analysis.
Recommendation System:
In current scenario exiting businesses are facing a big problem, that is how to present relevant content to the users.
- The consumers are facing an ever expanding array of alternatives to choose from
- Amazon sells 44 M books and 10 M Electronics goods
- Netflix stream 1 billion hours of content per week
- They are also facing a humongous amount of product and service related data
Companies do not know what content to show case to users. Results are not customized. As a result of this, a lot of issues have come up such as:
- Growing number of consumers fast forward ads, or avoid seeing them at all
- 90% of consumers unsubscribe from an e-mail list they opted into
- 50% of consumers abandons their online shopping cart
Solution to all these problems are Recommendation Systems. What these systems do for you:
1. Filter, prioritize and efficiently deliver relevant information to the user in a very short period of time
2. Provides the most relevant information that the user is seeking based on their preferences, or observed behavior
These engines can be of following types:
- Non-Personalized
- Content-based
- Collaborative
- Hybrid
During the course, they explained about how to make a Content Based Recommendation systems.
The process we followed was:
1. Create Corpus
2. Preprocess Data
3. Document Term Matrix
4. Calculate TF-IDF scores
5. Term Similarity Matrix
6. Cluster
Before going further we studied about Conjoint Analysis. Conjoint Analysis enables businesses to mathematically analyze consumer or client behavior and make decisions based on real insights from data. This allows them to better cater to consumer needs and develop business strategies that provide a competitive edge. This is because the fulfillment of customer wishes in a profitable way requires companies to fully understand which aspects of their product and service are most valued by the customer.
Conjoint analysis is considered to be the best survey method for achieving this purpose. It consists of creating, distributing and analyzing surveys among customers with the purpose of modeling their purchasing decision based on response analysis.
Choice-based Conjoint (CBC) Analysis: This type of conjoint analysis is the most popular because it asks consumers to imitate the purchasing behavior in the real market: which products they would choose, given certain criteria on price and features.
Adaptive Conjoint Analysis (ACA): This type of conjoint analysis is used more often in scenarios where the number of attributes/features exceeds what can be done in a choice based scenario. ACA is great for product design and segmentation research, but not for the price.
Now the focus has shifted from conjoint analysis to modelling based approach.
Text Mining and Sentiment Analysis play a major role in recommendation engines.
Text Mining:
When we have a document, we need to first understand what document is about. To determine that we have Term Frequencies, which tell us how frequently a word occurs in a document. Any kind of prepositions or is/am/are, we call these words as STOP WORDS and we remove them from analysis.
Another approach is to look at a term’s inverse document frequency (idf), which decreases the weight for commonly used words and increases the weight for words that are not used very much in a collection of documents. This can be combined with term frequency to calculate a term’s tf-idf (the two quantities multiplied together), the frequency of a term adjusted for how rarely it is used. The idea of tf-idf is to find the important words for the content of each document by decreasing the weight for commonly used words and increasing the weight for words that are not used very much in a collection or corpus of documents
We made use of R to code during the session. Using TF-IDF approach, we determined which words hold importance and gave us valuable insights.
Sentiment Analysis:
It is the process of determining the emotional tone behind a series of words, used to gain an understanding of the the attitudes, opinions and emotions expressed within an online mention. Not only can you see what people think of your own products or services, you can see what they think about your competitors too. The overall customer experience of your users can be revealed quickly with sentiment analysis, but it can get far more granular too.
Word Cloud:
A Word Cloud or Tag Cloud is a visual representation of text data in the form of tags, which are typically single words whose importance is visualized by way of their size and color.
We worked on airlines data and created word cloud specific to that data set.
Few tools they suggested:
1. SEMANTRIA: https://www.lexalytics.com/semantria
2. SHINY: https://shiny.rstudio.com/