Naive Bayes

Today I want to talk a bit about Naive Bayes Machine Learning Algorithm, it's logic, usage area, advantages and disadvantages, those are the topics which this article covers. Naive Bayes Machine Learning Algorithm is one of the most popular and widely used machine learning algorithms. It is straightforward and powerful and also it is the algorithm which is suggested to try even if you have millions of the records with their attributes, it is kind of classifier which is one of the members of Supervised Learning Algorithms.


This algorithm is called naive Bayes or idiot Bayes as well. The reason for calling this algorithm like that is the simple calculation of the probabilities. The algorithm works by assuming that given data is conditionally independent and this assumption may not fit in real data, but somehow this approach performs surprisingly well on the data even in case of not fitting.
 
Some of the real world usage of the Naive Bayes Algorithm
Text classification, spam filtering. recommendation systems, etc.

 
Advantages
It is the fast and highly scalable algorithm.
It is a simple algorithm which depends on computing probabilities.
It can be easily fed by the small dataset.
Good choice for Text Classifications problems.

 
Disadvantages
It cannot learn the relationship between features.
The strong assumption of independence class, it is almost impossible to find such data in real life.

To be able to understand this algorithm firstly we need to understand Bayes theorem, so here the brief explanation of the Bayes theorem;

Bayes Theorem

Bayes theorem named after Reverend Thomas Bayes (1701-1761), it is based on conditional probability, it means the probability that something will happen depend on something else has already happened. By using this theorem, we can calculate the probability of an event by using its prior knowledge.

Here is the formula for Calculating the Conditional Probability

P(T) is the probability of thesis is true, it is also known as the prior probability.
P(E) is the probability of evidence.
P(E|T) is the probability of the evidence when thesis is true
P(T|E) is the probability of the thesis when evidence is exist

After brief information about Bayes Theorem and it's logic, let's continue with Naive Bayes Algorithm.

Naive Bayes Algorithm Example

So now, it is time to do an example to understand how we can apply this algorithm to problem-solving.
Let's assume that we want to predict user comment if it is positive or negative, to be able to do this, we need to feed our algorithm with some knowledge which it can be used to make predictions later.

Let's start with a dataset which we can use for this purpose. You can download Dataset from this link.

As you can see below, the dataset contains User Comment and Class, '1' means positive and '0' means negative comments.

Before jumping in writing code, we need to understand that if we want to use this algorithm in production, we will need much more data than this dataset contains.

This dataset is an example to show how the algorithm works by looking prior knowledge for making predictions, it is not for production usage.

To be able to make predictions about the type of the user comments, we need to make some calculations for probabilities of the words according to their usage in the type of the comment. A different way to say, we need to calculate all words' probabilities according to the type of comments (positive or negative) which they used.

Let's Write Some Code

To be able to work with comments, we define a model class which we use for defining sentences in each comment, or in our case we can say that each sentence is the comment, so we need to model class which we can define our training comments.

Sentence Model

public class SentenceModel
    {
        public SentenceModel(string text)
        {
            Text = text;
        }

        public SentenceModel(string text, double category)
        {
            Text = text;
            Category = category;
        }

        public string Text { get; private set; }

        public double Category { get; private set; }
    }
    

We can write a helper function for loading all sentences from the file into our list of sentences and then we can define a list of the SentenceModel which we can keep all comments and their type and use for feeding our algorithm.

Load Data Function

        /// 

        ///     Loads data from the file
        /// 
        /// Filename
        /// Name of the sheet
        /// Columns which needs to fetch,if it is null all columns will be fetched 
        public DataSet LoadData(string fileName, string sheet, string[] columns = null)
        {
            if (!File.Exists(fileName))
                throw new FileNotFoundException();

            var connStr =
                string.Format(
                    "Provider=Microsoft.ACE.OLEDB.12.0;Data Source={0};Extended Properties=\"Excel 12.0 Xml;HDR=YES;IMEX=1\";",
                    fileName);

            //Create dataset
            var dataSet = new DataSet();
            dataSet.Tables.Add(new DataTable(sheet));
            var table = dataSet.Tables[0];

            if (columns != null)
                for (var i = 0; i <= columns.Length - 1; i++)
                    table.Columns.Add(columns[i]);

            var items = new List