AI for Sentiment Analysys with C# and ML.Net

We're setting out to build a sentiment analysis application with C# and ML.NET, a cross-platform, open-source machine learning framework. The goal is to determine public sentiment for the brand Nvidia using Twitter data.

What is Sentiment Analysis? Sentiment Analysis, also known as Opinion Mining, is the process of determining the emotional tone behind a series of words. It's a powerful tool in understanding human behaviour and opinions on a large scale. By using sentiment analysis, we can gauge public opinion about a product or brand, measure customer satisfaction, or monitor social media conversations about a trending topic.

In this series, we'll focus on the sentiment towards Nvidia. Nvidia is a brand renowned for its graphics processing units (GPUs), making it an interesting subject for sentiment analysis.

Part 1: Setting Up Your Development Environment and Twitter API

Before we delve into coding, let's set up our development environment. This is a crucial part of any software project. We want to ensure that all the necessary software and packages are correctly installed.

Software Installation

1. .NET Core SDK: This is the software development kit (SDK) for Microsoft's .NET Core. Make sure you have it installed on your machine. If not, you can download it from the official .NET website.

2. Visual Studio 2019 or later: We'll be using this integrated development environment (IDE) for our project. If you don't have it already, you can download it here.

3. ML.NET: This is Microsoft's open-source and cross-platform framework for machine learning. It allows .NET developers to create custom ML models using C# without expertise in machine learning. Install it using the NuGet package manager in Visual Studio (right-click on your project > Manage NuGet Packages > Search for 'ML.NET' > Install).

Twitter API Setup

We'll be using the Twitter API to gather tweets about Nvidia for our sentiment analysis. To access the API, you need to have a Twitter Developer Account.

1. Twitter Developer Account: Apply for a developer account on the Twitter Developer Portal and create a new App to get your API keys. These keys will be used to authenticate your application while making requests to the Twitter API.

2. TweetinviAPI: A client library that makes it easier to interact with the Twitter API in .NET. Install it using NuGet, just like ML.NET.

Part 2: Fetching and Pre-processing Twitter Data

After setting up the environment, it's time to start coding! Our first step is to fetch tweets related to Nvidia. For this, we'll create a new ASP.NET Core Web Application in Visual Studio. We'll name it 'SentimentAnalysis'.

Creating a Service to Fetch Tweets To fetch tweets, we'll use the TweetinviAPI library. Let's create a new class 'TwitterService.cs' and add the following code:

public class TwitterService
{
    private readonly TwitterClient _client;

    public TwitterService()
    {
        _client = new TwitterClient("your_consumer_key", "your_consumer_secret", "your_access_token", "your_access_secret");
    }

    public async Task<List<string>> GetTweetsAsync(string brand, int count)
    {
        var searchParameter = new SearchTweetsParameters(brand)
        {
            Lang = LanguageFilter.English,
            TweetSearchType = TweetSearchType.All,
            PageSize = count,
        };

        var tweets = await _client.SearchV2.SearchTweetsAsync(searchParameter);
        return tweets.Tweets.Select(t => t.Text).ToList();
    }
}

This service will connect to the Twitter API and fetch the English language tweets about a specified brand.

Processing the Data Machine learning algorithms need clean data. This means the data must be processed and transformed into a suitable format before it can be used. Here are the steps we'll take:

1. Removing Noise: We'll strip out unnecessary information such as URLs, numbers, and special characters that don't contribute to the sentiment.

2. Normalization: We'll convert all the text to lowercase to ensure uniformity. This is because our algorithm may treat the same words with different cases as different words.

3. Tokenization: We'll break down the tweets into individual words, or tokens, as our sentiment analysis model will consider words individually.

Part 3: Building and Training the Sentiment Analysis Model

Once our data is prepared, we're ready to start building and training our sentiment analysis model using ML.NET.

Creating the Model We'll create a SentimentAnalysisService class to train the sentiment analysis model and make predictions.

Here, we define a pipeline that specifies the steps to transform the data, train the model, and map the predicted label to its original form. Once our pipeline is defined, we use it to train our model on the preprocessed data.

public class SentimentAnalysisService
{
    private readonly MLContext _mlContext;
    private ITransformer _model;

    public SentimentAnalysisService()
    {
        _mlContext = new MLContext();
    }

    public void TrainModel(IEnumerable<SentimentData> trainingData)
    {
        var pipeline = _mlContext.Transforms.Text.FeaturizeText(outputColumnName: "Features", inputColumnName: nameof(SentimentData.SentimentText))
            .Append(_mlContext.BinaryClassification.Trainers.SdcaLogisticRegression(labelColumnName: "Label", featureColumnName: "Features"))
            .Append(_mlContext.Transforms.Conversion.MapKeyToValue("PredictedLabel"));

        _model = pipeline.Fit(_mlContext.Data.LoadFromEnumerable(trainingData));
    }

    public SentimentPrediction Predict(SentimentData sentimentData)
    {
        var predictionEngine = _mlContext.Model.CreatePredictionEngine<SentimentData, SentimentPrediction>(_model);
        return predictionEngine.Predict(sentimentData);
    }
}

Part 4: Analyzing the Sentiment and Visualizing the Results

The last step is to analyze the sentiment of the fetched tweets using our trained model. To achieve this, we'll create an ASP.NET Core MVC controller and corresponding views to handle the web interface of our application. We want to display the overall sentiment and a list of tweets with their respective sentiment.

Creating the Controller and Views

Let's start by adding a new controller SentimentAnalysisController in the Controllers folder. This controller will have an action 'Index', which will be responsible for fetching the tweets, predicting their sentiment, and passing the results to the view

public class SentimentAnalysisController : Controller
{
    private readonly TwitterService _twitterService;
    private readonly SentimentAnalysisService _sentimentAnalysisService;

    public SentimentAnalysisController()
    {
        _twitterService = new TwitterService();
        _sentimentAnalysisService = new SentimentAnalysisService();
    }

    public async Task<IActionResult> Index()
    {
        var tweets = await _twitterService.GetTweetsAsync("Nvidia", 100);

        var predictions = tweets.Select(tweet => new SentimentPrediction
        {
            SentimentText = tweet,
            Prediction = _sentimentAnalysisService.Predict(new SentimentData { SentimentText = tweet }).Prediction
        }).ToList();

        return View(predictions);
    }
}

Next, let's create a corresponding Index view in the Views/SentimentAnalysis folder. This view will display the sentiment prediction for each tweet.

@model List<SentimentPrediction>

<h2>Sentiment Analysis for Nvidia</h2>

<table class="table">
    <thead>
        <tr>
            <th>Tweet</th>
            <th>Sentiment</th>
        </tr>
    </thead>
    <tbody>
        @foreach (var prediction in Model)
        {
            <tr>
                <td>@prediction.SentimentText</td>
                <td>@(prediction.Prediction ? "Positive" : "Negative")</td>
            </tr>
        }
    </tbody>
</table>

Running the Application

Now that we have everything in place, it's time to run our application. After compiling and running the project, navigate to the '/SentimentAnalysis' route in your web browser. You'll see a list of recent tweets about Nvidia, each with a prediction of positive or negative sentiment.

With this, we've successfully built a sentiment analysis application with C# and ML.NET. This application can provide valuable insights about the public sentiment towards Nvidia, or any other brand. By analyzing this sentiment over time, marketers can better understand their audiences and make informed decisions.

In the future, you could expand this project to analyze sentiments in other social media platforms, conduct topic modeling to find common themes in tweets, or employ more sophisticated NLP techniques for better sentiment prediction. Happy coding!

I hope you found this series informative and inspiring. If you're seeking expertise in AI development, don't hesitate to get in touch. With my robust knowledge of various AI integrations and a passion for problem-solving, I can build AI-driven solutions tailored to your specific needs. Whether it's enhancing your application with sentiment analysis, predictive analytics, natural language processing, or any other AI functionality, I have you covered. Let's harness the power of AI to propel your business to new heights. Contact me today at njpatel@stackedsoftware.com, and let's turn your vision into reality.