#4 Project progress: Building a basic machine learning successfully

In this journal, we will continue to build our first basic machine learning after importing and cleaning data. Besides pandas module, sklearn.feature_extraction.text and sklearn.metrics.pairwise are two more modules that we will use to import TfidfVectorizer and linear_kernel methods. 

TfidfVectorizer comes from sklearn module, that uses to convert string (words) to matrix (vector) shape. Linear_kernel also comes from sklearn module, that uses to generate the linear similarity between two vectors. Cosine_similarity method can do the same work as linear_kernel, but we use linear_kernel because computer compiles data faster. 

The third new method that we use in this project is pandas Series. We use this method to assign index for a dataframe, you can take a look at line 31. Line 36 displays the code for creating a system. We use enumerate method for enumerate similarity and title, we put list() method in front of it because we want to create it as a list. Then we use sorted method and lambda function to sort the value of scores of 10 first similar movies. We use iloc method in order to retrieve the rows of sheet. From here, I believe we can start to demo our machine!



And here we go, that's what I got!


Comments

  1. UnknownOctober 28, 2021 at 4:47 PM

    I genuinely want to hear your feedback and advice about next journal.
    1. I should write detailed deeply about Python modules, such as pandas, numpy, scikit-learn,... and practice on these modules in order to improve my knowledge about Python. Or,
    2. I should learn how to create interface for this movie recommendation machine?

    I noted that you suggested me to think about front-end, can you go more detail about this please?

    Your advice is so helpful, it motivates me to study more and more!
    Thank you so much for your time!

    Bao Huynh

    ReplyDelete
    Replies
    1. EricaOctober 31, 2021 at 9:54 AM

      Bao,

      Very nice work! I’m glad the program worked for you.
      With regards to my comment on front-end planning: it’s more for future projects. The more detailed your project plans are, the faster and easier your projects will progress.
      For future directions, I’d like you to choose which path will be better for your future. So if learning and detailing more of python will help you most, choose that. If interfacing is the area you’d like more experience in, then go with that option. I support either direction.

      Delete

Post a Comment

Popular posts from this blog

#8 Sklearn - Python package - Model evaluate metrics for regression

Image
    One of the most important steps in building prediction model is evaluating how accurate our model is by the way using metrics to calculate the amount of deviation between predicted value and actual value. In this journal, we will dive into 4 metrics, they are Mean Absolute Error, Mean Square Error, Root Mean Spare Error, and R-squared score. All the functions we use that come from sklearn Python package (from sklearn import metrics).        Mean Absolute Error   (MAE) is the mean of the absolute value of the errors. In other words, we use the differences of each actual value with predicted value then divide by its number of observations. It measures the average of the residuals in the dataset. That means the weight of all the errors are the same. For example, the difference of first pair yi1 and y^i1 is 5, the difference of second pair yi2 and y^i2 is 1. The average of them will be 3. Then the error will be off by 3. So, "MAE  is bes...
Read more

#3 Project Progress: Import and Clean the Data in Python

Image
I am so excited to share with you about my first project in Python related to Data Science. In order to complete this project, first of all, we need to import the data. Jupyter notebook is a interactive computing notebook environment, that is came with Anaconda - a Python distribution for scientific computing. We will run our code on this notebook. We will use read_csv() method in pandas package to read the csv files. My file here is Data.csv, that I download from Kaggle ( TMDB 5000 Movie Dataset | Kaggle ). df.head() uses for printing some head rows of data. df.shape() uses for printing the shape of data, you can see in this data set, we have 8403 rows and 20 different features.  After importing the data, we need to clean them to make sure we won't mess them up when we manipulate to build a model. We have total 20 features here but we won't use them all. So, we need to decide what features are important and what are not. Because I want to build a content-based movie recommenda...
Read more

#10 Fighting The Semantic Gap On CBIR Systems

Image
      CBIR which stands for Content Based Image Retrieval is a technique that involves querying an image. Which you then retrieve matches to the queried image based on its similarity results. It typically involves comparing the low-level features of an image, including color, texture, and shape. However, what if you want to compare high level features? High level features mean, the semantic meaning behind an image that humans are basically able to discern. For example, say you want, a tall happy person, or a happy small dog. A person would be able to pick out images pretty easily from a set that match these criteria, a system not so much. What exactly can be done to bridge this Semantic Gap?       The academic article “Fighting the Semantic Gap on CBIR Systems through New Relevance Feedback Techniques” mentions using relevance feedback techniques to combat this.” The paper describes the process as “Therefore, by gathering the users´ indications, algori...
Read more