Natural Language Processing

What is Natural Language Processing?

The computational technique that helps computers understand human language is called natural language processing (NLP). The computer needs to read, decipher, understand and interpret the human language in a valuable manner. This technique has evolved from an era where processing time has drastically reduced from minutes to a few milliseconds. Today, millions of web pages can be processed by computers in a flash of time. An interaction initiated between human and machine using NLP involves a few simple steps. They are capturing the audio, converting the input audio to text, processing of the data using a particular algorithm; the processed data output is converted to audio and prompted to the user.

Working principle of natural language processing

NLP uses algorithms to extract meaning out of the unstructured language data. The data which are essential are stored in a form that the computer can understand and use later. The techniques used can be classified into two classes namely

• Syntactic analysis: It focuses on the syntax. The main task is to check if the order of words aligns with the grammatical rules.
• Semantic analysis: The purpose is to understand the meaning of texts, if the sentence is structured or not, interpret it. Initially smaller chunks say words need to be checked followed by a combination of words to draw an inference if it’s a meaningful sentence or not.

Algorithms used in natural language processing

A lot of progress has been made with NLP but still, there is a lot left to explore, understand and improve. Human language is complex and tricky. This makes things difficult for NLP. There is a need for better algorithms to handle this ambiguity in human language. However, some of the commonly used algorithms are discussed below briefly.

• Bag of words: This algorithm is based on the frequency of occurrence of words i.e. on multiplicity. It is quite a simple algorithm with the idea that if the content is similar, the document will be similar. The disadvantage is that it doesn’t take into consideration the word order or grammar while analysing it.
• Tokenization: It is a segmentation-based algorithm. It divides running texts into chunks of words or sentences. These chunks are called tokens. For instance, a word is a token in a sentence, a sentence in a paragraph and so on. Its disadvantage is that it removes the punctuation which might lead to wrong interpretation.
• Stop words removal: Here all the common words that add negligible or no value to NLP are rejected. Articles and pronouns are usually classified as stop words as they don’t add up to the power of discrimination. However, some stop words removal might wipe out the relevant information in it.
• Stemming: It is a slicing based algorithm which intends to remove affixes (suffix or prefix) thereby reducing a word to its root form. The main idea is to remove inflexion i.e. different grammatical forms of the same word. For instance, “working” on removing the suffix “ing” reduces to “work” which is its base form. It is quite fast and simple but can go wrong sometimes by over or under stemming certain words.
• Lemmatization: It also works on reducing inflexion but with an emphasis on vocabulary and morphology of words. It aims to return the dictionary form of a word i.e. lemma. Thus, here words with the same meaning are grouped taking into account the fact that the same words might have different meanings with the change in context. It demands more computational power as it considers the knowledge of the structure of language.
• Top modelling: Hidden structures can be unveiled using this statistical modelling method. It processes individual words and based on their distribution assigns them values. It uses unsupervised learning to cluster documents into different groups or topics based on the text content. Among the top modelling methods, the most well-known one is the latent Dirichlet allocation (LDA). It is simple and quite fast.

NLP Applications

• Healthcare: NLP can predict the health condition based on database and patient’s speech as input. A realistic example of NLP is amazon comprehend medical service. Many mental disorders can also be detected using it.
• Accuracy of texts: Whenever you write a document in Microsoft word or use any software say Grammarly, some words get underlined, it gives you options to rectify it, also tells you the arrangement of words in a sentence is erroneous. All this is done with the help of NLP.
• Sentimental and emotion analysis: The main idea behind sentimental analysis is to understand the opinion of the customers. A body of subjective text that is, a text containing mood, feelings, and emotions (might be a sentence, paragraph or document) is required for analysis by NLP. It can even use the information available in social media to draw inferences about a person’s likes and dislikes be it a movie or a product review. This sentiment of the people is quantified by a positive, negative or neural value called polarity score. Both supervised and unsupervised techniques can be used for this.
• Cognitive assistant: NLP can work as a cognitive or virtual assistant by communicating with us the way we do with humans. As a first step, it will retrieve data about us and then can serve as an assistant in remembering things at the moment of need.

• Filter: Have you ever wondered how your emails in Gmail get classified into different labels say primary, promotions, social, and others. It is all with the help of NLP. The texts are analysed and then classified into different labels. This is how even spams or fake news are detected.
• Voice-driven assistants: Sitting in a cosy place, just with your voice command today you can make reservations, know the current news, play music or call your friends. This comfort is the gift of NLP that responds to your voice prompts. It is a boon for the physically and visually impaired ones as they can initiate many activities at home and workplace.