Improving Natural Language Processing Algorithm in Search Engines
Improving Natural Language Processing Algorithm in Search Engines.
Natural language processing (NLP) refers to the capacity of a computer or machine to accept, analyze and generate human speech. The ultimate aim is to make the interaction between humans and computers and human to human possible. Although in our modern world we are equipped with artificial intelligence ( A.I ) and machine learning devices, but lack of proper communication has always been a problem among people. We do not understand each other although we speak the same language. I have been working on a project so that we will be able to understand what we really mean when we interact with each other. If you pay a closer attention, you will know that this lack of communication has created war between governments, organizations, families, classmates, friends and couples. In my opinion we need to improve the natural communication among humans and create a reform in language structure. In this article one is going to have an overview on NLP, its classification, weaknesses and finalize the discussion on voice search in search engine.
Natural language processing can be very helpful in making flow of communication easier through devices and search engines. People are addicted to their iphones these days. Nobody pays attention to their surrounding anymore.If this bad habit of people continues, in a few years, they will forget the real sound of nature because they have these headphones on their ears all the time like deaf people who need to hear better. It is a bad habit and damages ears. When they become older, they will have issues with hearing. Does anybody for a moment shut that off and “LISTEN” to the sound of the falling rain, wind, birds singing? No. I see a lot of beauty around us, some amazing things are happening around us which can trigger innovation, compassion, sympathy, empathy and understanding, but nobody really looks at things anymore. They all look at their iphones while they are in car, walking, at the restaurants, in tubes (underground),with their families and loved ones etc.They go to the parties to get together, instead of talking, they keep looking at their iphones and browsing on social media.
Anyway!Sorry for digressing. NLP can play an important role in communication and natural language. Let’s explain in the following what i mean by that.
Computer, in its current form work with an artificial language but it cannot decipher natural languages. NLP aims to solve it. If the NLP becomes successful, the interaction between human to human and human to computers will be easier. This way machine will understand the input better, analyze it and provide a valid response in light to the interaction. All these have to be done in natural language processing.
To achieve this, there are numerous complex parts that should be put together. I am talking about a reform in language processing even in search systems. The main reasons that search engines cannot fetch the exact search result based on users search queries are two things:
A) If a user has never added a data about certain search query to search engine , a search engine cannot give an exact output about it. The program gives out a similar result but not an exact match.
B) A search query is confusing for a search engine.Some people don’t know what to type in search engine when they want an answer to a question. They either use too broad query or too long query. This confuses a search engine. Therefore the exact output cannot be shown in search results.
You see, NLP has been around for quite a long time and we develop applications and machines with NLP capabilities in order to make the flow of communication easier.
How is natural language processing used today?
There are numerous tasks with numerous approaches that provide numerous answers that NLP can help accomplish. Applying NLP has been successful in the following areas:
Fighting against spam emails
Spam emails remain one of the biggest challenges of the digital world. NLP can be used to set up a first line of defense for services like the Gmail,Yahoo,Hotmail; this way, the emails are filtered into 2 categories of useful/good or spam. NLP scans through the content of the email and recognizes spam.
NLP in Trading Systems
NLP is implemented in some trading systems which helps to determine and analyse stock exchange and help a user to decide if s/he should buy the stock or not. However in my opinion and based on my research, the NLP algorithm is not programmed right so some of these trading systems’ results are faulty and misleading.
NLP In search engines
Google search engine ‘s natural language processing is not perfect, but has been improved. To see NLP in action, ask Siri for direction, conversational search algorithm in Google and voice search. NLP and machine learning are also implemented in Google search engine. I believe within a couple of years Google search engine will be better. I have also tested Yandex in Russian and Baidu in Chinese, my money is on Google.
NLP in content marketing
The level of information on the Internet is massive to say the least and most of it are in article and large documents. The use of NLP has made it easier to get a summary of a content so that it will be easier for users to process the information.
Classification of NLP
Natural language processing is classified in two: the Natural Language Understanding (NLU) and the Natural Language Generation(NLG).
You really have to take a closer look at these two classifications in order to fully understand how NLP works. They are unique in their own way and are achieved using different techniques.
A) Natural Language Understanding
It provides meaning of the content which is given to the machine. Here is how it works: First the machine receives the hardest part of NLP. It has to convert the information received from natural language to artificial language – just like what speech-to-text or speech recognition does. When the content is still in text form, NLU comes in to try and comprehend on what the text means.
Almost every speech recognition software available today is based on the Hidden Markov Models. They are models capable of deducing what you said through mathematical calculations.
They listen to what you say, break it down into 10-20 milliseconds and run it against the pre-recorded speech to ascertain what each unit sound would mean. Putting together these units, it becomes capable of reading meaning into what you said. And gives output in form of text.
Understanding the information is the hardest. Although they have a lot of similarities, but It tries to identify the nouns and verbs; present and past tense etc in a process known as the Part-of-Speech tagging (POS). It also has lexicon coded into its system. Machine learning rules are applied to the natural language in the modern NLP algorithms to ascertain the most probable meaning of what was said.
The machine should have an understanding of what you said, or at least, very close to what you said. There are certain challenges faced in this situation for instance, some words might have similar meaning in other languages such as Chinese and the other languages in Asia and south East Asia. Developers mainly in English language have tried to input rules that help the NLU apply the correct words based on the rules.
Natural Language Generation
This is an easier task to accomplish. It helps create text from the computer’s artificial language and with the help of text-to-speech software; it can further convert the texts to audio files.
First, NLP converts the information to text. For instance, if you wanted to know about the Italian restaurant and searched the search engines, the computer will then carry out a research on Italian restaurants on its own, it will ascertain that you need information on food, Italian, restaurants, your local area etc.
It then structures the result of your query based on how it will communicate the information to you; similar to that of NLU but backwards. The natural language generation system can create complete sentences using lexicon and a set of grammar rules.
Finally, text-to-speech is brought in and the information will be read out loud. It makes use of the prosody model in analyzing the texts so it can determine pitch, duration and breaks. After this, the machine combines all the recorded phonemes to form one coherent string of speech.
I did some tests on Google voice search. The problem with voice search is that NLP ‘s algorithm is not programmed the right way and the search system does not understand different speech patterns and dialects. Google’s engineers should improve the algorithm so that it understands different speech patterns. This way it gives the right search results.
The future of speech technology might lie within deep neural networks.
Creating Innovative Neural Networks
The Future of Artificial Intelligence in Digital Marketing
The next big technological break
Hard copy:
The Future of Artificial Intelligence in Digital Marketing
Google Books:
The Future of A.I in Digital Marketing