The third coming of Natural Language Processing

“SAP tried to introduce natural language processing based BI tools about five years ago and failed. Why would I use yours?”

Yesterday I was explaining to a customer that the QuickLogix natural language query engine would make it easier for his business users to ask questions and make meaning out of their data. Being the IT Director of a multi-billion dollar company, this was a question I was expecting him to ask and he didn’t disappoint! So why indeed does Gartner project that Natural Language processing is the next big thing in the world of data analytics and business intelligence? Why- if it has been tried before- not 30 years ago- but barely 5 years ago- and it didn’t really take off then?

It boils down to one major tech Trend in the past 10 or so years and one major Event in the past 3 years.

The TREND
I remember the days when the leading edge of innovation was done in the Enterprise world and the benefits bled into the consumer market. Sometime around the mid 2000s, perhaps with the introduction of the iPhone, that trend started reversing. Consumer products and requirements were on the leading edge of technological creativity. All things new and exciting in the enterprise world (cloud computing, SaaS products etc.) are dictated bleed-outs of the consumer market. More mobile devices meant more data being transferred(volume), more content being generated (variety) and more demand for quick turnaround on data accessibility and processing (velocity). Yes the familiar 3Vs of Big Data are a direct result of demands in the consumer market.

THE EVENT
When it comes to natural language processing, I like to think of the world as pre-Siri and post-Siri. Apple introduced Siri to the world with the iPhone4S in October 2011. Ever since, there has been a renewed focus among all other phone OS manufacturers to provide (or improve upon) a similar service. Google has been around a long time with their ground-breaking natural language search. However it is the advent of Siri that has set the average consumer expectation that all interactions- personal or otherwise- can (and should) work by using simple English.

The Trend and the Event together have subliminally revolutionized the mindset of the workforce. More and more business specialists and users are becoming inclined to use natural language in their work. The mobile evolution will serve as a potent catalyst for the acceptance of NLP by business users in their everyday functional tasks. The challenges of training them to ask the right questions and make meaning out of the results will remain. But the adoption of the technology in itself? It was tried in the 1950s & 60s, in the late 1990s and early 2000s- but in this third coming- natural language processing is here to stay.