As businesses continue to tackle digital disruption, the search for technologies to leverage the value of the vast volumes of data being produced is gaining serious momentum. Naturally, artificial intelligence (AI) springs to mind when organisations are looking to unlock new data-driven opportunities—but with a concept so immense, where do you start?

Quickly emerging from the AI pack is natural language processing (NLP). With its roots going back to the 1940s, it’s not a new concept, however NLP has made great strides in giving machines the ability to recognise and understand language—the most fundamental form of human intelligence. Today, its resurgence can be credited to a combination of the following factors:

  • decades of advancements in NLP techniques from academia and industry
  • faster and cheaper compute
  • increased availability of large-scale training data
  • democratisation of these techniques through open source tools and frameworks,
  • and the drive to better manage the deluge of data to drive competitive advantage.

The enterprise opportunity of NLP

NLP technologies tackle the complexities of human language, empowering organisations to process and bring order to their vast volumes of unstructured language data - yielding insights that would otherwise not be possible. The digital era has ensured that every action taken within an organisation creates more data—from customer reviews, internal documents, knowledge bases and regulatory workflows, through to customer behaviour and marketing campaigns. All of this information can be used for commercial advantage. Yet understanding and extracting meaning from this vast amount of language data—whether text, voice, or video—is beyond the cognitive capacity of the human mind.

The traditional database brings order to structured data—where columns have labels and data is classified accordingly—allowing the use of analytics and business intelligence tools to make sense of everything through description, visualisation, and prediction. The primary challenge and subsequent opportunity for organisations is the vast amounts of unstructured data, which traditional approaches to analytics cannot be directly applied to. When you consider all the information captured via queries, email communications, social media, chat bots, videos, customer reviews, and support requests, it’s no surprise that most pundits estimate at least 80% of organisations’ data to be unstructured. NLP enables you to process, analyse, understand, and make predictions with your unstructured language data to extract real, actionable insights.

The applications of NLP in the enterprise

From document retrieval, classification, and similarity, to trend analysis and fact extraction, NLP technologies support a wide range of business applications.

Information retrieval and extraction

Information retrieval enables the use of plain language search queries to quickly retrieve documents from large collections of heterogeneous document types, improving the discoverability of an organisation’s internal documents. Information extraction enriches workflows and processes by bringing structure to unstructured text through the extraction of entities - such as the names of people and organisations and the relationships between them. This information can be used to populate knowledge graphs which can be linked to other data sources, improving an organisation’s collective knowledge.

Use cases:

  • Fast retrieval of documents from internal knowledge bases
  • Customer support
  • Fraud detection
  • Document workflow augmentation

Document similarity

By building a structured representation of documents, NLP techniques can be used to identify other documents with similar content or characteristics to a target document of interest. This can be used to build a “more like this” feature, where a user looking at a specific document can explore documents identified as having similar content.

Use cases:

  • Legal discovery
  • Patent landscape mapping
  • Cross checking support cases
  • Improving discoverability of knowledge bases

Document classification

Document-based workflows often feature a fixed set of document types that are specific to the organisation or its industry and determine the path the document will flow through. Classification techniques can be used to automate the assignment of document types which can greatly improve the efficiency of their ingestion. This can also be applied to correspondence, such as customer support emails or transcribed voice recordings, in order to predict which customer support agent or department an inbound query should go to based on the support request. By routing it to the right person the first time, you can achieve considerable time and cost efficiencies.

Use cases:

  • Customer support
  • Patent classification
  • Document workflow augmentation

Emerging trend detection

NLP techniques such as topic modelling enable the discovery of topics in the form of frequently seen patterns of words in a collection of documents. These techniques can be used to better understand nuances in the contents of a document collection and also identify new trends that are emerging in a continually changing source of documents or texts. This can be useful in pinpointing significant changes or identifying topics in customer interactions and discussions that could indicate previously unknown issues in products or services, or new avenues of service/product offerings.

Use cases:

  • Social media conversations
  • Call centre/customer support trends
  • Product creation or expansion opportunities
  • Understanding product or service reception
  • Detecting customers at risk of churning

Sentiment analysis

Measuring levels of positive, negative, and neutral sentiment expressed within text is a powerful tool for organisations looking to better understand their customers. This is particularly useful when applied to consumer-generated text such as reviews, survey responses, call-centre transcripts and social media content.

Use cases:

  • Understanding sentiment expressed in user reviews
  • Analysing free text fields in survey responses
  • Social media conversations
  • Call centre/customer support trends
  • Understanding product or service reception

Towards NLP at your organisation

The above use cases demonstrate how the introduction of NLP initiatives has the potential to yield a considerable amount of value. But where do you begin? Some of the key considerations leading to NLP adoption are the following aspects of your organisation:

1. Business strategy

As with the adoption of any form of new AI technique, the first step is identifying the potential value for your organisation and developing a business case. This requires a solid understanding of your business strategy, your organisational workflows and the end-to-end value chain across your organisation. Your business strategy will impact the type of NLP application you might look for.

  • For cost-reduction/efficiency gains, applications that can be used for workflow augmentation – such as document classification, information extraction, and information retrieval – may be valuable.
  • For improvements to your core product or service, enhancing your capability to gain insights into customer understanding through topic modelling, sentiment analysis, and keyword extraction could be the right approach for you.

2. Available data

A fundamental determining factor in the NLP techniques that will be available to your organisation is the data you have access to in terms of its type, quantity and quality. NLP techniques that involve prediction, such as document classification and information extraction, make use of supervised machine learning techniques. This requires a sufficient amount of data labelled with the correct value that is to be predicted in order to develop, train, and test your models. Most of the time, unless an organisation has made an explicit effort to capture such labelled data, this won’t be immediately available. As a result, projects involving these types of NLP use cases need to factor in a component of data annotation, which may take some time. When a compelling business case has been identified, investment in this process is likely worthwhile.

3. Level of AI maturity

If your organisation is relatively new to adopting AI and machine learning techniques, it may be preferable to target shorter projects that are more likely to yield quicker wins. Initial execution of successful use cases that prove the value of AI and machine learning initiatives are important for building trust amongst your stakeholders and decision-makers. In this situation, a pragmatic approach may be to make use of unsupervised techniques that do not require labelled training data. As an example, topic modelling and clustering can be used for trend detection and to improve the understanding of the contents of a document collection by detecting frequent patterns.

Executing NLP contributes to:

  • an improved understanding of organisational data and goals
  • efficient resource management, and
  • data-driven analysis for more effective decision-making and forecasting.

Whatever the motivation, adopting NLP requires (1) an engagement process to understand existing organisational workflows and (2) identification of opportunities for efficiency gains and knowledge discovery. If you don’t have the right data to support high-value NLP initiatives, look at strategies to retrospectively add structure to your data and adjust internal processes to ensure it is captured effectively moving forward. Equally, aligning business value to metrics and defining what success looks like will ensure ROI can be measured and improved over time.

Engaging a partner well-versed in the design, implementation, and ongoing management of an NLP program will not only help you identify valuable NLP use cases and solve challenges around data acquisition, but will also help you adopt further NLP and machine learning initiatives as your platform capabilities mature.

If you’re ready for a modern-day approach to NLP, contact us to start the conversation.