Differentiating NLP and NLG Conversations With Quality Responses

A Business Model for Improving NLP Customer Experiences

In my last post, Natural Language Processing Augmented with AGI, I provided a little insight into how services like Alexa work and hinted at the importance and hence projected growth of this market segment. I closed with:

Stay tuned, there’s more writing to come on the topic of supporting artificial intelligence deep learning and natural language processing and generation supplemented with a general intelligence REST application programming interface and how such an API could be implemented in the very near future…

Let’s get the acronyms and definitions out of the way before starting.

  • AI: artificial intelligence
  • AGI: artificial general intelligence
  • DL: deep learning, the domain of building neural net computer models to solve what are effectively pattern matching problems
  • NLP: natural language processing — in this writing, dependant on AI DL models
  • NLG: natural language generation — in this writing, dependant on AI DL models
  • REST: representational state transfer — a common request/response API for web services
  • API: an application programming interface implementation
  • “black box” “a device, system or object which can be viewed in terms of its inputs and outputs (or transfer characteristics), without any knowledge of its internal workings” Wikipedia
  • SLA: service level agreement, or the performance definition a service has established to be measured by, or performance standard its operations are defined to operate within

This wiring should be considered a conversation, an exploration of one possible approach for supplementing existing NLP providers with information to enable them to address their AI DL NLP responses that exceed the DL NLP training in order to provide improved NLP-based customer responses and experiences. Granted, the initial implementation of this hypothetical AGI REST API service is not AGI. I want to be clear about this, yet, as an AGI API “black box” it does accomplish:

  1. Providing NLP providers with an AGI REST API “black box” service that would enable improved responses to their customer dialogs, a small set at first, increasing with time
  2. NLP providers would have something from which they could begin to answer the question, “How would we integrate an AGI REST API into our NLP models?” if only as an initial exploration, say a cost/benefit analysis
  3. The data and resulting analysis will enable a type of AGI development at a faster pace while also providing increased NLP quality at a decreased cost due to the economies of scale, data provided from the NLP ecosystem, inherent in this hypothetical implementation. In other words, this basic start would fund and eventually enable a form of an AGI

Since writing the last post, there has been a new article almost daily about the NLP space. I’ve also reached out to my contacts at the large “AI as a Service” providers to learn and to understand where they are with improving NLP results, and to bounce off for their feedback this hypothetical approach. The results were interesting. I’ve learned:

  1. More than a few contacts quickly went dark, although not all. Hmm. A hearty “Thank You!” to those who responded and engaged with my request to chat.
  2. There is a focus on what is considered to be lower hanging AI need that could more readily be solved thereby generating a more immediate AI-based revenue stream. In other words, this isn’t an area they deem an important business differentiator right now.

In this post:

  • I’ll share the data from the past few weeks of research under the arching theme question, “Is there enough market need to justify an affordable solution for NLP providers where their customer conversations exceed their AI DL training with the goal of supporting improved customer experiences?”
  • A hypothetical “Phase I” AGI REST API business model that would address a small subset of conversational “generalization” issues, yet is staged for scale
  • The “Phase I” AGI REST API model
  • Wrap it up with a conclusion and some additional supporting thoughts

The NLP Market: An AGI REST API Demand

The NLP market covers a large number of industry verticals, from banking to medical, to fun, just about every aspect of daily activity, which means NLP is a huge market.

Natural Language Processing market is estimated to be worth 16.07 Billion USD by 2021, and $22.3 Billion by 2025.

There is a lot of R&D going into the NLP space. Understanding the nuances of language is AI’s next great challenge. And make no mistake, understanding the nuances of language is the center of the target for this hypothetical AGI REST API service as well.

Hugging Face goes so far as to suggest as part of their vision alignment:

“…everyone will talk to an AI everyday in 5 years…”

Does that imply in 5 years Hugging Face expects 7.5 billion people to be interacting with NLP at some point or another across their day? 7.5 billion is a very large total addressable market. Which may explain why there is so much investment chasing NLP. Below is a link to an article listing VC’s and investment angles seeding and growing this domain, appropriately titled “Follow The Money…”. Be sure to check out “Appendix I: CB Insights overview of AI M&A activity” toward the end of this piece — very eye-opening.

Follow The Money: Meet The Investors Of Conversational AI

With regard to chatbot market growth:

…(chatbot) presence is only predicted to grow, with Gartner saying that 25% of customer service operations will use chatbots or virtual customer assistants by 2020.

Businesses using chatbots report increased customer satisfaction — but isn’t a good customer experience one with a personable, well-trained human involved?

Love Or Hate Glitchy Chatbots, There Are Big Benefits For Small Businesses

Of all the verticals either using or planning to use AI NLP, the banking sector is embracing the technology with impressive vigor, and for good reason. Check out this very brief overview of AI and NLP in the banking sector:

Millennial consumers are more comfortable than older generations in contacting their bank or credit union without having a conversation with an actual human. 90% of the Silent Generation (born 1925–1945) have a preference towards human service over the phone, while only 12% of Millennials prefer phone, with nearly all others looking for chat, social or text channels.

Finn AI is a Canadian company that provides financial institutions with a chatbot that integrates into legacy core systems and conversational apps such as Facebook Messenger. They are currently working with tier 1 and tier 2 banks across four continents, to automate hyper-frequent front office tasks and chop the time support staff spend on such activities in half.

…you can expect to see more financial institutions roll out conversational interfaces that improve the consumer experience

Artificial Intelligence and The Banking Industry’s $1 Trillion Opportunity

Clearly, NLP is the present and future customer experience for banking. Yet, as aggressive as the banking segment is with NLP, there are very real limits to NLP response capabilities which will impact customer experiences. AI NLP responses are constrained to their training. When customer conversations go beyond the AI NLP training, NLP responses are constrained impacting customer experiences. There are a few references below that address the limits of today’s NLP.

To get a good idea as to where NLP is today, check out the NLP Progress website.

A Financial Model for a Hypothetical REST API Service

Let’s build a financial model for our hypothetical AGI REST API that:

  1. Illustrates a price, margin, and cost for this hypothetical AGI REST API service
  2. The “dials” of which can be applied to a multitude of implementations, whether subscription, pay per use, or some other revenue model.

Since we are just starting, let’s start with something really basic, trying to keep this initial model as simple as possible. Let’s say the consumer of the API pays per second of request time. Let’s define the consumer of this hypothetical AGI REST API service. The service is intended to provide human insight into conversation data that exceeds NLP training data, therefore consumers for the service are NLP implementors.

The usage scenario for for the consumer of this hypothetical AGI REST API, as an NLP solution provider, is where the NLP process has reached a point in its workflow that it has determined that the “conversation” has strayed beyond its training and desires additional insight into the conversation in order to respond to their customer with something meaningful in order to provide a great customer experience. In this proposed implementation, the NLP process would post a request to the hypothetical AGI REST API. As a result of consumers who are time constrained for a response, the NLP requestor is waiting so that it can complete its task, let’s, therefore, create a completely arbitrary, but somewhat reasonable service level agreement (SLA) response time of 3 seconds. To keep this model as simple as possible, for now, let’s say that the service charges two pennies per request second. So the max amount an AGI REST API customer could expect to pay, per the SLA, two pennies times three seconds, or six cents per request.

Let’s also say that our cost for providing the response is one cent per request second. That yields, in this overly simple model, a margin of one cent per second.

This is a good point to reference the service implementation model explanation below, as it illustrates how the hypothetical service would drive down the consumer connection cost. The penny model below is meant only to illustrate a hypothetical revenue model as a baseline for future exploration.

Just to review the basic model as a starting point for our conversation:

  • Revenue: $0.02 / request second
  • Cost: $0.01 / request second
  • Margin: $0.01 / request second

As we all know, there are 864,000 seconds in a day. I’ll let you do the math for a year. For now, this simple model of two cents/second usage, one cent/second cost, and one cent/second margin is enough to lay the groundwork for whatever variation on this theme seems appropriate. Let’s say you hold true to the one cent/second margin, and varied the charge and cost dials. At a one cent/second margin, this hypothetical model “feels” like a relatively strong business to be in at the scale of thousands of requests per second.

Let’s say this hypothetical entity was able to engage and help just 1 NLP provider in its first year, say an entity like Hugging Face. Hugging Face has over a million messages per day. If Hugging Face used this hypothetical service for only 1/100th of those million messages at the three second SLA, the cost to Hugging Face at two cents per request response second, it would equal 2 cents/second X 3 seconds per the SLA X 10,000 messages (1/100 of the million daily messages) or roughly $600 for that day. That cost would be expected to drop significantly with time. See the section “Implementation Thoughts…” below for how the cost reduction would happen. It might be expected that after the first year, the cost is dropped 1/10 to $60 per 10,000 requests due to the AGI REST API gained efficiencies. That is a great deal to Hugging Face to help them provide improving message responses to their customers supporting their 10x (from a job posting for a Hugging Face software engineer) improvement goal. In fact, this hypothetical entity might enter into a sliding scale agreement, where the cost per request is guaranteed to be reduced over time.

Why would Hugging Face consider using this hypothetical service? The answer is in their software engineer job description:

“…Your task will be to build on this system and make it 10x smarter.


The Hugging Face goal is to improve their responses by 10X within a technology with very real “generalization” constraints!

More from Hugging Face:

“I hope that the chatbot (Hugging Face) is going to get better over time as the company can start aggregating conversation data. This is what could turn Hugging Face from a great first-time experience into a lasting friendship.”


Suffice it to say, Hugging Face has expressed a solid reason for improving its NLP responses. Hugging Face needs to turn initial first time-timers into “lasting friendships.” The above $/day hypothetical scenario seems like a reasonable amount to help Hugging Face achieve their 10x improvement goals, turning their first-time experiences into “lasting friendships.”

Hugging Face is only one entity desiring to improve their NLP return values in a sea of NLP differentiating need. Their goals are very real and very hard as venture capital funded startup so they have a very real incentive to leverage what they can where they can in order to succeed in their goals.

Alternatively, consider CapitalOne who recently built their own NLP solution.

“…Eno, Capital One’s intelligent assistant, we decided to build our own natural language processing (NLP) technology. It was important to be able to build Eno on a platform that deeply understands financial services terms, allowing us to deliver the best possible experience to our users. Capital One is committed to delivering a human-centered experience each and every time we interact with a customer.

Capital One’s motivation for using an affordable service to help them accomplish their goal on an as needed bases (the hypothetical on-demand AGI REST API) is pretty clearly stated above, “Capital One is committed to delivering a human-centered experience each and every time we interact with a customer.

The point should be made with these two examples that NLP providers will benefit from utilizing an on-demand service when needed within their customer interactions, which is the segway into the point of this financial model.

Consider the model above, where we started with Hugging Face and then added CapitalOne. If the above arguments for an AGI REST API service utilization at an initial price of two cents/request second makes sense, then the hypothetical entity goal for the number of requests per second might conservatively look something like:

  • End of year 1: Averaging 10 connection requests per second with an average margin of one cent per second
  • End of year 2: Averaging 20 connection requests per second with an average margin of one cent per second
  • End of year 3: Averaging 100 connection requests per second with an average margin of one cent per second

And so on… The “so on…” at the end of year 4 with an earnings multiple is where venture capital makes money. By year 4, after refining its supporting processes and augmenting responses with AI based on the analysis of request data from the prior 3 years which would enable it to reduce its pricing structure and grow its consumer base, this hypothetical AGI REST API service would be expected to be providing thousands of responses per second.

Let me leave the financial model exploration with one thought. Another way to look at this service is similar to the land banking investment model. It might just be that the value in this hypothetical AGI REST API entity is the long-term play, not just the short-term revenue noted above:

  1. Being first to the AGI market with a business model that is good to go “today” (or as soon as the infrastructure is built) supporting an industry that poised to grow into the future
  2. The underlying data it is collecting which enables it to grow and provide its resulting automated responses across its consumer ecosystem

The Hypothetical “Phase I” AGI REST API Model

The concept is to start “simple” by building an approximation of where we want to get to with a model that works today. Since AGI is the representation of human intuition, the approximation of the AGI is proposed to be the human element the model is eventually meant to model. The idea is to wrap the human element in an API so that to the API consumer the responses look like what they would expect from an AGI implementation.

Hypothetical REST service API for human insight responses

Over time, as the request data builds, it enables analysis and aggregation of request types for future response automation — driving toward a true AGI with time.

The bubble “t=1” illustrates that the initial set of responses will be a focused subset of conversational context possibilities. With time, the subset will grow to “t=2” and then “t=3”.

The above three graphics illustrate that the initial set of API request responses will initially be scoped to a very small subset set of conversation “types” to ensure initial success while providing value to the service consumers. This both ensures initial success and enables data to determine where to grow. Over time, with additional conversational request data, the subset will grow, t=2–3 above, providing greater value to the NLP REST API consumers.


The NLP market is:

  1. In its infancy
  2. Is highly competitive with providers looking for ways to differentiate through product/service conversation quality to support improved customer experiences
  3. Very, very large AI segment spanning financial services to fun and games, to unique stand-alone services (Hugging Face as an example)
  4. And yet, NLP is only one vertical in the AI space that is constrained with the “generalization” problem. Which means, there are other hypothetical consumers for an AI “generalization” API solution, for example, the self-driving car segment

Within the AI domain, it has been stated by some of the most respected leaders in the space that a generalization solution, AGI, is at least 10 years out. While the reason’s for the 10-year AGI solution horizon are reasonable given what we know today, that shouldn’t stop an implementation that provides a response that NLP providers would expect from an AGI API.

The pitch for this hypothetical AGI REST API entity might look something like:

We’re solving the generalization problem chatbots have with conversational context, the information they need when their customer conversations go outside of their AI neural net training so that they can respond appropriately to their customer, resulting in improved customer experiences.

Our product is designed for the NLP and chatbot providers of the world, for example, the CapitalOne and Hugging Faces of the world.

We make money on each hypothetical CapitalOne or Hugging Face request for context per second to our AGI REST API.

Our team is the right right team because we have expertise in providing human-based context insight through an online request response REST API service.

I’ll leave this post with a great AGI overview to consider from two of the AI leaders:

Geoffrey Hinton and Demis Hassabis: AGI is nowhere close to being a reality

AGI may be nowhere close to being a reality, just yet, although possibly sooner than 10 years with the right creative people, implementations, and investments.

Implementation Model Thoughts and Examples

In the spirit of conversation, I thought I would add a few examples for consideration to help frame this hypothetical service.

Microsoft gathers data about Windows 10 and device and application compatibility across its participating ecosystem. Microsoft then uses that data to:

  1. Understand where it, Microsoft, needs to address Win10 issues
  2. For participating enterprise customers, the Analytics services provide this data to help the participating enterprises IT customers manage Win10 upgrades across their device and application landscape. The device/application insights are based on ecosystem data.

The entire participating ecosystem gains from the insights that result from this shared underlying data.

Sift Science provides a conceptually similar ecosystem model to help its customers with fraud risk. Sift Science leverages data from across its ecosystem to provide fraud risk analysis within milliseconds of a customer request.

“Our models learn from real-time data received from thousands of sites and apps across the world…”

Sift Science provides an additional relevant model. Their AI model is able to utilize a humongous amount of data and then provide a fraud risk value within milliseconds of the request, making the Sift Science infrastructure model worth understanding and learning from. Any proposed AGI REST API will need to approach similar performance metrics.

The two examples above illustrate services which would be cost prohibitive for a single business to take on for its own benefit. Rather, by providing the service for the ecosystem, costs are reduced such that the entire ecosystem benefits.

A Software as a Service (SaaS) that provided enterprise expense reporting services enabled its customers to simply provide scanned/digitized images of receipts that were then magically converted into data that their corporate reporting systems could consume. The SaaS expense reporting service had found that optical character recognition (OCR) had certain limiting issues. This SaaS service took the scanned receipts and had humans read the image and re-enter the characters into the computer system. The customers using this SaaS enterprise expense reporting service only cared that their process was made easier because they only had to provide scanned images and the service would auto-magically convert those expense report detail images into usable data for their corporate reporting systems.

In years 1 & 2, the hypothetical AGI REST API entity might work with only a few NLP partners to refine the initial context response set, and then grow its consumer base as it grows its response set. It seems reasonable that as the underlying data from across the NLP ecosystem gets larger, analysis of the data provided by the requests would enable things like request/response types, and hence automated responses in place of what was human insight. These insights would reduce response costs, which would be passed on to the hypothetical AGI REST API consumers.

There is one other interesting element to this model being the underlying human provisioner of insight. This model leverages humans to provide insight into the requests. One could envision this human insight service growing both through the NLP space as well as across the AI space through this “black box” API implementation. As automation replaces certain well-understood requests, the demand set would be expanded, maintaining the need for human insight. Perhaps in 10 years, a true AGI would exist. There might still be a need for human auditing. This is a true human/technology solution blend, not a human replacement solution.

“Cheers!” “乾杯 / Kanpai” “Salud!” “Prost” “Salud!” “Santé!” “건배 / Geonbae” “skål” “Gesondheid” “gānbēi” “Υγεία / Yamas”

read original article at https://medium.com/@mpdickson2/differentiating-nlp-and-nlg-conversations-with-quality-responses-48bbf6475f19?source=rss——artificial_intelligence-5