What is Natural Language Generation (NLG)?

What is Natural Language Generation (NLG)?

NLG stands for Natural Language Generation. Every time a machine produces human language instead of data, NLG is at work. Business Intelligence tools like dashboards and reports, voice assistants, chatbots, automated customer service messages, and even predictive text all use NLG. The human language generated by machines is often far more useful than raw data.

Natural Language Generation (NLG) is one half of the natural language processing (NLP) package. The other half is natural language understanding (NLU). The purpose of NLU is to enable a machine to understand what humans mean. The purpose of NLG is to enable a machine to generate text or speech that others can read or hear. To summarize, the chatbot that understands you is NLU, and the chatbot that writes back to you in full sentences is NLG. Almost all conversational AI today combines NLU and NLG.

Two ways NLG can generate text

Not all NLG is created equal. The key differences are worth covering.

Extractive NLG: This form of Natural Language Generation uses the same words and even phrases as found in the input material. For reasons of safety and compliance this form of NLG is mostly used in situations where absolute precision of wording is required. Legal and even financial texts are examples of this kind of NLG. Examples of output of this kind of NLG are summaries of legal disclosures and of regulatory updates and even contract clauses.

Abstractive NLG: Here the model actually generates new text and thus can be used for very general tasks like Summarization, Content Generation or even chatting with users. For those tasks typical transformer models are used. In most cases nowadays so-called Large Language Models are even used directly for the NLG task.

Almost all modern generative AI tools work by abstract generation, as this is the only way for them to generate human-like output when they are faced with input that has never been seen before in the same words.

What actually happens behind the scenes

NLG doesn’t generate text in one step. It typically moves through a short pipeline before producing a final output.

Identifying the key information in the input data such as the main subject and the relations between the input data points. Interpreting the input data and generating meaning and context around it using NLU techniques for understanding human language. A planning stage for deciding on what information to communicate and the overall structure of the generated information. A more detailed planning stage for deciding on the individual sentences and how the paragraphs should be set up. The actual generation of the text in the final stage.

NLG has traditionally been achieved using one of four methods: templates, rule-based generation, statistical models and deep learning models. The use of templates to generate copy within fixed boundaries is obvious, but it has its limits. Rule-based systems, which apply ‘if-then’ rules to data to generate human language, can be encoded by domain experts to enable generation of complex copies such as weather reports. Where the temperature is below freezing, for example, the system can always generate the copy ‘below freezing’. Statistical models learn from large datasets how language functions and enable generation of new copy by selecting words and syntax from the trained data. However, such models require large amounts of training data, and can therefore only be used for broad applications. Deep learning models, such as recurrent neural networks, have increased the fluency of NLG, by processing language sequentially, one token at a time. And it is the transformer architecture, at the heart of ChatGPT and most other modern generative AI, that enables the highest quality NLG by using self-attention to follow long passages of text and understand the relationships between them.

NLG at work across industries

The most practical description of NLG for those interested in learning more about how Natural Language Generation works and where it can be put into practice.

For example, finance and operations teams use NLG to create reports from dashboards and analyze numbers within a summary that can be read by managers in a very short period of time and save them hours and hours of work that would have been required by analysts in order to create reports and summaries for their managers. Customer Support teams use NLG to create responses to customers’ inquiries in natural language. NLG is also used in Voice Assistants such as Siri and Alexa in order to translate intended human language to spoken human language, and then use NLU on human speech in order to understand what was said by humans in conversation.

NLG in Finance & Operations. Reporting analysts can now turn dashboards and long lists of numbers into reports that can be read by managers in a matter of seconds. Customer Service chatbots can deal with initial customer inquiries in a natural way before handing over to a human when required. Voice Assistants such as Siri and Alexa use NLG to translate intended responses into natural language that can be read by users. For machine translation between languages, NLG is used to create a very natural human language from the interpreted source data, with human translation reviewers checking the final output for accuracy. NLG in E-commerce & Marketing. NLG is used for mass production of product descriptions for thousands of products in online retail catalogs as well as first-drafts of emails and social media posts. NLG is used after sentiment analysis of customer reviews and feedback in order to automatically produce reports summarizing key points such as customer-perceived delay in delivery having dropped 8% this month. NLG in Financial & Legal sectors. NLG is used for Extractive purposes, i.e. to automatically extract portions of disclosures and other long compliance summaries from large databases of pre-written content, with rigid control over word choice in order to stick to approved wording.

Why businesses actually care

The benefits of NLG extend far beyond that of simple writing.

NLG can also generate reports quickly. There is often a big difference between how much time an analyst needs to spend to write a report and how much time they have available. Furthermore, NLG can generate a lot of content. For example, product descriptions for thousands of products, personalized confirmations for orders, and individually tailored recommendation texts. As well as enabling faster decision making by finance, operations and management, NLG can also enable organizations to generate vast amounts of content that would be too time-consuming for a team of human writers to produce. When used in combination with NLU, NLG can also enable self-service tools to provide natural language human-like responses to customers and clients.

By maintaining a constant writing style, NLG systems can offer a number of important advantages to businesses. For one, all output will be consistent in terms of tone and structure, which is critical in industries such as finance where even slight differences in wording can have major consequences. Secondly, NLG systems are able to write in a language that is native to a region, without the need for a native speaking writer. In this way, a company can easily generate large volumes of content in multiple languages, opening up a whole new market of customers.

The gap between fluent and correct

One very important distinction to make is that of fluency vs. accuracy. The vast majority of current NLG systems are designed to trade off some amount of accuracy in order to generate much more fluent output. (Indeed, most state-of-the-art models are capable of “hallucinating” facts that aren’t actually supported by the underlying data, and so need to be checked against the original source(s).) For high-stakes applications such as legal or financial writing, therefore, extractive NLG is generally to be preferred over abstractive NLG — not because the latter cannot generate accurate output, but because there is much greater risk that fluently-written but factually incorrect output will be misleading to read.

While these forms of NLG can generally be reliable within their designated parameters (i.e., report templates and the like), they are generally not designed to account for unforeseen circumstances that would necessitate divergence from a predetermined format. This is especially not to be confused with the risk of a system ‘hallucinating’ and generating content that is not only inaccurate but also sounds very plausible. While strong NLG systems can produce content that reads very naturally and thus ‘sounds right’ to the human reader, they can easily fail to hit the correct tone and thus are generally best reviewed by a human for higher-stakes content such as customer communications and financial reporting.

How it connects to NLP and NLU

When talking about NLP, people often mix up the 3 terms. NLP (Natural language processing) is the larger field of language based AI, NLU (Natural language understanding) is half of NLP and deals with the interpretation of human language, and NLG (Natural language generation) is the other half of NLP and deals with generation of human language based on input (data, intention etc).

A simple example for the working together of NLP, NLU and NLG could be predictive text on a mobile phone. NLU is used to predict the user’s search for the best words to complete a sentence, and NLG is then used to select the best words from the available choices to complete the sentence for the user. An example of a chatbot in customer service could be another application of NLU + NLG. The NLU is used to understand the customer’s complaint, and the NLG is then used to automatically respond to the customer.