Perceptyx Comment Analytics Capabilities

Introduction

This article gives you an overview of comment analysis at Perceptyx. We use natural language processing, which is a branch of artificial intelligence focused on understanding written text, to do three things: theme detection, intents detection, and sentiment analysis.

Below you will find the problems we are trying to help you solve, how our approach works, how it compares to others in the industry, and what it looks like in practice when you use it to uncover insights and make better decisions.

What does comments analysis do?

The open-ended, verbatim comments people leave on a survey are a rich source of information. They are the direct, unfiltered voice of the respondent.

That said, reviewing text data has traditionally been an arduous, manual process. Someone has to read through every individual comment to understand what a group of respondents is writing about. That is time consuming enough for a direct line manager. For human resources and people analytics leaders who have to report on an entire organization, reading all the comments is essentially impossible. This is where artificial intelligence tools come in, to turn an otherwise impossible task into an easy one.

Theme detection, intents detection, and sentiment analysis together address a critical question: how do you understand what issues employees are talking about, zoom in on the most critical ones, and decide on the right action items, without painstakingly reading every comment?

Comment analysis gives you a way to parse through the sea of comments in an informed manner. You navigate the data using ad hoc combinations of topic filters, which tell you what the comments are about, and intent filters, which tell you how those topics are being talked about. That might surface comments offering praise on certain benefits, or comments suggesting how management could improve. The sections below walk through how it all works in more detail.

Our approach to comments analysis

Perceptyx takes a multi-prong approach to comments analysis so you get a comprehensive picture rather than a single angle. The three approaches are:

Theme detection

Theme detection refers to a set of methods for mapping comments into a predefined set of topics. Out of the box, we offer two approaches.

Custom theme detection (Lexical Detection)

Custom theme detection is a highly flexible way to find themes in comments. With this method, the themes we search for are based on a list of keywords and phrases. This approach enables customers to customize, update, create, or delete the themes and keywords applied to their data.

Perceptyx provides a curated set of 38 default themes related to employee engagement, covering things like employee benefits, safety, trust, employee recommendations, and response types. These default themes can be expanded over time as new issues and areas of focus emerge, informed by feedback from internal and external stakeholders. Customers can also tailor the themes and associated keywords to reflect the language, terminology, and priorities of their organization.

Standard theme detection (Supervised Model)

Standard theme detection is a highly accurate deep learning method built on a vetted set of predefined themes. Rather than relying on keyword and phrase matching, this approach fine-tunes a deep learning model on a large volume of labeled comments from across industries. The model learns the common language patterns and the meaning behind each predefined theme, which lets it categorize comments with greater accuracy.

One thing to note: the Perceptyx standard theme detection model is trained to detect our curated set of 38 employee engagement themes. Its accuracy on those 38 themes is excellent, but it cannot be extended to detect custom or company specific themes. If you need themes that are unique to your organization, custom theme detection is the one for that.

Intents detection

Once comments have been categorized into themes, you will often want to know what comment types are showing up inside a theme, so you can understand the discourse around it. Knowing that benefits came up in a large fraction of comments is a useful insight on its own. But are people describing what they really enjoy, voicing general preferences, making recommendations for improvement, highlighting pain points, or detailing real frustrations with company benefits? That is where intents detection comes in as a deep dive tool for exploring individual themes.

Independent of which theme detection approach you are using, we categorize each comment into one of five intents:

The intents categories are inspired by a similar model in cognitive psychology used to measure and rank attitudes toward a topic or concept. Each intent captures a specific framing or emotional emphasis. The aim is to pick up on emotional indicators alongside satisfaction and action indicators, so you can easily filter and isolate comment types within a theme and narrow a large theme down to a focused subset of similar comments.

Our intents detection model is a machine learning approach trained on the same backbone as the standard theme detection model, so it offers the same high degree of accuracy.

Sentiment analysis

Sentiment analysis is a model that identifies the polarity of a comment, on a range from highly negative to highly positive. It does this by mapping each comment into one of three categories that do not overlap: negative, neutral, or positive.

The Perceptyx sentiment model is specifically designed and fine-tuned for employee feedback, helping it better understand the nuance, language, and context employees use when sharing their experiences. This provides a more reliable sentiment signal for exploring comments, identifying patterns, and understanding broader trends across employee feedback.

Multilingual support

By default, when the Perceptyx system receives a comment that is not in English, it translates the comment into English using Microsoft Azure AI Translator. That translated English comment is what gets fed into each of the comment analytics models described above.

There are some situations where the translation step falls short. Sometimes a survey goes out in English but a respondent answers entirely in another language, or weaves phrases from another language into their response. As a backup for those cases, our sentiment analysis, intents detection, and standard theme detection models are trained on what are called multilingual embeddings. That means they can accept text in another language directly, with no English translation needed, and still return sentiment, intent, and theme classifications at nearly the same level of accuracy.

Motivation and industry comparisons

Our multi-pronged approach gives you a full set of complementary methods rather than one way in. It lines up with the trends and best practices of the employee survey analytics industry, and all three approaches match the expected norms of methods used across the space.

The industry has historically used custom methods for theme detection, usually shipping a base set of engagement themes plus a way for users to create custom ones. So the custom method is expected to be available. It is a norm, not a differentiator.

Where we differ on custom themes is that we have gone beyond employee engagement by building additional theme sets, described above, covering diversity, equity, and inclusion, patient safety, specific industries, remote work, and more. That gives you off-the-shelf sets covering a range of topic areas that is not matched in scope elsewhere. It also allows for maximum flexibility, because you can tailor the themes and keywords to your needs and update them quickly when a new issue comes up.

The drawbacks of the custom approach are worth restating plainly. First, because it relies on keyword or phrase matching, false positive matches can happen simply because a word is being used in a different context than the theme is looking for. Second, it is limited to the list of themes it has been given to search for, which means new issues can be missed.

That is why we introduced deep learning models into our product line. They increase the accuracy and flexibility of theme detection, and they open up two alternative, complementary ways to explore comments through intents detection and sentiment analysis.

Appendix 1. Default themes

Appendix 2. Intents set

• Angry and Unfair
• Needs and Concerns
• Should and Suggest
• Wants and Preferences
• Praise