This November, we updated the way Impact Analysis is calculated. You can learn more here.

When we receive a new comment in your Chattermill Project, our powerful AI automatically applies one or more themes to it in order to help you understand how your customers feel about your product. Impact analysis allows you to see which themes contribute positively to your NPS and which have a negative effect, making it easier to decide where your attention needs to be focused to improve customer experience.

The theoretical minimum that your NPS can be is -100, so the positive part of your NPS is the difference between your NPS and the minimum. Using the example in the graph above, you can think of it as if you have 151 NPS points.

The theoretical maximum that your NPS can be is +100, so the negative part of your NPS is the difference between your NPS and the maximum. Using the example in the graph above, you can think of it as if you have lost out on 49 NPS points.

The positive and negative parts of your NPS are then split before the impact is calculated, so that positive and negative contributions are assessed separately.

The contribution to Net Sentiment now reflects both positive and negative sentiment separately for each theme. Themes that appear in both positive and negative contexts are assessed independently for their impact on the Net Sentiment score.

If you sum up the absolute values of the NPS points you have gained and the ones you have lost out on, you should get the theoretical maximum amount of NPS points you can obtain, which is 200 since there are 200 points on the NPS scale.

Our analysis can tell you which themes were responsible for the gains and losses on this scale.

We calculate scores using all responses (with and without responses).

There is no hard minimum threshold, but we would advise having at least a total of 500 responses. Additionally, themes with less than ten responses are filtered out.

What method is used to calculate the impact analysis score?

Impact analysis now uses a Linear Approximation method, which compares the average NPS scores or Net Sentiment for comments where themes are present or absent, split by sentiment (positive/negative). This provides clearer insights into both positive and negative drivers.

Theme contribution scores can be greater than 100 or lower than -100 because of the way Impact Analysis is calculated. For example, when calculating the positive drivers of CX, we start at -100 (the largest negative number for NPS) to calculate a theme's impact on NPS and vice versa for negative drivers.

I am getting a 'Sample size exceeds maximum limit' error. What does this mean?

The maximum response limit for single Impact Analysis is 150,000 and 300,000 for multiple Impact Analysis. If you reach this limit, we advise adding filters to reduce sample size or contacting [email protected] for a manual export.

Can I add the Impact Analysis chart to dashboards the same way as reports?

Yes. You can add the chart to dashboards as you would with your reports. Once you have made your changes, add a title to the report and click 'Save report'. You will then find this report in the list of reports when adding to a dashboard.

Can I share the Impact Analysis chart with my colleagues or externally?

Yes. Once your chart has been saved as a report, you can share this the same way as your other reports. You will find the share icon in the top right of the app, and you can select your teammates to share with internally or use the link to share externally (those without a Chattermill account will then have read-only access).

I can select Impact Analysis by NPS or Net Sentiment. What is the difference between the two metrics?

The main difference between both metrics is that Net Sentiment reflects the positivity and negativity of comments without taking the score into account, whereas Impact Analysis by NPS also explains the score because we have both sentiment and score used within the calculation.