When we receive a new comment in your Chattermill Project, our powerful AI automatically applies one or more themes to it in order to help you understand how your customers feel about your product. Impact analysis allows you to see which themes are contributing positively to your NPS and which ones are having a negative effect, making it easier to decide where your attention needs to be focused to improve customer experience.
Reinterpreting your NPS
The theoretical minimum that your NPS can be is -100, so the positive part of your NPS is the difference between your NPS and the minimum. Using the example in the graph above you can think of it as if you have 151 NPS points.
The theoretical maximum that your NPS can be is +100, so the negative part of your NPS is the difference between your NPS and the maximum. Using the example in the graph above you can think of it as if you have lost out on 49 NPS points.
The positive and negative parts of your NPS are then split among the positive coefficient themes proportional to their individual total effects.
Interpreting the sums
The "contribution to NPS" for the positive coefficient themes should sum up to the positive part of the NPS and the same for the negative coefficient themes for the negative part of the NPS.
If you sum up the absolute values of the NPS points you have gained and the ones you have lost out on, you should get the theoretical maximum amount of NPS points you can obtain which is 200 since there are 200 points on the NPS scale.
Our analysis can tell you which themes were responsible for the gains and losses on this scale.
Using contribution in comparisons
You can use Impact Analysis to compare two periods of time and the score contributions that different themes had to your NPS over time, as shown in the following model.
Can I add this to a chart or dashboard? What about exporting my data into a CSV?
During Open Beta the Impact Analysis charts cannot be added to dashboards or generate CSV exports; stay tuned for future updates!
Do you take all responses (inc without a comment) into account when calculating Impact Analysis?
We calculate scores using all responses (with and without responses).
What is the minimum number of responses I need to run Impact Analysis?
There is no hard minimum threshold, but we would advise having at least a total of 500 responses. Additionally, themes with less than 10 responses are filtered out.
Some themes have scores greater than 100, isn't that higher than the theoretical maximum NPS score?
Theme contribution scores can be greater than 100 or lower than -100 because of the way Impact Analysis is calculated. For example, when calculating the positive drivers of CX, we start at the -100 (the largest negative number for NPS) to calculate a themes impact on NPS and vice versa for negative drivers.
I am getting a 'Sample size exceeds maximum limit' error, what does this mean?
The maximum response limit for single Impact Analysis is 150,000 and 300,000 for multiple Impact Analysis. If you reach this limit we advice that you either add filters to reduce sample size or contact [email protected] for a manual export.