This November, we updated the way NPS Impact Analysis is calculated to help you better understand the scores generated in the NPS Impact reports.
This update will affect all of your existing reports, with the Impact Analysis scores within each report likely to change.
We're moving from a Regression Analysis method to a new, Linear Approximation method. Here's an explanation of what this change involves.
Linear Approximation vs Regression Analysis
The previous Regression Analysis method used statistical models to determine the impact of each theme on your NPS or net sentiment, which helped to minimize the double counting of impact.
The new Linear Approximation method for impact analysis calculates how specific themes affect NPS by comparing the average NPS or net sentiment scores when themes are present versus absent.
Advantages of the new method
Why are we introducing the new method?
It’s easier to understand: For less data-savvy users, the simpler calculation is much easier to grasp and to communicate to other team members.
It’s more consistent: The new method ensures you see the same impact score for a theme, even when you’re comparing it to another period. This is unlike the old method, where scores could change due to the regression model recalculating impact when comparing different time periods.
It works on smaller datasets: Impact analysis now is available for users with datasets of any volume.
FAQs: Understanding the Impact of the New Linear Approximation Method
How will this change affect my existing reports?
The update will impact all your existing Impact Analysis reports. You may notice that the scores within each report have changed due to the new calculation method. This change is intended to provide more accurate and intuitive insights into how specific themes affect your NPS.
When will this change happen?
We rolled these changes out on the 21st of November 2024.
Why are the scores in my reports different now?
With the shift from Regression Analysis to the Linear Approximation method, the way we calculate the impact of themes has changed. The new method compares average scores when themes are present versus absent. This results in scores that may differ from those calculated using the previous method. The difference in results should not be significant and the generally observed change is impact scores per themes being smoothened out compared to the Regression Analysis method where it was more common to see outliers in reports.
Do I need to update or recreate my existing reports?
No action is required on your part. The changes will automatically reflect in your existing reports once the update is implemented. However, we recommend reviewing your reports to familiarise yourself with the new scores and insights provided by the updated methodology.
Can I still compare data from before and after the change?
While you can compare data across different time periods, please be aware that the calculation method has changed. This means that direct comparisons of Impact Analysis scores from reports generated before and after the update may not be consistent. We suggest focusing on trends and patterns rather than exact numerical comparisons.
Will I need to learn new concepts to understand the updated Impact Analysis?
The core concepts of Impact Analysis remain the same. We’ve designed the new method to be more intuitive and easier to interpret without requiring advanced statistical knowledge.
Are there any limitations with the new method?
While the Linear Approximation method offers improved interpretability and granularity, it may not account for complex interactions between themes as the Regression Analysis did. However it provides clearer insights into each theme’s overall effect on NPS.
Will this change affect other analyses or reports outside of Impact Analysis?
No, this update specifically affects the Impact Analysis calculations. Other analyses and reports will remain unchanged.
Who can I contact if I have questions or need assistance?
Our support team is here to help you with any questions or concerns. Please feel free to reach out to us at [email protected].