User Engagement Analysis of Microsoft Viva Engage
​Key Learnings:​
1. SQL query writing for Data Analysis
2. Hypothesis formulation and testing for Business Analysis
Project Overview:
In this project, we aim to investigate and address a noticeable decline in user engagement within Microsoft Viva Engage, a social network designed for effective communication among coworkers. The primary goal is to analyse the causes behind this decline and propose solutions to revitalize user engagement, ultimately driving better product and business decisions using data-driven insights.
Dataset and Tools
A SQL dataset similar to the structure of actual Microsoft Viva Engage data is used for this case study. PostgreSQL is utilized for carrying out the required data analysis.​
Observation
User engagement is tracked using the metric of weekly active users, defined as the number of unique users logging into the application each week. The chart below illustrates the trend of weekly active users. Notably, there is a noticeable decline in user engagement starting from the week of July 27th.
Implementation Plan
-
Hypothesize the Root Causes: Identify potential factors contributing to the dip in user engagement.
-
Test Hypotheses: Systematically validate or refute potential causes to gain insights.
-
Recommend Solutions: Propose actionable solutions to address the identified issues and improve user engagement.
Hypothesis Formulation
-
Growth Factor: Fluctuations in user engagement may be linked to changes in user base or behavior.
-
Vacation Impact: A significant portion of users might be on vacation during the observed dip in engagement.
-
Feature Breakage: Technical issues may hinder users' access to the platform, especially on mobile devices or laptops.
-
Client Impact: Loss of major clients could result in decreased overall user engagement.
-
Engagement Metrics Analysis: The decline in engagement may be attributable to specific factors related to user engagement patterns or content relevance.
Hypothesis Testing and Investigation
Growth Factor:
Understanding the impact of user growth on engagement can provide valuable insights into long-term trends. It is easy to measure and straightforward to check. A SQL query is written to examine the number of new users created and activated each day. From the graph below, we can infer that there has been no significant change in the growth rate; it remains high on weekdays and low on weekends.
Since the growth appears normal, the dip in engagement may be stemming from existing users rather than new ones. One effective approach is to cohort users based on their sign-up dates. Another SQL query is executed to plot the engagement of users in a week, grouped by user age (1 week to 10+ weeks). The below graph indicates a decrease in engagement among users who signed up more than 10 weeks ago. Recognizing that the issue is localized to older users suggests that it may not be related to a one-time spike from marketing traffic or new traffic impediments like being blocked or changing ranks on search engines.
Vacation Impact:
While vacation-related dips may be less likely, it's essential to rule out external factors that could affect user engagement. Analysis of the user engagement chart reveals a steep decline in engagement numbers from the week of July 27th to the week of August 3rd. A SQL query is executed to examine the change in user engagement (both absolute and percentage) by location between the two weeks. The plot indicates no discernible pattern, suggesting that the drop in engagement is not due to a public holiday or vacation in a specific location.
Feature Breakage:
This hypothesis addresses potential technical issues affecting all users and can be tested relatively quickly. A SQL query is executed to calculate the engagement rate by device type (mobile, tablet, and laptop). The plot reveals a significant drop in phone engagement rates, indicating a likely problem with the mobile app that affects long-term user retention.
Client Impact:
Loss of major clients could have significant repercussions on user engagement and warrants prompt investigation. Analysis of the user engagement chart shows a steep decline in engagement numbers from the week of July 27th to the week of August 3rd. A SQL query is executed to examine the change in user engagement (both absolute and percentage) by clients between the two weeks. The data suggests that the loss of major clients is not the reason for the decline (Top 15 clients based on number of users is listed below).
Engagement Metrics Analysis:
Understanding changes in user behavior is crucial for identifying underlying issues and implementing targeted solutions. The purpose of the digest email is to bring users back into the product. Given that the problem seems related to the retention of long-time users, it's worth investigating whether the email is a contributing factor. A SQL query is executed to plot the weekly digest emails sent, email opens, email click-throughs, and re-engagement emails sent. The analysis indicates a drop in email click-through numbers.
A more detailed SQL query is executed, revealing a steep drop in the weekly click-through rate.
​
Recommended Action/Result:
Through systematic hypothesis testing, it is revealed that the decline in user engagement may be attributed to issues related to mobile usage and digest emails.
​
-
Mobile Application Investigation: Initiate an investigation into potential technical issues with the mobile application that may be impeding user access and engagement. Collaborate closely with the mobile development team to identify and rectify any issues affecting mobile usage promptly.
-
Email Campaign Optimization: Notify the relevant team responsible for email campaigns to conduct a detailed analysis of engagement email performance, focusing on factors influencing open rates. Implement strategies to improve email content, timing, and delivery to boost user engagement.
Conclusion:
In conclusion, the investigation into the decline in user engagement within Microsoft Viva Engage has yielded valuable insights and actionable recommendations. By systematically testing hypotheses and analyzing data, it was determined that issues with the mobile application and email campaigns are significant contributors to the decline. Initiating investigations into mobile app functionality and optimizing email content and delivery strategies are essential steps toward revitalizing user engagement. This project underscores the importance of data-driven analysis in identifying challenges and driving informed decision-making for improving digital platforms.