Overview
Note: Some deliverables and specific details are unavailable due to a NDA.
During my second contract with Meta, I led research to support the launch of end-to-end encryption on Instagram, addressing urgent user security needs and tight deadlines. Using qualitative and quantitative methods, I provided insights that guided leadership in crafting solutions to enhance user trust and experience.
Read "Expanding Testing for End-to-end Encryption on Messenger", to learn more about the challenges of bringing additional encryption to Messenger and Instagram DMs.
Business Need
- Engagement with the security update among monthly active users was below expectations.
- The security update changed a common account feature and decreased engagement with the feature would slow rollout and launch of the feature.
User Problem
- Users didn't fully understand how adopting the security feature would change the their experience on Instagram going forward.
- Users were quickly dismissing or tapping through the security feature onboarding flow update without reading what the security feature was for.
My Responsibilities
- I met with stakeholders to understand the business problem and research questions.
- I reviewed previous research to define user problems and crafted a project roadmap with feedback from stakeholders.
- I created research plans that included goals, anticipated impact, research questions, previous research, hypotheses, recruitment specifications, screeners, discussion guides, and project timelines.
- I gathered and synthesized data into into research reports including insights and user quotes that were shared with stakeholders.
- I influenced product decisions and impact business outcomes by sharing and advocating for user needs with stakeholders and including them throughout the entire research process.
Impact
- Increased completion rate for users opting into the security feature.
- Reduced negative impact to sentiment and engagement from changes to Instagram that would have been unexpected by users.
Project Details
Diving in Head First
- Upon starting this contract, my first project was to understand the user behaviors, experiences, and pain points of the feature onboarding flow over time. I chose a diary study to because we needed to better understand user's decision making in the context of using Instagram in a real-world setting, rather than a research environment.
- I managed a two-week long international diary study with over 40 participants to understand the longitudinal experience of a user faced with the decision to adopt the security update.
- Alongside the logged tracking metrics provided from the data team, it was clear that users were not proceeding through the onboarding flow at the expected rate. Due to a major infrastructure project, our team was granted a short period of time to develop a better solution.
The Four Month Sprint
- At this point in the project, my entire stakeholder team had changed. A new product designer, content designer, and product manager joined the weekly sync. I helped onboard these stakeholders to the project by creating literature reviews: synthesizing 23 existing research sources into clear user problems, hypotheses, and a strategic point-of-view about what additional research questions were unanswered.
- I worked with the design and product management teams to develop two new designs to test. The initial launch of the feature included two international markets and four user clusters. One of the clusters was a sensitive population. I designed and managed two usability tests with 48 participants total that included the markets and user clusters.
- In addition, I designed and executed an unmoderated concept test with 30 participants in UserZoom to evaluate which of three content variations for the onboarding flow would be most effective.
The Big Pivot
- This project had multiple technical, legal, and regulatory constraints. Testing the solution in research environments was not providing the in-context evidence design leadership needed to finalize decisions. They needed to know how users would feel if another common feature was impacted by the security feature. I designed and executed an on-platform intercept survey with over 500+ users to understand attitudes and sentiment in real-time.
- Until this point, the constraints above were pushing the solution to be an opt-in experience. My research was used in a leadership discussion to make a decision to pivot the experience to an opt-out model.
Results
- My research and synthesis of colleagues work contributed to persuading leadership to change the experience from opt-in to opt-out.
- My research was also used to justify decisions during a critical design review for the experience that would be built and tested in the following year.
Reflection
- During this project I learned how to better communicate with senior leadership and stakeholders by being more proactive, concise, and clear during meetings and when presenting deliverables.
- I was often intimidated by the seniority and expertise of the people I worked with which led to me hesitating to question the purpose of a research request or how exactly the results would be used. In the future, I will be proactive in ensuring that meetings with stakeholders result in more clearly defined business needs.