Overview
Note: Some deliverables and specific details are unavailable due to a NDA.
See more detail about the product here: Introducing Microsoft 365 Copilot – your copilot for work
Business Need
- Log data report suggested that “Translate” requests are occurring in Word Copilot at above average rates, with lower than average “Keep Rates”.
- No overall strategy for Copilot language translation across product teams.
- Little evidence for stakeholders to invest in Copilot language translation.
Research Questions
- How are Word users currently translating content for work?
- What are the benefits and challenges of using Word Copilot Canvas and Chat to do language translation?
- What would be the preferred language translation experience for Word users?
My Responsibilities
- I met with stakeholders to define the problem and align on research questions.
- I ran an exploratory survey with ~30 participants to better understand the general use cases and problems people have with Word Copilot for language translation.
- I wrote the research plan including screener, discussion guide, and synthesis of relevant previous research - including survey data.
- I moderated 8 in-depth interviews, inviting stakeholders to participate.
Impact
- Actionable bug fixes and errors were provided to enable quick improvements to the experience.
- This research highlighted a clear need for two different product teams to work together, improve the design, and build technical consistency.
- The stakeholders were given clear evidence to invest in improving language translation in Word Copilot Canvas and Chat.
- Insights and recommendations from this report were included in efforts to create an overall strategy for Copilot Language Translation.
Project Details
Users
- Word daily active users.
- Needed to do language translation for work.
- Were at least somewhat familiar with AI tools.
Methods
- I ran distributed exploratory survey in UserTesting to ~30 participants to better understand the general use cases and problems people have with Word Copilot for language translation.
- I recruited for an moderated in-depth interviews with 8 participants who were asked to demonstrate how they would use Word to translate content. They were prompted to use Copilot if not found organically.
Challenges
- Deciding how the scope the project for maximum impact.
- Two siloed teams working on Word Copilot Canvas and Copilot Chat.
- It was difficult to find participants who used Word Copilot Canvas and Chat.
Findings
- Learned that participants navigated first to Copilot Chat, however, it had severe usability issues and could not provide an accurate translation.
- Learned that the Copilot Canvas output most closely represented participants’ ideal language translation experience in Word, however, it had severe discoverability issues.
Impact
- Actionable bug fixes and errors were provided to enable quick improvements to the experience.
- This research highlighted a clear need for two different product teams to work together, improve the design, and build technical consistency.
- The stakeholders were given clear evidence to invest in improving language translation in Word Copilot Canvas and Chat.
- Insights and recommendations from this report were included in efforts to create an overall strategy for Copilot Language Translation.
Reflection
While scoping this project, we decided to take a crawl, walk, then run approach - keeping the score very narrow to address Word Copilot. While I was unsure about this at the beginning, it turned out to be the right decision for helping to identify specific areas of improvement and gaining momentum.
Photo credit:
https://www.computerworld.com/article/1612034/microsoft-copilot-pro-review-office-joins-the-genai-revolution.html
https://blogs.microsoft.com/blog/2023/03/16/introducing-microsoft-365-copilot-your-copilot-for-work/
https://support.microsoft.com/en-us/topic/copilot-tutorial-chat-about-a-word-file-d168b785-4b08-4311-b368-2daa15eb6283