User Researcher
Diary Study for Understanding Adoption
Diary studies gather data over a period of time that might be seen in a single user session.
Context and Goals
Service Analytics App went GA (general access) but there was interest in getting more internal adoption so that our internal service teams could use the product. There was also interest in understanding the initial walls that people faced when first using the product. This meant we needed a longitudinal study to get product feedback over time.
Persona
The target persona was still the End User but more specifically we targeted tier 3 service agents and service managers directly managing agents. We were able to recruit four internal service team members to participant, which are designated here as Participants A, B, C, and D to protect their identity.
Method
Traditionally diary studies require participants to carry a diary/journal to write down their daily experiences and feedback (hence the name). This doesn’t always work as people may forget the diary or lose it. Instead I created a digital survey form they could fill out on their smartphones and created a daily calendar event to remind each person. The calendar event had a link to the survey form where they’d write about why they did or didn’t use the product, how it helped or didn’t helped them, and any other comments about their experiences.
The study lasted about 3 weeks and required me to increase the incentives each week in order to boost consistency and participation. Diary studies can be a risky method because participants often attrit. But when done right it can provide rich information on how a product fits in a person's daily life.
Insights
What happened is that only participants A and B actually used the product and consistently filled out the survey. Participant C filled out the survey daily but never used the product and Participant D dropped out after the first week.
After the entries were compiled, I synthesized the data and later held retro interviews with the participants. Participant A and B were the most consistent but weaned in their usage because the product required customization, which was hard to do since it was designed (at the time of GA) to be out-of-the-box. Participant C never used the product due to preferring Excel and other reporting tools due to their ease of use. And participant D dropped out because work was too busy and he never remembered to fill out the “diary” entries.
​
“I just had too much to do. I can get to the data I want quicker with what other tools and I didn't feel like learning something new.”
--Participant C, Service Manager
​
It was these themes of easy of use and learnability that led to my work on the Einstein Analytics Journey Map project.