Embedded Files
Client: iMiracleProject (a charitable nonprofit that works toward providing Ukrainian people with medical, educational, and food aids since the Russo-Ukrainian War)
Duration: 4 months (May – August 2022)
My role: UX researcher
Used methods: Semi-structured interview, Usability testing, Literature review
Disclosure: This project was only made possible by collaborations among a cross-functional team. Yet, this article mainly documents the work on my end.
TL;DR: Skip to Outcome to glimpse my research toolbox and result.
TL;DR: Skip to Outcome to glimpse my research toolbox and result.
Project Overview
Project Overview
Goal
Goal
iMiracleProject is a new small-scale organization with limited resources. Their website was roughly put together and its trustworthiness -- one of the most important qualities of a charitable organization in order to attract donors -- had never been tested among users. Recently, iMiracleProject faced two main challenges: 1) few donations generated from the website and 2) increasingly stiff competitions due to a surge in the amount of Ukraine-focused nonprofits. Hence, iMiracleProject felt the urge to improve the trustworthiness of their website. Due to the time restriction, my team aimed to redesign the homepage instead of the whole website because a homepage is where users form their first impressions of a company.
Team
Team
Our team was mentored by Anna A. Prisacari, a User Experience Researcher working full-time at Amazon and part-time at iMiracleProject. The team consisted of 3 UX researchers including me, Hira Jamshed, Josh Slocum (worked at phases 1 & 2), and Ruoxi (Crystal) Qi (worked at phase 3); 4 UX designers including Shaivi Ganatra, Parth Parekh, Nazila Zarei (worked at phases 1 & 2), and Guru Prasad (worked at phase 3); and 1 UX writer Katy Stainton (worked at phase 3).
Our objective: to redesign a more trustworthy homepage.
Process
Process
This is a past-paced 4-month project. Here is a high-level overview of what our process looked like. The tasks led by me are bolded.
We followed an agile workflow to ensure transparency, collaboration and speed in work. We had weekly meetings amongst the team and with stakeholders to update progress, generate suggestions, and most importantly, make sure everyone was on the same page.
PHASE 1: Learn & Strategy
PHASE 1: Learn & Strategy
Stakeholder Interview
Stakeholder Interview
To kick-start the research and design process, we wanted to interview stakeholders in order to learn: 1) the scope and parameters of the project; 2) technical and resource limitations; and 3) how can the new design help users and support business objectives. We also wanted to earn stakeholders' trust and bring the team and stakeholders on the same page .
From the stakeholder interviews, we learned that the stakeholders expect the final design to build user trust in iMiracleProject's commitment to the cause and abilities in handling donations. The executive leadership said:
"There are so many factors contributing to whether or not people give money, so the primary goal is to evoke feelings of comfort and confidence within users to the point that they will consider or think about donating."
Preliminary Research – Literature review
Preliminary Research – Literature review
Understanding that the goal is to build trust, I reviewed and synthesized 20+ research papers on website trustworthiness. I assembled a list of key takeaways (see below), which were corroborated by the direct and indirect competitive analysis conducted by Josh.
Literature Review Takeaways
Literature Review Takeaways
Dimensions of website trustworthiness
Clarity: content is presented in a clear, concise, and readable manner.
Informativeness: content is of high quality and useful.
Likability: the website is exciting and enjoyable to look at.
Credibility: information is authentic.
Factors determining website trustworthiness
Visual design: Trustworthiness is rated particularly high when the visual design is assessed as aesthetic.
Content: As stressed in the previous section, good content is clear and informative. For nonprofits, the types of content that predict user trust include but are not limited to:
Explaining causes and impacts to demonstrate awareness of the supported issue
Explaining the organization’s history and background to ensure transparency
Highlighting awards and organizational accreditations to display expertise
Research strategizing & planning
Research strategizing & planning
Based on the preliminary findings, we divided into two sub-teams. I flew solo on the iMiracleProject-focused team, using user interviews to probe users' thoughts on iMiracleProject homepage's visual design and content, including things they liked, pain points, and new features they wanted to see. I also involved UX designer Nazila during interview script preparation and the actual interview process to understand what information is digestible, helpful and crucial for designers. Hira and Josh work on the Nonprofit-general team, using user interviews to investigating what content users expect to see on a nonprofit website homepage in general. We probed these two sets of research questions:
Research Questions for Phase 2
Research Questions for Phase 2
iMiracleProject-focused team:
Which visual components of the homepage look trustworthy versus suspicious?
What information provided on the homepage feels trustworthy versus suspicious?
How do these aspects encourage versus prevent users from considering donating?
Nonprofit-general team:
What are users’ expectations of content on a nonprofit website homepage in order to trust the organization?
What content do users need to see to make donation decisions?
PHASE 2: Discovery – Remote Semi-Structured Interviews
PHASE 2: Discovery – Remote Semi-Structured Interviews
Here, I focus on the research on my end. I recruited and personally interviewed 11 users, all of whom had previously donated to charitable nonprofits so that they were potential donors, but had never heard of iMiracleProject. During the 30-minute interview sessions, I asked users a series of questions probing what they thought of the visual and content components of the iMiracleProject homepage. In addition, users completed a desirability test during which they selected words from a list that best described how the page made them feel. You can access the interview script here.
This is the affinity diagram I created that gives an overview of my interview findings:
Meanwhile, Hira and Josh on the Nonprofit-general team interviewed 9 users, aiming to understand what kinds of content are expected to be on nonprofit websites and how the expected content can evoke feelings of confidence and trust among users and encourage them to donate. After Hira and Josh also finished analyzing their interviews, we synthesized two teams' findings into the following research insights:
Most important insights
Most important insights
Mission & Work: one of the biggest problem users had was that they could not tell iMiracleProject's exact cause and work and who iMiracleProject was helping. This is especially detrimental because all users pointed out an organization's mission was most important to help them judge whether the nonprofit’s cause matched their own values and whether they would like to help. On the right side is the original iMiracleProject homepage we tested. You can tell at the first glance why people had a hard time grasping key information -- the mission statement and work description were lengthy, content was scattered and unorganized, and no information stood out visually.
Financial Transparency: all users were expressed concerns that they did not see tangible proof -- statistics on past work, received donation amount, and results -- to trust the organization's competence in spending donations effectively, particularly due to a lack of.
Look & Feel: the majority of users thought the website looked unpolished, boring, and empty, which, as users pointed out, implied that iMiracleProject was inexperienced and did not have resources to refine the website, thereby impairing their confidence in the company.
Other insights
Content was too lengthy and redundant.
Photos looked unprofessional because of the low resolution and busy background.
Information about iMiracleProject board members did not stand out and was incomplete.
Typos and incorrect capitalizations
Inconsistent layout
Original iMiracleProject Homepage used for interviews
User Needs
Here are some examples of what experience a user, or more specifically a potential donor, expect to have:
PHASE 3: Prototype & Test – Usability Testing
PHASE 3: Prototype & Test – Usability Testing
To address the above research insights, the UX designer team (Shaivi, Parth, and Guru) went through a design sprint and created a high-fidelity prototype of the homepage.
Key features of the new design includes:
Improved layout and addition of colors and images to increase aesthetics
Split sections to make it easier to locate information
Addition of work description and statistics to increase content informativeness
Addition of president information, media presentations and videos to fulfill user needs
Remote Unmoderated Usability Testing
Remote Unmoderated Usability Testing
The UX research team (Hira, Crystal, and I) then evaluated this prototype -- especially its aesthetics, content clarity and informativeness, and credibility -- via usability testing. We specifically wanted to examine whether the prototype fixed the pain points and fulfilled the user needs discovered in Phase 2. Hence, we asked the following research questions:
Research Questions for Phase 3
Research Questions for Phase 3
Does the prototype clearly deliver key information about iMiracleProject, including its mission, work, and board members?
Does the prototype look aesthetic?
Does the prototype meet users' expectations of a typical nonprofit website homepage?
Does the prototype make users feel confident in iMiracleProject's ability in designating donation money?
Hira, Crystal, and I recruited 9 participants, all of whom had donated to nonprofits before, and 3 knew iMiracleProject and had seen the original homepage prior to usability testing. I personally conducted 4 usability tests. During the 30-minute test sessions, participants completed a series of tasks including summarizing iMiracleProject’s mission, work, and personnel using information on the homepage. In addition, participants completed the same desirability test as in Phase 2 to gauge their first impressions and feelings toward iMiracleProject based on the prototype. At the end, we also asked participants whether the prototype met their expectations of a typical nonprofit website homepage. You can access the usability testing script here.
This affinity diagram I created provides an overview of our findings:
Results & Insights: Users were able to complete the tasks and generally liked the prototype!
Results & Insights: Users were able to complete the tasks and generally liked the prototype!
What did users like?
What did users like?
When asked to locate and summarize information on iMiracleProject's mission, work, and board members, all users were able to complete these tasks with ease, meaning that the prototype had better content clarity and informativeness.
8 out of 9 users found the homepage either met (5) or exceeded (3) their expectations of a nonprofit website homepage.
All users explicitly pointed out that their confidence in iMiracleProject's commitment and competence were ensured by the statements, numbers, photos, and/or media presentations.
All users who had previously seen the original homepage commented that the prototype had improved. Some quotes include: "prettier and more exciting" and "easier to read and understand."
Users had more positive first impression of the prototype compared to the original homepage, as you can see from the graph below that plots how users' desirability test word choice differed between the original homepage evaluation and the prototype evaluation, meaning that the prototype was a huge improvement.
Gaps & Opportunities
Gaps & Opportunities
Of course, the design was not perfect. I synthesized the following insights to open room for improvement:
Lack of clarity regarding iMiracleProject's work. This is the biggest pain point discovered at Phase 3. No one grasped the key feature of iMiracleProject that it is a project-to-project organization, meaning that iMiracleProject used donations to fund individual projects that ran by different people/groups.
Lack of background information and updates on the Russo-Ukrainian War, which could help users emotionally connect with the cause and feel more urgency in helping iMiracleProject.
Lengthy text was a pain point already discovered at Phase 2, but we had not addressed this issue because the UX writer joined relatively late, so the team did not have time to streamline the content before usability testing.
Finally, there were some visual design insights regarding section background color, font, and layout.
Hira, Crystal, and I presented these gaps and opportunities to the writer and designers. Using the insights discovered in Phase 3, the designers went through another design sprint to create the final design. Katy, our UX writer, rewrote the content.
The team together created a presentation and a written report to deliver digestible insights, actionable recommendations, and the final products (design and content) to stakeholders. I personally was in charge of writing the executive summary and research insights, and presenting .
Outcome
Outcome
As a cross-functional team, we went through a fast-paced cycle of user-centered design process (planning ➞ exploration ➞ concept generation ➞ evaluation) toward the primary objective set by the stakeholders: redesign the homepage to improve user trust.
We started with a homepage that users remarked as "unpolished", "boring," "impersonal," and "can't understand the organization's mission and work." With literature review and interviews (generative research) as well as usability testing (evaluative research), I provided designers and content writer with user needs and pain points, which informed the final design that users commented as "trustworthy," "exciting," "touching," and "easy to find important information," fulfilling the goals and expectations set by the stakeholders.
Below, you can see how the design evolved from the original homepage to the prototype and to the redesigned final homepage based on. Note that the revised content was not included in the final design, thus the unchanged mission statement and question marks in the final design.
Next Step: We are currently still in the process of coordinating with iMiracleProject to bring the design to life. Once the design is built, we plan to keep an eye on the ultimate goals of building trust and boosting donations.
Reflections
Reflections
Obstacles & Solutions
Obstacles & Solutions
Here, I record some obstacles we encountered during the research process and the solutions we came up with.
Limited resources: iMiracleProject could not afford to pay users to stimulate participation. Meanwhile, I had a tight schedule, which required me to finish recruiting and scheduling participants within one and a half weeks.
Solution: For both the interview and usability testing, I relied on my teammates' personal connections to find people who were willing to participate without monetary rewards. Then, I used snowballing sampling technique to ask those people to reach out to their social circles. To avoid potential bias, I did not recruit people I knew. To make sure that even with a convenience sample, I'd still be able to distill applicable insights, I used screening surveys to exclude people who were not targeted users (e.g., had never donated to any nonprofit).
Difficulties in putting statistics on the homepage: A majority of interviewed participants said that seeing the organization's impacts in numbers was a key factor determining whether they would donate. However, giving statistics on past work turned out to be difficult and tricky for iMiracleProject. iMiracleProject is a project-to-project-based organization, meaning that it collects donations and use them to fund individual projects run by people outside of iMiracleProject. As a results, iMiracleProject usually is not the sole funder of a project, and board members normally do not know what percentage of a project's achievements can be attributed to iMiracleProject. Hence, the conventional ways of reporting past work (such as how many people were helped, how much food was delivered) were unfeasible.
Solution: While giving statistics was hard, I recognized that it was necessary based on the interview findings. Thus, I worked brainstormed with iMiracleProject's accountant about how iMiracleProject could get creative yet still transparent with reporting numbers. In the end, we came up with this format for reporting: "iMiracleProject has provided $20,000 to Project X, making up for 40% of the total funding. Project X has so far accomplished goals A, B, and is working toward goal C." I provided this as an actionable recommendation to stakeholders, and stakeholders agreed that this was a doable solution.
Constant changes in team composition: While we had some team members stayed for the whole project (Hira, Shaivi, and I), the rest of the team was constantly changing. Yet, because we had a tight schedule, new members did not have a lot of time to ease into the job, but had to instantly pick up where previous team members left off.
Solution: After experiencing some hassles during earlier personnel changes, the research team and the design team started keeping a detailed log of our daily tasks (including what was initiated, in progress, or completed, and what went smoothly vs. wrong that day) and plan for the next day (including what to start, continue, fix, or wrap up). By doing so, whenever we had a new member join the team, the member just needed to read through both teams' logs to understand what their immediate tasks and goals were and where to find all the necessary materials for their tasks.
What Would I Do Differently?
What Would I Do Differently?
If time and resources allowed, I would take a more mixed-methods approach by adds some quantitative components. More specifically, after the interviews at the discovery phase, I would bring insights uncovered by the interviews to administer a survey to a larger sample asking 1) their agreements with the pain points and 2) the extent to which each pain point would potentially impact their donation decisions. This way, the deliverables to designers and stakeholders would be more informative and validated. Also, I would administer a survey just asking respondents' agreements with the pain points during the prototype evaluation so that I could compare the scores for the original homepage and those for the prototype to further inform whether or not the prototype addressed the pain points.
We had a UX writer later in the project. While she was able to provide revised content based on research insights for the final deliverables, the content did not make it onto the prototype evaluated by the usability testing. In the future, I would like to have a writer earlier in the design process so that we could validate the revised content via evaluative research such as usability testing or A/B testing, which could give stakeholders more confidence in our final deliverables.
I noticed later on that iMiracleProject had some merchandises such as T-Shirts, mugs, and stickers. If I noticed this earlier, I would have requested merchandise as participant payments, which may expedite the recruitment process and/or help me get a more diverse sample. In the future, if I lack monetary resource for participant recruitment, I will make sure to check whether there are any alternative types of payment.
Page updated
Google Sites
Report abuse