Daylio and Moodpath App Study: Impact of Paywalls & In-App Marketing on Brand Perception and User Engagement

Looking into how in-app marketing and content paywall strategies affect freemium user engagement with the mental health apps Daylio and Moodpath.

Project Details

Background

• The rise in the popularity of mental health apps highlights the potential of these apps to become more accessible and effective therapeutic tools for users in addressing common mental health issues, providing a wide range of services such as information, therapy-based exercises, meditation, symptom tracking, and professional resources. 

• However, the biggest complaints and negative reviews surrounding popular apps are the content paywalls and advertisement strategies employed by apps – such as through locked subscription-only content, in-app promotions (e.g. marketing for app memberships) or external advertisements (e.g. banner ads for external companies) – which may be intrusive and interfere with the app’s usability and consequently impact user trust and engagement with the app.

• This presents an opportunity to:

  • Identify what key obstacles are present in maintaining user engagement with freemium mental health apps that utilize content paywalls, subscription-only features, and/or other in-app marketing

  • Provide suggestions as to how apps might more effectively incorporate or improve on these monetization features while also maintaining and boosting user engagement

• This study was conducted as a personal project.

Research Methods

• Surveys

• Focus Group (remote)

• Thematic Analysis

Collaborators

• N/A

Time Frame

• March 2020 – April 2020

Tools

• Google Forms

• Zoom Video Call

Objective

• Identify key obstacles impacting user engagement with freemium mental health apps presented by in-app marketing and content paywall strategies

• Provide recommendations on how apps may more effectively frame or modify these monetization features to maintain and increase user engagement

Research Questions

• Business-oriented research question: How can we balance active freemium interest with app monetization strategies?

• General research question: How do content paywalls and in-app marketing affect how users perceive and engage with the app?

• Overarching research question: How can we develop monetization strategies to support user trust and engagement while mitigating its negative impact on user experience?

Process

Recruitment

• I contacted local community and social media groups in mental health and wellness and asked them to share a description of the study, along with a link to a screening survey that I created, within their social circles.

• The screening survey recorded past use of mental health phone apps, familiarity with advertisements and marketing tools within the apps, and whether the user had previously purchased a mental health app subscription or membership before. For the objectives and scope of this study, no identifying or demographic information aside from a name and contact method were requested.

• Users who had purchased subscriptions or memberships before were screened out to limit the potential bias from users less predisposed towards utilizing the freemium version of an app. 

• Out of the 18 respondents to the screening survey, 14 users were eligible to participate in the primary survey.

Survey

• The survey consisted of two sections. The first section asked for general information such as frequency of mental health app use, what forms of marketing tools users may have encountered before, and open questions gauging attitudes towards the impact of advertisements and content paywalls on engagement with the app.

• The second section presented a brief series of screenshots and scenarios from mental health apps on advertisement placements and content paywall transitions. Users were asked questions on describing their impressions and expectations of particular app features and advertisement strategies based on those screenshots.

Sample screenshots of the Daylio app and Moodpath (now MindDoc) app included on the survey.

Survey Insights

• The survey focused on two types of app advertisement/monetization approaches:

  • In-line advertisements such as banner ads, including both external and self-promotional ads (e.g. promoting subscriptions, premium content to freemium users)

  • Content paywalls, e.g. blocked access from subscription-only content within the app

• Thematic analysis on the respondents’ answers to the survey led to identifying two emerging themes: the negative brand perception and user trust/app credibility due to these monetization strategies:

• Negative brand perception: Users were presented with screenshots of the mental health journaling app Daylio with in-line ads present on the screen, and asked a series of questions.

  • 9 out of the 14 respondents in the survey indicated that the in-line ad placements “impacted” or “greatly impacted” their experience and choice to continue using the app

  • Comments included that the ads were disruptive, flashy, or irrelevant, and distracting from the app’s content. Users wrote that “[the ads] distracts from whatever you’re trying to focus on” and that the ads “feel intrusive and take up too much space”

    • One user commented that they “understand [apps] need to profit too but a bunch of flashy ads on the diary page makes it seem unfriendly and unenjoyable”

  • Insight: The use of external or self-promotional ads (in the form of in-line or banner ads most commonly) is largely understood as necessary for many free apps by users.

    • Comments from respondents in this survey suggest that the repetitive or distracting placement of these ads in contrast with the app’s focus on mental health and as a helpful resource for users contributes to a negative perception of the app’s motivations, and by extension, its brand.

    • Users of mental health and wellness apps in particular may be more sensitive and aware of distractions and immersion-breaking features such as advertisements on the same page of the activity they are completing.

  • Recommendation: Some suggestions to mitigating negative brand perception due to use of in-line ads may include: minimizing repetitiveness (number of times ads are encountered while scrolling down the same page), distraction/flashiness (e.g. increasing relevance of ads, decreasing size of ads on page relative to rest of content, not placing ads in the middle of a feature, static over dynamic ad content), and increasing ad relevance.

  • Taking into account the perception of the app by these users in the design and inclusion of in-line ads may improve overall app experience, brand impression, and effectively increase user motivation and engagement with the app, mitigating negative connotations of the app’s utility and preserving engagement while still achieving business goals mindfully.
    

• User trust/app credibility: The survey presented screenshots portraying different instances of encountering content paywalls on the free version of an app.

  • These content paywall transitions were similarly commented on as distracting and irritating for users, and contributed to a break in immersion from the app’s content - particularly if the paywall transition was placed in the middle of completing an activity or feature (i.e. blocking users from accessing results w/o subscription).

    •  A respondent wrote, “Saying the app is free then making you pay to be able to use it is misleading and deceitful to people who could benefit from them.”

  • Insight: The use of content paywalls may be an effective strategy for promoting both freemium and subscription features to the user. However, comments from survey respondents suggest that when subscription-only features are unclearly marked and paywalls are encountered by freemium users, or paywalls are placed in the middle or end of a feature (e.g. blocking the user from accessing results or completing the feature w/o subscription), user irritation and dissatisfaction with the app is significantly increased.

    • Users of mental health apps in particular may be more sensitive and aware of monetization strategies, which may come off as disingenuous and misleading to their intentions of providing mental health– and wellness-focused content. 

  • Recommendation: Taking these points into account, suggestions for increasing app credibility and user trust may include ensuring being upfront with subscription-locked content, ensuring that it is clearly marked or made obvious to users on the freemium version before being clicked on, and is not placed in the middle or at the end of an activity, so as to not confuse or be misleading. 

  • Emphasizing clarity and transparency in the app’s freemium content and paywalled features is an important factor in increasing user trust, and increasing satisfaction with the overall app experience.

• Summary: These two general themes of users’ attitudes and impressions towards in-line ads and content paywalls suggest where major pain points for users may arise when using freemium mental health apps. Taking these general themes and resulting suggestions may improve overall app experience and effectively increase user motivation and engagement with the app while continuing to use these monetization strategies.

Focus Group Discussion

• Following qualitative analysis done on the surveys, the two themes identified — negative brand perception and user trust/app credibility due to in-app marketing strategies — generated further questions towards understanding user expectations and motivations surrounding using freemium mental health apps. 

• With the goal of clarifying and expanding on these emerging themes and more fully fleshing out the particular influences and motivating factors involved, and how those might shape particular pain points, I decided to carry out a focus group discussion in this study. 

• Three respondents whose survey answers suggested possibility for deeper insight were invited to a remote focus group discussion over video-call.

• The focus group was recorded with each participant's permission, and lasted 45 minutes. I prepared a list of semi-guided questions, which included probing their previous app experiences, thoughts on mental health apps and app monetization strategies, and some of their recorded survey answers. 

• Findings from the focus group discussion were then coded and qualitatively analyzed, and the results were combined with the findings from the survey. 

Focus Group Insights

• Thematic analysis of the focus group discussion led to a deeper understanding of how these two monetization strategies contributed to negative brand perception, expanding the concept into two subgroups, passive (in regards to in-line advertisements) and active (content paywalls) negative brand perception. The theme of financial accessibility was also identified.

• Negative brand perception (passive): Some users noted that a significant factor contributing to their impression and decision on continuing to use freemium apps or uninstalling it was how “intrusive and aggressive” or “overbearing, pushy” self-promotional ads advertising their subscription or premium version on freemium users in the app were.

  • In-line ads (whether external or self-promotional ads) that are flashy, distracting, and/or irrelevant may contribute to a slow accumulation of negative brand perception, decreasing user motivation to engage with the freemium app over time, and even disincentivize users further from considering the subscription or premium version.

• Negative brand perception (active): Freemium users repetitively encountering blocked access/content paywalls to features find these paywalls to be particularly irritating and frustrating when they affect an activity flow, or impede in the middle or near the end of an activity, or lock a feature that was misleadingly marketed or unclearly stated as paid content.

  • In contrast with the in-line ads, these paywalls are a hard block, actively disengaging the user from the app, and affects their perception of the app in a more involved and significant way, regardless of the quality of the free and subscription-locked content as a whole.

  • Regarding content paywalls, one user said, “Apps that say they’re ‘free’ but then lock a bunch of their stuff behind paywalls when you actually download and try to use the app, it just comes off as disingenuous, no matter how good the app is.”

  • “I get [apps] needing to maintain themselves and the free versions won’t include as much, but when [the freemium version] is the bare minimum and not useful at all, it just makes encountering paywalls very frustrating.”

• Recommendation: For freemium app users, encountering repetitive inaccessible key content has a significant impact on the perception of the app’s functionality and usefulness, regardless of the quality of the app content as a whole.

  To help mitigate this negative perception while employing content paywalls, apps might:

  • 1) ensure the free or subscription-only content being marketed is accurate in their promotional materials and differences are clearly stated or marked to the user

  • 2) conduct a competitive analysis with other similar apps with free– and subscription-tier versions to understand what the “bare minimum” expected by free-tier users might be

  • Reframing how the free and subscription versions are presented to the user, or making available additional features to address the above frustrations to better complement and incentivize freemium-only users to maintain and increase engagement.

• Financial accessibility: For users who cannot purchase full-app subscriptions or access to paid content, and/or cannot afford professional or external resources, repetitive encounters with paywalls drew frustration. 

• Regarding being unable to afford mental health resources, one user said: “Of course it’s irritating when you see a new promising app come out with all these features and capabilities and it’s being marketed as ‘free’ and ‘accessible’ to everyone but once you install…you find that there’s maybe three activities that are free and aren’t useful to your needs at all. And every other app has it. And you have to subscribe to access anything else.”

• “There’s so many [mental health] apps out now but so few come off as genuinely wanting to help you and not just capitalize on you as a consumer or potential subscription…comes off especially misleading and kind of exploitative if their whole mission is to be all focused on your wellbeing and what’s good for you but don’t try to help you access it.”

• Recommendation: Consideration of financial barriers and accessibility, particularly for a mental health-focused audience, can be important actors contributing to an app’s credibility.

  To help increase app accessibility to free-tier users, apps might:

  • 1) have an option or button to view a list of external free resources or supplementary information the user could use in addition to the app or as a guide for more specific help than the app can provide for. This may help to emphasize consideration of financial inaccessibility to use the full app, and provide other avenues for help. 

  • 2) conduct a competitive analysis with other similar apps with free– and subscription-tier versions to understand what features might be expected by free-tier users, and adding or improving certain features to provide better services, complement and incentivize users on the free-tier version of the app and increase engagement. 

• Further insight was generated on user trust and app credibility:

  • One user said, “It’s frustrating trying to get help and only realizing after downloading and giving over my information that to use it I have to pay up…and they keep pushing ads on you”

  • “[Apps] should be clear and upfront about how much they actually cost especially if it turns out that more than half their features are paid…”

• Recommendation: To increase user trust and app credibility, apps might:

  • 1) be upfront with the user about what content is included in the free vs. subscription versions of the app, and ensure that the differences between both are clearly stated or marked to the user, to avoid frustration or misleading expectations.

  • 2) limit overly repetitive and excessive self-promotional ads (e.g. pop-ups or banner ads), which may come off as intrusive and especially distracting for those who consistently use the app. 

Thematic Analysis

• After the surveys and focus group were completed, qualitative data was coded and thematically analyzed for users’ impressions, motivations, and frustrations of the two types of in-app advertisement strategies. These were then further grouped with relevance to answering the research questions and providing actionable insight in mind. 

Key Questions - High-Level Takeaways

Aside from the specific design recommendations and insights above, here are the main takeaways from the research answering the original key questions.

Business-oriented research question:
How can we balance active freemium user interest with app monetization strategies?

Takeaway: For attending to users in a mental health-focused space, monetization strategies must be developed with care to understand where it is appropriate to implement.

General research question:
How do content paywalls and in-app marketing affect how users engage with the app?

Takeaway: Content paywalls and in-app marketing may have unintended brand implications that discourage users from investing time and/or money into the app.

Overarching research question:
How can we develop monetization strategies that support user trust and engagement while mitigating its negative impact on user experience?

Takeaway: Monetization strategies’ impact on brand perception must be accounted for during development and are not an afterthought.

Challenges

• The study budget and logistic constraints limiting the number of participants in the study introduced potential sampling bias in representing the range of users who utilize these mental health apps. Ideally, with a larger research team and study budget, more participants can be recruited into the study.

• Only participants who had not previously purchased a mental health app subscription or membership before were included in this study. As this study was to focus on the impact of in-app marketing and content paywall on users within the free version apps, this screening parameter was included to limit the potential bias from users who had experience with previously purchasing full access to any app (eliminating in-app advertisements and paywalls) —thus being less predisposed towards utilizing the freemium version of an app or having a biased view against the experience of it.

  • This parameter, while limiting potential bias, also further reduces the sample size representing the range of users who utilize these mental health apps. Though this study is focused on a specific sub-population of app users, there ideally would be a much larger sample size of participants, particularly for the survey.

• Due to participants being recruited from social circles focused on mental health and well-being, there is potential for selection bias as those who volunteered for this study are not representative of all users who utilize mental health apps. However, for the purposes and scope of this study, following the participants who due to this background may be generally more inclined to invest in sustained app use, might be more helpful in garnering insight into the motivations and attitudes involved in how users engage with in-app marketing and paywalls.

Other Considerations

• As this was a solo research project, had there been stakeholders or other collaborators involved, communicating feedback at various points during the process – for example engaging stakeholders at various steps of the study process or involving team members with carrying out the focus group and qualitative analysis – would be important to ensure everyone is informed and on-board with the direction and results of the study. 

• Future directions

  • Run a workshop to gain deeper insight into the effects of the pricing models and paywalls on brand perception

  • Expanding research beyond existing users; exploring markets whose sentiments are not captured by the scope of this research / product  

    • What untapped markets exist that may have been excluded by existing strategies?

• With a larger study budget and/or research team, the research strategy might be modified to include a larger number of participants, use of further methods such as participant interviews or more focus groups, incorporating participant techniques (e.g. card sorting, desirability study) or including additional analysis methods such as a competitive analysis.