Author: whatsgoinhuang

Background: At Philanthropy University, our mission is to increase the capacity of grassroots CSOs to progress the 2030 UN Sustainable Development Goals (SDGs). We approach our mission with 3 products: online courses, community, and capital investments. The majority of our social entrepreneurs work in regions of the world where internet access is expensive and sparse, so designing for accessibility in their environments is crucial.

Financial independence and revenue generation became a priority towards the middle of 2019. Generating at least $50m in ARR by 2025 became our north star. But before we could start design and development processes to build a revenue generating business line, we had to audit our finances.

Until then, only our Director of Marketing and CEO knew that we were spending more than $70,000/mo to acquire free users. When I joined at the end Q3 2019, a few of of my first tasks were to severely decrease the cost of acquisition(CAC), define our new target market user (TMU), and triple acquisition of our TMU.

_{For the sake of focus, I will not go into how our TMU was determined. We defined our TMU as social impact organizations with between 10-100 employees who work in the global south and are high potential candidates for funding from major US-based foundations. This TMU has since changed now that we’re more aggressively pursuing revenue and building a spinout BCorp.}

Our major question: How might we halve CAC, triple TMU acquisition and maintain rigorous data collection?

Our approach: Luckily, I’d just worked on lowering CAC at Mystery Science and learned from their outstanding growth and data team, so I had a head start. This was our approach:

1. Marketing spend analysis: How much are we spending? On what platforms? How are the campaigns performing? What have we tried, not tried?
2. User behavior data analysis: Where to leads go when they’ve clicked through an ad? Where do they get stuck? Where and when do they bounce? What do we do (if anything) to get them back? Do we encourage existing users to refer? How?
3. Case studies and conversations with mentors: Lots of secondary research to understand marketing and registration flow best practices, interesting approaches from mentors who’ve been optimizing acquisition for decades.
4. Primary research: Interviews with existing and prospective users about registration and the perceived value of our product offering, as presented in our ads.
5. Screen recordings: Watching users interact with our registration flow to understand bottlenecks, limitations, and frustrating experiences
6. Experiments: After reviewing our data and research, we prioritized hypotheses to test via study groups or on-platform experiments.

Synthesized research here from Q4 2019

• 

• 

• 

• 

• 

• 

Research highlights:

1. Keeping ad spend the same, we had about 100,000 unique visitors weekly. Of those, around 3000 visitors completed registration. Of those, ~100 were our TMU. Overall CAC for our TMU was over $400.
2. ~63% of leads never made to registration, but those who did were often stuck on a few of the 40+ questions, many of which we didn’t use in our analysis or reporting to funders.
3. ~75% of leads that completed registration did not activate their account, which is required before they can take any actions on the site.
4. English is a second (or third, fourth) language for many leads, so particular questions did not translate well to their native language or to the political or legal structures in their countries (eg. business type: social enterprise, nonprofit, NGO, etc. have various meaning in different countries)
5. Major opportunities included:
   1. Decreasing the complexity and effort to complete registration
   2. Giving users an opportunity to explore the platform before creating the friction of registration or account activation.
   3. Creating the opportunity for users to easily make referrals.

---

Experiments: We prioritized 4 RCT A/B tests to run over the following month. Our KPIs were overall registrations, TMU registrations, TMU CAC and TMU activation (account activation via post registration).

_{Control: The majority of new users came from facebook ads and were prompted to complete 3 basic registration questions, followed by 40 demographic questions. Users then had to activate their account via email before they could explore the platform. Dropoff from registration start to account activation looked like the image below.}

1. Localized Language: Treatment group was given the same registration questions, but in their local language with tooltips for words without direct translations
2. Simplified Registration: Above, plus the treatment group is given only 7 registrations questions v. 40. Also, registration is in a modal on top of any page v. on a separate page.
3. Removing Account Activation within the same session: Treatment group can take actions (enroll in courses, chat in forums, etc) within the same session without having to activate their account first.
4. Profile import: Above, plus the treatment group is given the opportunity to connect their LinkedIN, Google or Facebook account data to complete the 40 registration questions automatically, instead of manually completing each question.
5. Smart Referral: All leads are limited to 7 questions, but the treatment group is prompted to connect their email address book and empowered to automatically refer 10 peers with the same email domain.

Experiment Outcomes: In sequential order based on effort/potential impact.

• Localized Language:
  • Overall registrations: No significant change
  • TMU registrations: No significant change
  • TMU CAC: No significant change
  • TMU activations: No significant change

• Simplified Registration:
  • Overall registrations: ~2x increase in registration completions
  • TMU registrations: ~2x increase
  • TMU CAC: ~1/2, or ~$200 per TMU registration
  • TMU activations: No significant change
  • ***However, we lost a lot of valuable data by cutting registration down from 40 to 7 questions.

• Removing Account Activation within the same session:
  • Overall registrations: No change
  • TMU registrations: No change
  • TMU CAC: No change
  • TMU activations: ~3x increase

• Profile import:
  • Overall registrations: ~1.5x increase in registration completions
  • TMU registrations: ~1.5x increase
  • TMU CAC: ~3/4, or ~$300 per TMU registration
  • TMU activations: No significant change
  • ***Not as good as simple registration on its own bc some users do not have LinkedIN, Google or Facebook accounts so they didn’t reap the benefits of automatic profile creation
  • Because that population was so small, we decided to add a ‘skip’ option.

• 

• 

• Smart Referral:
  • Overall registrations: ~7x
  • TMU registrations: ~10x
  • TMU CAC: ~1/10 or $40 per TMU
  • TMU activations: No significant change

• 

• 

• 

• 

• 

• 

After about a month of experiments, we decided to build the following:

1. Simplified Registration
2. Remove Account Activation within the same session
3. Smart Referral

Overall effort was ~2 months of engineering and 1 week of product design. Pretty high, but worth it! Within 2 months of being live, we saw the following combined outcomes:

• Overall registrations: ~7x
• TMU registrations: ~14x
• TMU CAC: 1/20 or ~$20 per TMU
• TMU activations: ~5x

Network effects of smart referral were compounding. For every TMU we attracted, another 7 more would be referred and successful register. After about 4 months, we decreased TMU CAC to ~$4/user and were able to save ~$70,000 per month on ads.

Network effects were also beneficial for NPS, completion rates and skills growth because more Learners were able to create team-based learning groups or classrooms to learn alongside their peers.