Reinventing Subscriptions (Team)
Problem
How might we address our “leaky acquisition bucket” and improve the user experience for existing customers?
Approach
Over a year, multiple scrum teams designed, built and tested a series of experiments that de-risked our transition to an opt-in subscription model.
Impact
Increased CSAT for new and existing customers. Successful pivot to opt-in product subscriptions go-to-market approach.
The Problem
During my tenure, customers repeatedly told us about the challenges of buying household products on subscription and the surprise of monthly shipments.
Several initiatives focused on improving financial results and reducing legal risk provided an opportunity to champion new UX that would benefit both business metrics on retention and acquisition — and customer satisfaction.
My Role
As head of design, I mentored the individual contributors, while also representing design in leadership.
Guided the research team to present their work in compelling visual metaphors and artifacts.
Championed transparent design process with broad cross-functional involvement.
Connected research and the design vision to provide an experience roadmap.
Communicated with executives and contributed to decision-making.
Client
Grove Collaborative
Duration
1 year (2021-2022)
My Role
Design Leader / Strategist
Team
VP of Product, CTO, CEO, Multiple EPDA Scrum Teams (3+ Product Managers, 3+ Product Designers, 10+ Engineers, 3 Data Scientists)
Design Approach
We designed a series of in-market tests to validate key assumptions and de-risk implementation.
Each phase included research, design, development, A/B tests, and analysis. Test results were compared to the legacy experience.
Click any title below to jump to that effort.
➊
Foundational user research
Goal: Capture the key pain points of the subscription model.
Impact: Earned buy-in on user needs from executives and cross-functional partners.
➋
Surfacing product subscriptions
Goal: Improve cart pain points and integrate product subscription controls.
Impact: No statistical difference in metrics. Reduced technical and design debt for future projects.
➌
Optional shipments
Goal: Give user choice and comply with Google PLA terms for acquisition experience.
Impact: Failed experiment showed customer distaste for shipments as compared to product subscriptions.
➍
Optional product subscriptions
Goal: Modify returning customer experience to educate and offer optional product subscriptions.
Impact: Successful experiment validated user preference for product subscriptions over shipments.
➎
Smart cart
Goal: Test a new cart generation model with new settings “in the wild” to compare CSAT across experiments.
Impact: Informed future data science and automation approaches.
Final impact of all experiments
The entire company pivoted in this direction and significantly streamlined the subscription models.
We improved customer satisfaction for new and existing customers, when compared to the legacy experience (4.06 versus 3.99 out of 5.00).
➏
Vision storyboard
Goal: Visualize all the workstreams to bring alive the customer lens and to show changes.
Impact: Well-received and praised. Shared with entire company.
Foundational User Research
A “loud minority”
Prior user research, NPS and customer support teams had documented complaints for years, despite increasing average order values.
Show, not tell
We designed several studies that brought the complaints alive.
I encouraged the researchers to “show not tell”: leveraging media to show the impact of our business-centric system defaults inside our customers’ homes.
Impact: Durables exempt
Thanks to our research, durables were removed from subscription generation and an audit was begun on product subscriptions defaults.
Our research led the CEO to exclaim, “We’re not just dish soap. I’m scarred.”
Users’ stockpiles made real
In one study, we visualized the pain of repeat shipments, by using the service as it was designed. We acquired 6 months of automated shipments using our default settings and presented that to the senior leadership team.
Hierarchy of commitment
In another study, we described how customers filled their shipments, creating a hierarchy of product commitment that impacted product purchase behavior.
Surface Product Subscriptions
Why need subscriptions in the cart?
Improve on pain points from the research,
Reduce technical debt of this legacy page to unlock future experimentation
Prevent full-service cancellation with more edit control access, and
Minimize time spent managing cart, creating more time for discovery of new products.
Result: No negative impact
Our A/B test showed the new design had no negative impacts to average order value or order conversion.
Challenges
How do we manage engineering lift?
Engineering leads were involved in all design reviews..
Lead designer ran regular solo sessions with them to weigh effort and impact to the technical architecture.
What did we update?
Moved promotions under in-cart items to create a “discovery zone” with previously purchased, recommended and favorited items.
Created dash for order status and gift information with the upcoming recurring order’s total and processing date. The gift widget was simplified into a square layout.
Compressed product tiles and added frequency information to subscribed items. The entire layout was reimagined and subscriptions could be edited via modal.
What if we can’t pinpoint which changes resulted in financial impact?
Ran a transparent design process with multiple reviews per week of in-progress designs.
Built stakeholders commitment to the entire vision, who did not see a path forward releasing only certain elements.
What if we lose valuable revenue by moving the carousel that’s working?
Moved carousel under cart, where customers would be more prepared to browse.
Documented our hypothesis and provided relevant user research.
Agreed to test cohorts sizes and defined success metrics for that component, before widespread changes.
Optional Flexible Monthly Shipments (FMS)
How are we going to define the product during design/build?
This high-profile test would test a whole new marketing approach, and thus requirements changed often thanks to stakeholder/executive discussions, financial modeling and test requirements.
Design was very adaptive — never holding any concept too dearly.
Product management and design held daily standing meetings (separate from standup).
What did we update?
Added FMS pricing on product detail pages to show the monetary benefits of the subscription. Because FMS applies to the order, and not individual items, we did not allow opt-in on those pages.
Added opt-in FMS widget to cart page. We also integrated value propositions relevant to customer’s most common questions.
Added Explainer modal accessible at every mention of the service. The modal provided more detail on shipment reminders and billing, thus decluttering the cart page.
How do we agree on the default shipment type?
For default FMS, we borrow from the idea of smart defaults: defaulting to the company’s recommendation. For default one-time, we would know that customer’s chose the option, rather than missing it and checking out.
With test goals focused on comprehension, the team decided to default one-time. We’d measure whether customers made the active choice to receive the recurring service.
Challenges
How do monthly shipments and product subscription work together?
This was a common pain point in our current experience and this test focused on new customer acquisition and comprehension: If we presented ourselves as a monthly subscription, would people sign up?
This test was also run concurrently with the optional product subscriptions experiment, so we could compare results.
Result: Failed experiment
Our A/B test, was fraught with issues thanks to surprisingly low acquisition and test reporting issues. Ultimately, we decided, this should not be the direction for Grove.
How does our vision impact new customers?
Understand new customers response to the choice of automated, recurring shipments at first checkout,
Provide more context during the browsing experience about shipments,
Test incentives that would encourage shipment opt-in, and
Comply with Google PLA terms, unlocking this acquisition channel.
Optional Product Subscriptions
Design & Test Goals
Understand how 2nd order customers would respond to the subscription prompt,
Understand how long-term customers would respond to controlling subscriptions for newly purchased items,
Collect data on which products and how many products customers chose to subscribe to, and
Document where customers made subscription changes.
What did we update?
Added a educational primer popup on value of subscriptions that intercepted customers on login.
Added a “Shopping list” modal of recently purchased products so opt-in to product subscriptions for automated reorder.
Added subscribe switch to cart, which inherited choices from the modal.
Result: Improved CSAT for all customers
As a result, CSAT was higher for this new product subscription experience for new users (3.99 vs 3.76) and for existing customers (4.06 vs 3.98). We asked additional questions in the CSAT survey regarding control and intent to purchase that provided mixed, but compelling, sentiment.
Challenges
How do we agree on default switch state?
Much like the optional shipments test, the debate continued: do we provide a smart default of our desired behavior or require action of the customer?
It was decided to keep subscriptions off by default for a cleaner comparison between the subscriptions and shipments tests.
What if this feature tanks our existing customer revenue?
This existing customer test would have the greatest impact on monthly revenue and order values.
Design led development of the UX copy, ensuring copy matched user context.
Presented several design prototypes to help stakeholders finalize their decision from the user experience.
How do we ensure this new fundamental subscriptions logic doesn’t impact those outside the test?
For this test, engineering would need to maintain two distinct subscription systems.
Design participated throughout engineering development/ discussions, if needed to modify the experience.
Modal experience was chosen to hide the obvious loading times.
Smart Cart
Design & Test Goals
Test a new cart generation model “in the wild” which dynamically generated carts based on subscription status and retake rates,
Gauge sentiment on the removal of subscription frequency options,
Compare CSAT across manual and automated subscriptions, and
Measure product retake rate for generated carts.
Result: Lower CSAT compared to optional subscriptions
As a result, CSAT was lower for users in the test (4.00 smart cart vs 4.06 for optional subscriptions) on all measures.
Unfortunately, I left Grove before the qualitative analysis was complete.
What did we update?
Added a educational primer screen describing cart generation, like previous tests.
Duplicated the Subscriptions Modal from the optional subscriptions test.
Added a cart tooltip that explained subscriptions for those skipping the primer or who noticed cart changes.
How do we align teams that disagree?
Most of the individual contributors wanted frequency options exposed. By removing them, designers were concerned that we were ignoring the positive results from previous tests.
However, certain executive team members felt automation eliminated the need for the frequency switch.
Following the principle of “disagree and commit”, I convinced the team that this test could prove at scale whether the user research held true.
Challenges
How do we get credit for having a “smart cart”?
Customers needed to know why their cart items may change month to month. But, we didn’t want to scare them with talk of “algorithms”.
By surfacing the factors that went into cart generation, we could point customers to which of their actions helped create their monthly cart.
How do you rally a team around a project they don’t believe in?
The design and research team believed the smart cart had little value based on user research and competitive analysis. Data scientists showed customers with 15+ subscriptions would benefit most.
Others believed that each customer’s unique needs needed a scalable, automated solution.
While acknowledging concerns, I reminded them that the only way to truly know was to test with real customers.
Vision Storyboard
My goal was to visualize all the subscriptions workstreams into a single approachable artifact, bringing alive the future customer experience.
I used a comic format with a simple color palette, short text and low-fidelity experiences. This also helped the document feel illustrative and inspirational, not prescriptive on specific details.
I illustrated two distinct journeys: new customers from new acquisition channels and existing customers who return to our shopping experience.
This was celebrated in EPDA and company forums, and I received multiple thank you notes from product managers leading initiatives.
Impact: Streamlined Experience, Improved CSAT & Trusted Team
Most of the tests were an improvement on our legacy experience. And those that failed allowed us to simplify the experience and accounting.
The team successfully pivoted our user experience — despite it seeming impossible due to the technical complexity, potential financial impacts and large stakeholder group.
Legacy
new user CSAT = 3.76
existing user CSAT = 3.98
Surfacing product subscriptions
No impact to AOV
No impact to order conversion
Optional monthly shipments
Decreased acquisition
Optional product subscriptions
new user CSAT = 3.99
existing user CSAT = 4.06
Smart Cart
existing user CSAT = 4.00
“Cart redesign [surfacing subscriptions] is very good and holy moly was that complicated... I’m proud every time I go to the site in a way I wasn’t 2 years ago.”