Plans: Use 2year and 3year default interval type #96174

chriskmnds · 2024-11-08T01:20:38Z

Related to https://github.com/Automattic/martech/issues/3403
Branched from #96357 (term-savings logic)

Proposed Changes

Introduces pbxNRc-4tx-p2 that will show longer term plans by default in the plans grid.
Experiment variants:
- Control: Current pricing page, defaulted to annual plans
- Treatment 1: Savings emphasized, grid defaults to annual plans
- Treatment 2: Savings emphasized, grid defaults to 2-year plans
- Treatment 3: Savings emphasized, grid defaults to 3-year plans
Adds a new experiment hook use-longer-plan-term-default-experiment.ts
See Plans: Enable term savings price display - as a grid feature #96357 for the term-savings logic (this PR builds on top to hook the experiment and add the default term)

Media

Why are these changes being made?

Low-tier, Personal and Premium Plan customers represent, by volume, the majority of dotcom plan purchases. With the Personal Plan alone, 45-50% of all users purchase said plan at signup. When considering monthly, yearly, 2 year, and 3 year billing terms, 35% of all users purchase an annual Personal Plan at signup.

Our low-tier users, however, are extremely price sensitive.

One potential way to address price sensitivity is to make our 2 year and 3 year plan discounts more prominent. By showing longer term plans on page load ( Ex. 2 year billing ), the plans grid monthly costs will render with 20% lower prices than with annual billing.

See pau2Xa-6wD-p2 for more context

Testing Instructions

Switch to USD and go to /start/plans and /plans/:site. Confirm:
- For treatment variations:
  - Treatment 1: plans grid defaults to annual plans (and term in toggle) with savings emphasized on term yearly+
  - Treatment 2: plans grid defaults to 2-year plans (and term in toggle) with savings emphasized on term yearly+
  - Treatment 3: plans grid defaults to 3-year plans (and term in toggle) with savings emphasized on term yearly+
- For control variant:
  - plans grid defaults to annual plans (and term in toggle) with no savings emphasized

Pre-merge Checklist

Has the general commit checklist been followed? (PCYsg-hS-p2)
Have you written new tests for your changes?
Have you tested the feature in Simple (P9HQHe-k8-p2), Atomic (P9HQHe-jW-p2), and self-hosted Jetpack sites (PCYsg-g6b-p2)?
Have you checked for TypeScript, React or other console errors?
Have you used memoizing on expensive computations? More info in Memoizing with create-selector and Using memoizing selectors and Our Approach to Data
Have we added the "[Status] String Freeze" label as soon as any new strings were ready for translation (p4TIVU-5Jq-p2)?
For changes affecting Jetpack: Have we added the "[Status] Needs Privacy Updates" label if this pull request changes what data or activity we track or use (p4TIVU-aUh-p2)?

github-actions · 2024-11-08T01:20:50Z

Calypso Live (direct link)

	https://calypso.live?image=registry.a8c.com/calypso/app:build-125570

Jetpack Cloud live (direct link)

	https://calypso.live?image=registry.a8c.com/calypso/app:build-125570&env=jetpack

Automattic for Agencies live (direct link)

	https://calypso.live?image=registry.a8c.com/calypso/app:build-125570&env=a8c-for-agencies

matticbot · 2024-11-08T01:45:16Z

Here is how your PR affects size of JS and CSS bundles shipped to the user's browser:

App Entrypoints (~134 bytes removed 📉 [gzipped])

name                   parsed_size           gzip_size
entry-stepper               -585 B  (-0.0%)     -134 B  (-0.0%)
entry-main                  -504 B  (-0.0%)     -110 B  (-0.0%)
entry-subscriptions         -126 B  (-0.0%)      -52 B  (-0.0%)
entry-login                 -126 B  (-0.0%)      -52 B  (-0.0%)
entry-domains-landing       -126 B  (-0.0%)      -52 B  (-0.0%)
entry-browsehappy           -126 B  (-0.1%)      -52 B  (-0.1%)

Common code that is always downloaded and parsed every time the app is loaded, no matter which route is used.

Sections (~17009 bytes added 📈 [gzipped])

name                                parsed_size           gzip_size
update-design-flow                      +7660 B  (+0.6%)    +2577 B  (+0.8%)
link-in-bio-tld-flow                    +7633 B  (+0.4%)    +2492 B  (+0.5%)
jetpack-app                             +7551 B  (+1.9%)    +2887 B  (+2.2%)
plugins                                 +7471 B  (+0.2%)    +2580 B  (+0.3%)
plans                                    +669 B  (+0.0%)     -239 B  (-0.1%)
jetpack-connect                          -196 B  (-0.0%)     -693 B  (-0.2%)
checkout                                 -196 B  (-0.0%)     -966 B  (-0.2%)
signup                                   +176 B  (+0.1%)       +2 B  (+0.0%)
jetpack-cloud-pricing                    -103 B  (-0.0%)     -256 B  (-0.1%)
gutenberg-editor                          -81 B  (-0.0%)      -25 B  (-0.0%)
with-theme-assembler-flow                 +27 B  (+0.0%)       -1 B  (-0.0%)
update-options-flow                       +27 B  (+0.0%)       -1 B  (-0.0%)
trial-wooexpress-flow                     +27 B  (+0.0%)       -1 B  (-0.0%)
tailored-ecommerce-flow                   +27 B  (+0.0%)       -1 B  (-0.0%)
start-writing-flow                        +27 B  (+0.1%)       +8 B  (+0.1%)
site-setup-wg                             +27 B  (+0.0%)       -1 B  (-0.0%)
site-setup-flow                           +27 B  (+0.0%)       -1 B  (-0.0%)
site-migration-flow                       +27 B  (+0.0%)       -2 B  (-0.0%)
reblogging-flow                           +27 B  (+0.4%)       +8 B  (+0.4%)
readymade-template-flow                   +27 B  (+0.0%)       -1 B  (-0.0%)
onboarding-flow                           +27 B  (+0.1%)       +8 B  (+0.1%)
newsletter-flow                           +27 B  (+0.1%)       +8 B  (+0.1%)
new-hosted-site-flow-user-included        +27 B  (+0.4%)       +8 B  (+0.4%)
new-hosted-site-flow                      +27 B  (+0.3%)       +7 B  (+0.3%)
migration-signup                          +27 B  (+0.0%)       -2 B  (-0.0%)
migration-flow                            +27 B  (+0.0%)       -2 B  (-0.0%)
import-flow                               +27 B  (+0.0%)       -1 B  (-0.0%)
hosted-site-migration-flow                +27 B  (+0.0%)       -2 B  (-0.0%)
free-post-setup-flow                      +27 B  (+0.0%)       +0 B
entrepreneur-flow                         +27 B  (+0.0%)       -1 B  (-0.0%)
design-first-flow                         +27 B  (+0.1%)      +11 B  (+0.2%)
connect-domain                            +27 B  (+0.1%)       +9 B  (+0.1%)
assembler-first-flow                      +27 B  (+0.0%)       +0 B
ai-assembler-flow                         +27 B  (+0.0%)       +0 B
hosting                                   +13 B  (+0.0%)      -18 B  (-0.0%)

Sections contain code specific for a given set of routes. Is downloaded and parsed only when a particular route is navigated to.

Async-loaded Components (~4995 bytes added 📈 [gzipped])

name                                             parsed_size           gzip_size
async-load-signup-steps-plans-theme-preselected      +7799 B  (+2.0%)    +2973 B  (+2.3%)
async-load-signup-steps-plans                        +7799 B  (+2.0%)    +2973 B  (+2.4%)
async-load-calypso-my-sites-checkout-modal            -103 B  (-0.0%)     -544 B  (-0.2%)
async-load-calypso-blocks-editor-checkout-modal       -103 B  (-0.0%)     -571 B  (-0.2%)
async-load-signup-steps-design-picker                  -81 B  (-0.1%)      -25 B  (-0.1%)

React components that are loaded lazily, when a certain part of UI is displayed for the first time.

Legend

What is parsed and gzip size?

Parsed Size: Uncompressed size of the JS and CSS files. This much code needs to be parsed and stored in memory.
Gzip Size: Compressed size of the JS and CSS files. This much data needs to be downloaded over network.

Generated by performance advisor bot at iscalypsofastyet.com.

jeyip · 2024-11-12T22:58:46Z

This PR is on pause until further discussion about technical implementation is resolved in p9Jlb4-e5j-p2#comment-14161

chriskmnds · 2024-11-20T09:56:41Z

...nt/my-sites/plans-features-main/hooks/experiments/use-longer-plan-term-default-experiment.ts

+const useLongerPlanTermDefaultExperiment = (): {
+	isLoadingExperiment: boolean;
+	term?: string | null;
+	// TODO: Consider removing this and always return concrete term values (where undefined/null would mean no term savings)


I'll study this a bit more. I want to make the "clean up" phase after the experiment as easy as possible, e.g., this hook remaining to define the default term/interval type.

chriskmnds · 2024-11-20T15:26:14Z

@jeyip I think we are ready here to test. The experiment should be hooked to the logic. I still need to investigate something, but not blocking.

jeyip · 2024-11-21T01:58:23Z

Wonderful! Thanks for your work here @chriskmnds. Will review this tomorrow morning.

jeyip · 2024-11-22T00:22:14Z

Testing

Requirements

Used bookmarklets in 22092-explat-experiment to assign myself to different treatments and control

For each of the flows listed below:

✅ Treatment 1: plans grid defaults to annual plans (and term in toggle) with savings emphasized on term yearly+
✅ Treatment 2: plans grid defaults to 2-year plans (and term in toggle) with savings emphasized on term yearly+
✅ Treatment 3: plans grid defaults to 3-year plans (and term in toggle) with savings emphasized on term yearly+
❌ For control variant: plans grid defaults to annual plans (and term in toggle) with no savings emphasized
- I'm still seeing the longer plan term savings emphasized in the header price. The bookmarklet was acting a little oddly though in that it wouldn't successfully assign at times, so it might be on my end. If you wouldn't mind trying the control variant assignment, we can confirm if it's a local issue for me.

Flows:

/start
/plans

Browsers

Chrome

Notes

I realized that the savings label is showing for the logged in spotlight plan. It can be handled in a separate PR. I believe that we can remove it in that case, but lemme know if you think otherwise.

...nt/my-sites/plans-features-main/hooks/experiments/use-longer-plan-term-default-experiment.ts

chriskmnds · 2024-11-22T10:45:54Z

@jeyip control variation should be fixed now, if we can give it another round of testing. I guess we can also ship this if it tests out right?

concerning:

I realized that the savings label is showing for the logged in spotlight plan. It can be handled in a separate PR. I believe that we can remove it in that case, but lemme know if you think otherwise.

Shouldn't that be the case? i.e. show the savings if user is looking to upgrade from yearly to biennial (etc.)? 🤔

matticbot · 2024-11-22T10:58:13Z

This PR modifies the release build for the following Calypso Apps:

For info about this notification, see here: PCYsg-OT6-p2

notifications

To test WordPress.com changes, run install-plugin.sh $pluginSlug update/plans-grid-longer-term-default on your sandbox.

jeyip · 2024-11-22T20:37:56Z

@jeyip control variation should be fixed now, if we can give it another round of testing. I guess we can also ship this if it tests out right?

Yes will test again before I sign off today :)

jeyip · 2024-11-22T20:40:54Z

Shouldn't that be the case? i.e. show the savings if user is looking to upgrade from yearly to biennial (etc.)? 🤔

Oh man that's silly of me 🤦 For some reason, I thought I was on the same term view as the current plan ( annual current plan in the annual view ) and missed the biennial upgrade CTA. Yes you're right we should show savings in that case.

jeyip · 2024-11-25T06:57:14Z

For control variant: plans grid defaults to annual plans (and term in toggle) with no savings emphasized

I smoke tested the all treatments and the control again. Can confirm that everything works well now 🙂

jeyip · 2024-11-25T07:00:31Z

...nt/my-sites/plans-features-main/hooks/experiments/use-longer-plan-term-default-experiment.ts

+		isLoadingExperiment: isLoadingExperimentAssignment,
+		term: isLoadingExperimentAssignment ? undefined : term,
+		isEligibleForTermSavings: !! (
+			experimentAssignment?.variationName && experimentAssignment.variationName !== 'control'


Wondering if I'm missing something, but if variationName returns null for the control, do we still check if experimentAssignment.variationName !== 'control' here?

Wouldn't just the experimentAssignment?.variationName check be enough?

do we still check if experimentAssignment.variationName !== 'control' here?

We don't (probably?). I kept it out of reaction for the most part. was a bit surprised how control is treated as a non-entity/null variation. Almost like a type-safe system is missing internally. I guess no harm in keeping the extra check?

I kept it out of reaction for the most part. was a bit surprised how control is treated as a non-entity/null variation.

Sounds good 🙂

chriskmnds assigned jeyip and chriskmnds Nov 8, 2024

jeyip added the Multi-year Plans by Default Experiment label Nov 8, 2024

chriskmnds force-pushed the update/plans-grid-longer-term-default branch from 4450634 to a0abfcc Compare November 9, 2024 02:46

chriskmnds mentioned this pull request Nov 13, 2024

Plans: Enable term savings price display - as a grid feature #96357

Merged

9 tasks

chriskmnds force-pushed the update/plans-grid-longer-term-default branch from a0abfcc to 009f03e Compare November 19, 2024 08:39

chriskmnds changed the base branch from trunk to update/plans-grid-term-savings-price-display November 19, 2024 08:40

chriskmnds commented Nov 20, 2024

View reviewed changes

chriskmnds marked this pull request as ready for review November 20, 2024 15:19

chriskmnds requested a review from jeyip November 20, 2024 15:24

matticbot added the [Status] Needs Review The PR is ready for review. This also triggers e2e canary tests and wp-desktop tests automatically. label Nov 20, 2024

jeyip reviewed Nov 22, 2024

View reviewed changes

...nt/my-sites/plans-features-main/hooks/experiments/use-longer-plan-term-default-experiment.ts Show resolved Hide resolved

jeyip approved these changes Nov 25, 2024

View reviewed changes

jeyip reviewed Nov 25, 2024

View reviewed changes

chriskmnds force-pushed the update/plans-grid-term-savings-price-display branch from 021e5d8 to 67bd5de Compare November 25, 2024 09:14

chriskmnds requested a review from a team as a code owner November 25, 2024 09:14

chriskmnds requested a review from aneeshd16 November 25, 2024 09:14

Base automatically changed from update/plans-grid-term-savings-price-display to trunk November 25, 2024 09:38

Plans: Use 2year and 3year default interval type

52f2936

jeyip and others added 7 commits November 25, 2024 11:39

Implement explat assignment in useLongerPlanTermDefaultExperiment hook

f6afdf3

Current plan with variation

f35e137

fix up experiment code - still intermediate

505e6fb

fix up experiment code - still intermediate

f2302d5

enable term-savings via experiment

83e7675

fix tests

26e28a3

fix control variant conditioning for term-savings

32c9f83

chriskmnds force-pushed the update/plans-grid-longer-term-default branch from 3090e5f to 32c9f83 Compare November 25, 2024 09:39

chriskmnds merged commit 1f562c3 into trunk Nov 25, 2024
11 checks passed

chriskmnds deleted the update/plans-grid-longer-term-default branch November 25, 2024 09:54

github-actions bot removed the [Status] Needs Review The PR is ready for review. This also triggers e2e canary tests and wp-desktop tests automatically. label Nov 25, 2024

chriskmnds mentioned this pull request Nov 25, 2024

Plans: Fix error from undefined longerPlanTermDefaultExperiment in Stepper use of plans step #96736

Merged

8 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Plans: Use 2year and 3year default interval type #96174

Plans: Use 2year and 3year default interval type #96174

chriskmnds commented Nov 8, 2024 •

edited

Loading

github-actions bot commented Nov 8, 2024 •

edited by matticbot

Loading

matticbot commented Nov 8, 2024 •

edited

Loading

jeyip commented Nov 12, 2024

chriskmnds Nov 20, 2024

chriskmnds commented Nov 20, 2024

jeyip commented Nov 21, 2024

jeyip commented Nov 22, 2024 •

edited

Loading

chriskmnds commented Nov 22, 2024

matticbot commented Nov 22, 2024 •

edited

Loading

jeyip commented Nov 22, 2024

jeyip commented Nov 22, 2024 •

edited

Loading

jeyip commented Nov 25, 2024 •

edited

Loading

jeyip Nov 25, 2024 •

edited

Loading

chriskmnds Nov 25, 2024 •

edited

Loading

jeyip Nov 25, 2024

Plans: Use 2year and 3year default interval type #96174

Plans: Use 2year and 3year default interval type #96174

Conversation

chriskmnds commented Nov 8, 2024 • edited Loading

Proposed Changes

Media

Why are these changes being made?

Testing Instructions

Pre-merge Checklist

github-actions bot commented Nov 8, 2024 • edited by matticbot Loading

matticbot commented Nov 8, 2024 • edited Loading

jeyip commented Nov 12, 2024

chriskmnds Nov 20, 2024

Choose a reason for hiding this comment

chriskmnds commented Nov 20, 2024

jeyip commented Nov 21, 2024

jeyip commented Nov 22, 2024 • edited Loading

Testing

Requirements

Browsers

Notes

chriskmnds commented Nov 22, 2024

matticbot commented Nov 22, 2024 • edited Loading

jeyip commented Nov 22, 2024

jeyip commented Nov 22, 2024 • edited Loading

jeyip commented Nov 25, 2024 • edited Loading

jeyip Nov 25, 2024 • edited Loading

Choose a reason for hiding this comment

chriskmnds Nov 25, 2024 • edited Loading

Choose a reason for hiding this comment

jeyip Nov 25, 2024

Choose a reason for hiding this comment

chriskmnds commented Nov 8, 2024 •

edited

Loading

github-actions bot commented Nov 8, 2024 •

edited by matticbot

Loading

matticbot commented Nov 8, 2024 •

edited

Loading

jeyip commented Nov 22, 2024 •

edited

Loading

matticbot commented Nov 22, 2024 •

edited

Loading

jeyip commented Nov 22, 2024 •

edited

Loading

jeyip commented Nov 25, 2024 •

edited

Loading

jeyip Nov 25, 2024 •

edited

Loading

chriskmnds Nov 25, 2024 •

edited

Loading