Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Signup: Add alternative theme selection AB test #3284

Merged
merged 1 commit into from
Feb 15, 2016

Conversation

iamtakashi
Copy link
Contributor

The current default theme selection (Boardwalk, Cubic, etc.) is way overdue for a update. We need more recent themes for it and start to iterate on it and improve it. This is a first step of the effort and I'd like to test with the following new themes.

screen shot 2016-02-12 at 18 05 16

Currently, Headstart test is going on (#3233), and to avoid to break the test, this alternative theme test will only be conducted with the user who are not in the Headtstart test, and who are coming from the two button WordPress.com homepage. This might make the sample size smaller but will avoid variations that could disturb analysing the data we get.

Testing Instraction:

  1. Change active-tests.jsL77 and L86 and set it to a high number, to force the assignments.
  2. Log out if you're logged-in. (Or just use the incognito mode)
  3. Visit the signup flow /start/website/en/?ref=homepageand verify that you see the survey.
  4. Choose some categories, and verify that you see the new themes as shown in the screenshot above.
  5. Complete the signup flow and verify that the new site is created correctly.
  6. Repeat all steps with the blog flow /start/blog/en/?ref=homepage.

@iamtakashi iamtakashi added the [Status] Needs Review The PR is ready for review. This also triggers e2e canary tests and wp-desktop tests automatically. label Feb 12, 2016
@sirbrillig sirbrillig added [Feature Group] Appearance & Themes Features related to the appearance of sites. [Feature] Signup & Account Creation All screens and flows for making a new WordPress.com account. labels Feb 12, 2016
@michaeldcain
Copy link
Member

:shipit:

After you update the test date, that is :)

@michaeldcain michaeldcain added [Status] Ready to Merge and removed [Status] Needs Review The PR is ready for review. This also triggers e2e canary tests and wp-desktop tests automatically. labels Feb 12, 2016
@lucasartoni
Copy link
Contributor

Change active-tests.jsL77 and L86 and set it to a high number, to force the assignments.

you can also force the assignment from your JS console:

localStorage.setItem('ABTests','{"altThemes_20160211":"altThemes"}')

@iamtakashi
Copy link
Contributor Author

you can also force the assignment from your JS console:

A nice tip!

Signup: Change the commence date for default theme A/B test
iamtakashi added a commit that referenced this pull request Feb 15, 2016
Signup: Add alternative theme selection A/B test
@iamtakashi iamtakashi merged commit 15b8aaf into master Feb 15, 2016
@iamtakashi iamtakashi deleted the add/alt-themes-test branch February 15, 2016 20:46
@melchoyce
Copy link
Contributor

Love these theme selections 🙌

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
[Feature Group] Appearance & Themes Features related to the appearance of sites. [Feature] Signup & Account Creation All screens and flows for making a new WordPress.com account.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants