Removing the 'Experimental' tag

[CharlotteDowns]

What

Consider removing the 'Experimental' tag.

Definitions of the 'Experimental' tag available to users

Most new components and patterns will be published as experimental at first. The Design System team will remove the experimental status once research proves the component or pattern meets user needs. The research must be from different services and with different types of users.

Components and patterns which are not proven usable can be published as experimental. But they must be clearly based on relevant user research from other organisations and best practice, and meet the other criteria.

Why

We don't seem to have a complete, reliable source on the origin of the tag that's written down.

It's also not clear what user research criteria a component/pattern/guidance must prove in order to meet user needs. This has meant that some component/pattern/guidance have been tagged as 'Experimental' for a number of years.

Work on visually differentiating WCAG 2.2 guidance seems to overlap with space on the page, colour of the tag and the need for a strategy around when it is removed.

Comments seen in previously conducted team User Research

Going back to your earlier question [about the 'Experimental tag'], it would be quite nice to be able to understand what that actually means. So that links to an explanation about what is an experimental feature.

I get [experimental]. But like, it's still more research in more depth than I've ever seen on a tab on the internet to me...

Experimental - Not ready yet. error potential

I would be more inclined to feedback on experimental patterns than standard ones P32 ... Experimental: you can use it but it might not be exactly what you are looking for P32 ... How do you keep up to date with an experimental pattern? P32

experimental seems vague, right? It feels like- It's done or it's not done. P55

That’s interesting because I would have to know that it’s experimental, so I would have to do my own research first. But if that was the case I would want to see that they were recording their evidence and sharing it back to the community. I think people should be experimenting and stuff but… you also want things to be consistent. I think with regards to patterns and assessments to the things that are people not using them and not feeding them back into the community. So, if you are going to do this, let other people know because they can also learn from it, it’s just communication across the community. P56

with an experimental tag, solves a problem that we know and understand, we want people to start using it and see if we’re correct in our assumptions

Conversations that have happened while team members are on support

"one support interaction where a user wanted to know if they could use our bank details pattern or "if it was still experimental". The subtext there is that they presumed:

• experimental meant that it wasn't usable in services
• there was a process for it not being experimental anymore

I explained this to them and I think they got a bit confused about what experimental meant for their service and their service assessment."

"I think a question is, how should teams treat Experimental components or patterns differently to non Experimental? Some suggestions, I have no strong opinion on them:

• a team shouldn't be required to use them in a service assessment
• a team should do their own research specifically on this component/pattern

came up on support today, whether you need to start with an experimental pattern in order to pass assessment"

Resources to investigate the issue

• A google sheet of all the experimental components/patterns/guidance and when they were labelled as 'Experimental'

Who needs to work on this

Charlotte, Volunteers

Who needs to review this

The whole team

Done when

• Find references to the 'Experimental' tag in user research we have already run
• Determine why we have made the decision to tag whole things as 'Experimental' or not experimental in components/patterns/guidance released in the last 3 years
• Ask the GOV.UK Design System working group for their thoughts on the parameters of the 'Experimental' tag
• Log all 'Experimental' components/patterns/guidance and when they were first published, including anything that has had a label removed
• Open a pull request to remove the 'Experimental' tag
• Decide whether to merge the pull request based on team confidence and working group feedback
• Prepare comms message for Slack and backlog items that were affected (new card)

Stretch goal

• If the pull request gets merged, draft a proposal for the working group to review on process tagging which I will cover in a new sprint goal

[CharlotteDowns]

This work seems to be similar in nature to this previous epic Make "Experimental" tag more understandable or remove #1093 – December 2019

We added "status" and "statusMessage" in frontmatter in this pull request to allow us to start tagging components/patterns/guidance as 'Experimental' – January 2018

[CharlotteDowns]

Definitions from team members

• As far as I can tell from the history, the tag was introduced as a way to add new components and patterns where we didn’t have concrete data but the team was confident enough and the risk was low enough that an element could be introduced anyway.
• The original idea was to give ourselves permission to publish things that we had high confidence in, but that hadn’t technically been tried and tested in multiple services yet. because sometime that will never happen until you put it in the design system.

Reasons/arguments to remove

• Users constantly misinterpret what it means

Reasons/arguments to keep

• It would be good to actually document it in a place where users can easily see it. At this point I feel like it's makes loads more sense to design and document a tagging system, if we need one at all, from the ground up.
• I don’t think the tag is a problem, but I do think that we don’t review and iterate the catalogue as much as we could. Having a process for something moving from 'Experimental' to 'Stable' or 'Supported' would be good.

[StephenGill]

FWIW, I'd remove it.

As people have said, it causes confusion.

Plus it's not really a useful label on its own terms. Being experimental or non-experimental isn't a binary thing: you can have a high degree of confidence that something will work in one context, but not another.

The important thing is to be transparent about the contexts in which the pattern has been tried, and any known limitations. The 'Research' section already does that: having an 'Experimental' tag just takes attention away from the nuance.

[CharlotteDowns]

First of all, thank you @StephenGill for supporting this :).

We have done some additional research and are confident that there has never been an instance where an experimental tag has been removed from an experimental entity in the catalogue.

The team has searched closed and open pull requests and the website commit history and they suggest that we have only ever added the experimental tag, never removed it.

[CharlotteDowns]

We do have another tagged status: Being worked on that is currently applied to the Task list pages pattern because a cross-government group is collaborating on work to update this pattern and build it as a component. The creation of this tag was done as part of Update Task List pattern to show it is being worked on #2105.

Work to remove the status: Experimental is scoped only to the 'Experimental' tag.

[frankieroberto]

💯 agree, it should be dropped. Far better to note any concerns about aspects of the component or pattern in context within the page.

[stevenjmesser]

I’d like to see #2580 done and published before we remove the tag. We need to let people know how to share research and findings that back up a component or pattern as usable.

[CharlotteDowns]

Working group thoughts and suggestions on the definition and purpose of the tag

Knee-jerk response: It denotes a component or pattern that has yet to be tested with users or is currently undergoing said testing. This means the component or pattern should be thoroughly tested in our own work before releasing it in the wild and we should feed back our findings to better inform the Design System team and others regarding its use or inclusion in the design system. (Something like that anyway! 😊)

For me, ‘Experimental’ means the pattern or component hasn’t met a certain threshold of acceptability. But what is that threshold? Is it three services? Is it at least one service in three departments? Etc.
What are the acceptance criteria?
The difficulty, as I see it, is that one service could have rigorously researched a component or pattern, but it would still be experimental for every other service.
If a pattern or component is accepted into the design system, do we need to give it a status (e.g., ‘Experimental’) rather than guidance? Can we simply rely on a service team to make a qualitative assessment of the pattern or component’s validity?

I interpret "experimental" to mean it is likely to change in the near future. GDS are still experimenting with it and might make changes based on the results of those experiments.
I would expect those changes to be small tweaks, as a certain threshold of quality and design must have already been met for it to be on the Design System in the first place.

[stevenjmesser]

Bit meta, but was the Experimental tag added as an experiment?

[frankieroberto]

I think it was used as a bit of a hedge...? Have there been any components or patterns which’ve ever had the "experimental" tag removed? I can't think of any...

[CharlotteDowns]

@frankieroberto we did an investigation and there doesn't seem to be any that have been removed.

I think the conclusions I'm coming to are:

• although there seems to be a broad understanding that the tag suggests that more user research is needed, there seems to be discrepancy in the overall definition, purpose of the tag and how the tagged component, pattern or guidance should be used as a result.
• when it has been used in the Images | Styles and Resources and tools | Community pages it maybe has a different set of criteria for what 'Experimental' means in these places or at least a different interpretation of what the user research looks like
• components and patterns where we have updated the <h2>Research on this component</h2> sections, for example, Details, Select etc. should have probably been relabelled as 'Experimental'.

[CharlotteDowns]

Decide whether to merge the pull request based on team confidence and working group feedback

On the basis of the evidence in this thread, I think we should merge the pull request to remove the tag.

As the tag has been in use since May 2018 we will let users know about it's removal on Slack and in the backlog, pointing users to our recent guidance on how to share user research findings.