Drupal CMS Quarterly Check In Survey and Rollout Plan

[productmike]

How can we measure editor understanding of the separation of content from display?

Questions we might want to ask and explore include:

• How confident are you creating content (plain language/ for va.gov?
• How confident are you when you reach out to the help desk, that I'll get help promptly?
• What problems are editors trying to solve, and what is the confidence level in those actions?
• Are you confident creating alt text for images?

This is not an exhaustive list of questions; it is a jumping-off point and example of the types of things we would like to learn about as a CMS Team. However, the questions may change each time depending on specific topics we would like to aggregate data on.

Background

As we explore feedback loops and baseline health metrics for the CMS, we must define and evaluate what trust looks like for editors. Trust as a metric is crucial to understanding the overall health of the CMS. We want to understand the editor's trust level so we might understand tasks/experiences they are having problems with and determine ways the CMS Team can help them through various improvements within product initiatives.

This survey should go out at least once per quarter but more often if deemed necessary. It should also go out to all editors of the CMS. We will want to create a frequency with similar questions. The midpoint of the quarter might be next, possibly every 5th sprint.

Hypothesis

• Baseline trust metrics for the CMS will allow product teams to build solutions that advance the mission of providing quality content to Veterans.
• Editors' trust levels vary by product.
• Editors are accustomed to surveys and will participate in a quarterly survey via email.

Acceptance Criteria

• The baseline survey is created, rolled out, collected and analyzed that measures baseline trust in the CMS.

Tasks

Beta Give feedback

1. Roll out Plan for Editor Survey #18443
   CMS Team +1
   
2. Create Editor Survey into Optimal Workshop #18468
   CMS Team +1
   
3. Create Scope and Schedule for Quarterly Editor Survey #18467
   CMS Team +1
   
4. Create Comms Plan to Editors about Editor Survey #18469
   CMS Team +1
   
5. Create Runbook for VA Stakeholders about Editor Survey #18470
   CMS Team +1
   
6. Editor Survey Analysis Prep #18471
   CMS Team +1
   
7. UX Audit for Editor Survey #18921
   +0
   

[EWashb]

This epic is blocked due to external requirement of processing through VIEWS.

[EWashb]

Still in VIEWS process as of today. No new updates.

[EWashb]

Checked with my contact for the VIEWS process. This is still very early in the process. Unsure of the timeline, but I will update when I have more info.

[EWashb]

@joagnitti when you're back in the office. I had a chat with Randi H from CAIA and they are onboarding a group of new contractors to their team. Those folks will also be our benefits and Resources and Support product editors. This could be an amazing opportunity to start some research with people as they are onboarding and, throughout their tenure getting acclimated to our CMS. Let's see what we can do. They are a historically overlooked group of our editor base, but with new folks coming in, it could be prime time to get some information. cc: @BerniXiongA6

[EWashb]

Still blocked as of 10/23. Checked with Erica Robbins.

[BerniXiongA6]

cc: @maortiz-27-80 (per our convo today with VHA DM)

[EWashb]

Update: Erica Robbins and I are working with the union representative to include the proper language in our survey. There was feedback to add language, and we must include it and send it back for submission.

[EWashb]

Update: Survey memo and verbiage updated and sent back to union rep.

[EWashb]

As of 1/25/24, I am working with Erica Robbins to submit our editorial list in LEAF so that the appropriate unions can be contacted for sign off.

[EWashb]

As of 4/3/24, we have been stuck trying to find a VA contact in the org that can provide us with the individual BUS codes for union signoff. Erica and I are now reaching out to Abe George to ask for assistance. We are also in close contact with our rep in OIT.

This is being handled at the OCTO level. If a contract team has questions, please reach out to me.

[EWashb]

Update from Erica Robbins- the new memo for CoS review is pending CTO signature. Once that is completed, the memo will be uploaded into VIEWS and the package will be routed to CoS for review and concurrence. Once concurrence is obtain it will be routed to the labor unions. We do not have a good understanding of the entire timeframe we have right now, but I will update once I do.

[EWashb]

Update 5/13: CTO has tasked OIT Front Office (FO) and VA Chief of Staff (CoS) in VIEWS - requesting concurrence to route to Labor Board and disseminate survey.

[EWashb]

Survey has been routed for concurrence

[EWashb]

As of June 7, 2024 the survey has been approved through VIEWS and is able to be distributed. Let's chat about it @gracekretschmer-metrostar @MDomngz

[EWashb]

Files for Survey in the CMS Team Sharepoint

[gracekretschmer-metrostar]

@EWashb rereading the background and context for this ticket, do you still agree that we need to be focused on trust? Or do you want me to refine this ticket into a different direction?

[EWashb]

@gracekretschmer-metrostar I think that we should refocus our context towards the overall CIO goal of measuring the satisfaction of our internal customers. As a result, I hope that we can begin to understand how editors feel about the CMS and if they trust it to do what they need it to do, but baselining should really be our main focus. I see trust, now, as more of a feedback loop. They give us this feedback, we do something tangible with it, satisfaction goes up and therefore trust improves.

[aklausmeier]

@gracekretschmer-metrostar @MDomngz This is the GitHub folder that CMS research should be located vs. Confluence to allow access to be open and accessible to all. va.gov-team/platform/cms/research at master · department-of-veterans-affairs/va.gov-team (github.com)

The Sharepoint CMS UX Research folder is where would keep any PII like the editor recruitment database.

cc @EWashb