SEO loss on catalog

[jbrown-xentity]

Once the new catalog was released in Feb 2021, the number of page views from google decreased 70-80%. We need to investigate why this happened (ckanext-dcat misconfiguration? sitemap error? robots.txt change?) and make the necessary fixes to be indexed by Google better.

User Story

In order to understand significant changes in Data.gov page views in Google Analytics, Data.gov PMO needs to investigate possible causes and make necessary fixes.

Acceptance Criteria

• GIVEN the significant drops in page views that started to occur in early 2021
  THEN Data.gov PMO has an identified cause
  AND has identified necessary actions to correct any errors or misconfiguration.

Background

There has been significant drops in page views, when comparing the same month from year to year, in GA. See details at https://docs.google.com/spreadsheets/d/14zlpisixgr4_KrEwmRlWQsVk-0tE_o-5eRfGRzLNHqA/edit?usp=sharing

Sketch

• Review numbers on Data.gov account in GA (UA-42145528)
• Correlate with events at key times, such as launch of new version of catalog (CKAN 2.8)
• Investigate causes including change in catalog URLs

[jbrown-xentity]

Could consider improving analytics, see email on 9/28/2021 at 2:23 ET. Get a new tag.

[hkdctol]

In 2020 and prior months, google was top source, and sometime in February-March 2021, google drops off significantly and top source is direct

[hkdctol]

I looked through this time period, and there was not an obvious drop off point in the switching places between google and direct. For instance it did not appear tied to the launch of the new catalog (2/5/21)

[hkdctol]

There's also an email today about GA 4: https://analytics.google.com/analytics/web/?utm_campaign=2021-q4-gbl-all-gafree&utm_source=google-growth&utm_medium=email&utm_content=ga4-sot-ga4-setup-assistant-nonadv#/a42542568w72309185p0/admin/ga4-setup-assistant

[hkdctol]

The switch seems to have happened in mid-February 2021

[hkdctol]

Situation remains the same in recent analytics view:

[hkdctol]

Searching for some popular datasets, such as GSA federal real property, or GSA per diem, the GSA pages appear at top of search results, catalog.data.gov version not found until second page for the real property dataset, and not found at all for per diem.

[hkdctol]

Everyone on the team should have access to Google Search Console now for data.gov. There's a good DAP training on GSC at https://www.youtube.com/watch?v=uuP0FAHOrz8&list=PLd9b-GuOJ3nEz1NYl66orgVZIu17laKba&index=48

[hkdctol]

The training points out parts of Google Search Console to check for things that might prevent Google indexing

[hkdctol]

The training also emphasized submitting a sitemap in Google Search Console.

[hkdctol]

A month by month comparison is in this doc

[jbrown-xentity]

With recent sitemap publishing, we hope that this trends upward soon. If not, we should consider finding expert help in guiding how to improve SEO, not investigate how/what we broke. In the case of long term improvements, this should be considered an epic where details would need to be filled in during some type of planning session.

[hkdctol]

So far (January 2023), google as referral being low has not changed

[hkdctol]

We'll close this once we've created all the other new ones.

[hkdctol]

I think we're clear now on where we are on newly created GA4 account after our meeting today

[btylerburton]

Currently https://github.com/ckan/ckanext-googleanalytics only supports piping to a single account. We should be able to fork the repo and add support for multiple accounts.