TT AvData Teleconference 2023 Jul 27

Details

2023-Jul-27, 13:00-14:00UTC, Webex

Agenda

1. Welcome
2. Outcome of MIE/10:
   
3. Items for discussion:
   • In a recent discussion on the interoperability between WIS2 and (MET-)SWIM, there are several interesting findings (see Draft Guidance on Technical Specification of WIS2):
     • The IT parts (infrastructure adopted, protocol used) are basically the same.
     • However the difference in underpinning concept (Resource Oriented Architecture (ROA) Vs Service Oriented Architecture (SOA)) made the configuration and operation of the two very different.
       • WIS2 has a Global Catalogue for users to search for datasets (resources) while SWIM has a Registry for users to search for information services (services).
       • WIS2 has a Global Cache containing datasets that can be retrieved via req/rep. Dataset links are made available through Dataset discovery metadata (via Global Catalogue) as well as data notification messages (via Global Brokers with pub/sub). This arrangement is similar to Tier 1 (B2B) exchange in MET-SWIM.
     • This doesn't mean that in SOA environment there is no need (or room) for resource discovery and exchange. In previous discussions, the need of "MET Service Provider Registry" in addition to the SWIM registry (see diagram was brought up. This registry is in fact equivalent to the WIS2 Catalogue which contains information on available resources (datasets).
   • It seems that a natural way to move forward, is to retain both Tier 1 (B2B) and 2 (B2C) arrangements, with global exchange of legacy/core MET information through regional brokers in Tier 1, and direct access to providers' pub/sub and req/rep servers in Tier 2, in the end-state MET-SWIM environment.

Discussions

1. While there is no doubt that pub/sub and req/rep enables direct producer/consumer interaction, attendees are not too confident this will be the end state of MET-SWIM. In fact, consumers today tend to rely on a single service provider providing all the required information, instead of requiring oneself to fetch data here and there, and neither of the attendees are of the view that this will be changed in short terms.
2. Keeping both Tier 1 and 2 operations, on the other hand, is also not a favourable option. In fact, there are already challenges on who is going to pay for the future regional hubs (acting as pub/sub relay and/or data cache) serving similar purpose of RODBs today. There is a need to re-visit the need of regional centres after full deployment of MET-SWIM.
3. Today consumers are not familiar with SOA (i.e. looking for services instead of resources); there may be a need to provide resource discovery services (which could be a reverse lookup of the SWIM service overview, or an independent implementation of a resource registry).
4. Making reference to WIS2, more detailed definitions of the operation of MET-SWIM will have to be made.
5. In AFS, bulletin headers are required for routing of messages. With pub/sub, there is no need to set up routes for message exchange; the distribution order is intrinsically created during the subscription process, and as long as the right message enters the right MQ. Having said that, some consumers do expect to have a general idea of what the payload is, through its file name, index or otherwise, without actually opening it up.
6. Since IWXXM is basically a GML application, any of the IWXXM reports could be uniquely identified with the gml:identifier tag. This well suits the operation of req/rep services like OGC Web Services, as the latter can filter the output with the content of gml:identifier. A methodology will be required to define unique gml:identifier.
7. While there is no need for bulletins headers in full MET-SWIM, removing this from existing arrangements need complete overhaul of existing preparation, exchange and consumption systems.

Actions

1. Choy to prepare a draft SN to seek other's view on the likely infrastructure of end-state MET-SWIM, and to consolidate views on:
   • Whether there is a need to have regional hubs and inter-regional gateway and if so their use cases
   • If there is no regional hub how to enable global distribution of data
   • Is there still a benefit to have Tier 1 (peer-to-peer global distribution of baseline/essential data) and Tier 2 (consumer's direct access of the producers' extraordinary data (like high fidelity data), as well as global data (i.e. broker))?
   • What is the role of MSP and a broker?
   • The need to provide discovery services on resources
2. Matt would like to separately handle this SN and the SN/IP on bulletin and bulletin headers, although it is worth indicating the use of gml:identifier.

Attendees

• Matt Wagner
• BL Choy