OGC Best Practice for Providing Quality of Service Metadata for OGC Web Services

This is a drafting space for the QoS metadata Best Practice document.

The rationale for a new Best Practice document

This best practice specifies the key functionalities and the extended capabilities based encoding for Quality of Service metadata, including

• operational service status & times of operation,
• information about upcoming service maintenance and downtimes,
• a set of Quality of Service metrics for the entire service, like the minimum availability percentage and capacity, and
• a set of Quality of Service metrics for a specific representative operations, like the maximum response time.

The existing OGC Web Service standards do not cover declaring QoS service metadata. A new OGC Standard is not necessary, as no new or extended service operations are required. The metadata will be encoded inside the existing capabilities documents of the targeted services according to the capabilities extension mechanisms a specified in OGC Standards describing those services. The Best Practice will be written in the form of technical specification to make the it easier to implement the necessary additional functionality in the software products already compliant with the Standards documents for the implemented OGC Web Services.

"... TC subgroups, or Interoperability Initiatives may generate Best Practices (BP) Documents for the industry covering best practices related to the use of an OGC standard or other technology relevant to one or more OGC standards. A best practice is a technique or methodology that, through experience, implementation and research, has proven to reliably lead to a desired result." (OGC PnP, 8.6 Best Practices Documents)

Measuring and collecting QoS metadata for geospatial web services is not a novel idea, but there is no widely accepted technical method and format for declaring this information for the users in the context of the service capabilities, or a referred ISO 19119 metadata record. Within the INSPIRE Directive, the QoS metadata is mandatory for so called Interoperable Spatial Data Services (Interoperable SDS).

According to the "Technical Guidance for the implementation of INSPIRE dataset and service metadata based on ISO/TS 19139:2007" information about three QoS criteria:

• Availability (minimum percentage of time the service is able to provide the declared performance and capacity),
• Performance (maximum response time in a normal operational situation) and
• Capacity (minumum number of simultaneous requests that can be completed within the performance criteria in a normal operational situation).

The values for these criteria shall be declared within the service ISO 19116/19139 metadata record using the DQ_ConceptualConsistency elements as value of the gmd:report element, see InspireSDSMetadataForQoS. For other types of Network Services such as View and Download Services (implemented as WMS or WFS accordingly, among other options), similar QoS criteria fulfilling requirements are given in the Technical Guidance documents, but there is no guidance on providing the estimated values for these criteria for a service.

The Best Practice document provides a commonly agreed solutions to the following existing problems:

• It's unnecessarily complicated for the data/service providers' to enable service level monitoring of their OGC Web Services in monitoring software: declared service level criteria should only have to be configured once for each service, and communicated in a machine readable, standardized format to any external client applications.
• It's very difficult for SDI level monitoring tools, situation maps or health dashboards to automatically identify if the monitored services are behaving as expected, because different services and their operations may have very different response times in a normal situation.
• There is no standardized format or method for declaring the operating hours and scheduled maintenance or other expected downtime for an OGC Web Service instance. While 24/7/365 operation should be the goal in most cases, communicating the limitations for a service in this sense to the end users would improve the user experience.
• There is no standardized format or method for retrieving the operational status of a particular OGC Web Service instance. Declaring a service as "operative", "test", "experimental" etc. would clarify the general user expectation about the service (as long as these stata are well-defined and understandable).
• Crawlers, search engines, geoportals etc. interested in providing QoS information of the catalogued OGC Web Services may create biased availability results or unwanted strain to the monitored services by automatically enabling service availability monitoring for these services using operations and/or request parameters which are not typical for the service. It should be possible for the service to provide a list of operations and boundary conditions for request parameters used for creating representative requests for the particular service.
• The current InspireSDSMetadataForQoS within the ISO 19119/19139 service metadata as currently required by the European INSPIRE Directive is unnecessarily complicated and stretches the meaning of the DQ_ConceptualConsistency element: there should be a more straight-forward solution to provide this data.
• There should be a set of commonly agreed and well-defined metrics for measuring the quality of service or service level of OGC Web Services to enable comparisons both for the same service instance over time and between different service instances. Different testing tools should be able to to calculate similar, if not identical results for these metrics.

The scope

What's in scope:

• The XML encoding for the declared QoS metadata elements specified by XML Schemas for the core elements and the service specific bindings.
• The functional requirements for a compliant server software providing the service, including UI for publishing and maintaining the information.

Out of scope:

• The list of the actual QoS metrics to use, refer to an external codelist revision / metrics document?
• QoS measurement methods and technologies
• The Best Practice shall not mandate the use of a particular set of QoS metrics, nor shall it state acceptable values for those metrics to indicate good or bad Quality of Service.

The document structure

The requirements for the specification structure:

• It must be possible to add additional conformance classes to the BP core conformance classes for providing OGC Standard specific details for providing the QoS metadata, such as operation specific constraints defining representative operations.
• It must be possible to share common information models (such as XML Schema types and elements) specified by the core BP conformance classes for declaring the QoS metadata between the OGC Standard specific QoS metadata elements.
• It must be possible to use the QoS metadata capabilities encoding separately for a particular set of QoS metrics. The best set of metrics to use for particular services and the acceptable values for those metrics will be different in a different environments and use cases.

How many separate documents is good?

Extreme split:

• OGC Best Practice for Providing Quality of Service Metadata - OGC Web Services Core
• OGC Best Practice for Providing Quality of Service Metadata - OGC Web Map Service 1.3 Bindings
• OGC Best Practice for Providing Quality of Service Metadata - OGC Web Feature Service 2.0 Bindings
• ...
• OGC Best Practice for Key Quality of Service Metrics for INSPIRE View Services

Planned process

"In order to be considered for approval as an OGC BP, the document submitters shall provide the following: Evidence of implementation. Evidence of implementation shall include but not be limited to: Implementation in commercial product, implementation in open source applications or software, and/or implementation in deployed applications. A single research related implementation is not proper evidence of implementation." (OGC PnP, 8.6.1 Submission of documents to be considered as an OGC Best Practice)

1. Draft a complete version (0.5?) of the document suitable for implementing experimental software support for it.
2. Agree on at least two software implementations for the DB document, possibly within the OGC interoperability experiments.
3. Collect implementation and user feedback on the draft DB document
4. Create an 1.0 version of the BP document, and submit it to TC for approval

-- IlkkaRinne - 13 Jun 2017