Welcome to the BigDataDwg web

Overview: [ background | goals | activities | documents ]

The purpose of the OGC Big Data DWG is to provide an open forum for work on Big Data interoperability, access, and analytics. To this end, the open forum will pursue collaborative information collection and liaisons with other Big Data working groups. The group will consolidate findings on this public wiki, and later on in a Best Practice paper, to inform OGC and the greater public. In this respect, this wiki also is meant to encourage and collect feedback.

Co-chairs are Peter Baumann (Jacobs University), John Herring (Oracle), Chuck Heazel (WISC).

This Twiki as well as the corresponding email list provide a public forum for communication. Following registration anyone can contribute to this wiki, but, of course, responsibly. Instructions can be found on the TWiki Text Formatting Rules page.

Background

“Big Data” is an umbrella term coined by Doug McLaney and IBM several years ago to denote data posing problems, summarized as the four Vs:

• Volume – the sheer size of “data at rest”
• Velocity – the speed of new data arriving (“data at move”)
• Variety – the manifold different
• Veracity – trustworthiness and issues of provenance

Several additional Vs have been suggested meantime, including value, verisimilitude, visualization. Generally it seems accepted that a core challenge is doing rapid, flexible analytics on Big Data. Major efforts are undertaken on Big Data worldwide, involving and affecting science, industry, government, and citizens alike. Manifold research and development is going on, mobilizing massive financial resources.

This development needs to see a response from OGC – in particular, as location-based and geo data form substantial contributors to the Big Data deluge. Further, with the advent of increased machine-machine communication, interoperability is gaining even more importance. OGC, therefore, should establish a position addressing all levels, including – but not limited to – science, implementation, market value, and societal effects.

BigData.DWG Goals

The OGC Big Data WG will specifically focus on spatio-temporal data, in line with OGC’s mission. With the same inaccuracy as the term “Big Data” itself, we give them the working title “Big Earth Data” for now

• What does Big Earth Data mean in an OGC context? What characterizes them?
• What are the challenges, if any, of Big Earth Data for OGC’s data and service interface specifications?
• What is the market value of Big Earth Data, and how can OGC support leveraging it?

Thus, the BigData.DWG will aim to clarify some foundational terminologies in the context of data analytics understanding differences/overlaps with terms like data analysis, data mining, etc. Further, a systematic classification of analysis algorithms, analytics tools, data and resource characteristics, and scientific queries will be established.

Key Activities

The following activities of the BigData.DWG have been spotted initially:

• Establish a working communication infrastructure, including a public wiki.
• Meet regularly at TC meetings and through telecons: 2014-mar-25 08:00, Crystal City ( TC agenda)
• Establish liaisons with relevant OGC WGs, such as WCS.SWG, and maintain exchange.
• Establish liaison with relevant OGC-external entities, such as RDA, ISO TC211 and ISO JTC1/SC32/WG3 SQL, and maintain exchange.
• Foster an agile, member-driven agenda of topics and facilitate information sharing and consolidation.
• Proactively publish discussion and findings through wiki and other appropriate channels.
• Encourage research and experiments on spatio-temporal Big Data topics.
• Encourage Big Data topics in OWS-X, AIP-X, and other suitable activities

The WG will identify additional activities as it sees fit.

Available Information

• Big Data DWG Charter (adopted), as of 2014-04-22
• Big Geo Data Analytics
• raw use case collection
• standards and bodies

BigDataDwg Web Utilities

• WebTopicList - all topics in alphabetical order
• WebChanges - recent topic changes in this web
• WebNotify - subscribe to an e-mail alert sent when topics change
• WebRss, WebAtom - RSS and ATOM news feeds of topic changes
• WebStatistics - listing popular topics and top contributors
• WebPreferences - preferences of this web