SoilDataFederator

Australia is fortunate to have a large number of soil profile data observations and measurements publicly available for use in an ever-increasing range of applications. This data is collected and managed by a broad range of custodians across the country. These custodians collect the data for their own specific business purposes and manage it in a disparate range of data management environments. Until now individuals wanting to bring this data together in a unified way had to source data from each of the individual custodians, on a case by case basis, and transform it to a useful form for their specific application. A challenging task rarely undertaken with any degree of excitement.

At a conceptual level, there is a broad spectrum of approaches through which data unification can be achieved, from the creation of a centralised behemoth database through to the case by case collation of datasets. The “SoilDataFederator” is a federation approach to data unification, where data is managed by custodians but is federated on the fly to into a consistent form

The SoilDataFederator.is a web application programming Interface (API) implemented in the R programming language. The API is used to query data over the internet via a standardised set of URLs with standardised parameters. Data can be returned in a range of formats but always in a standard form optimised for delivering data on a per attribute basis. The SoilDataFederator consists of a catalogue of available datasets and a series of associated “backend” modules which query the individual data systems and transform the data on the fly to the standard form.

The code base is publicly available HERE with the idea that the soil community will develop it further in the future.

Demo R code available HERE