GeoMesa 3.1: A Minor Release with Major Improvements
The GeoMesa team has released LocationTech GeoMesa 3.1.0. This minor release adds new exciting features and upgrading from GeoMesa 3.0.0 should be easy.
Faster Data Integration with GeoMesa-NiFi and the NiFi Record API
Apache NiFi provides a visual programming interface to aid in creating and maintaining enterprise data flows. For years, GeoMesa has included processors that can stream data using Kafka and persist data to distributed databases like HBase and Accumulo. These processors leverage the GeoMesa converter framework, which helps one express how to transform input data into Simple Features for storage in GeoMesa.
Over the last few years, NiFi has developed and built out a Record-based API, which allows for more flexibility than relying on processors which work on entire FlowFiles. In GeoMesa 3.1.0, the GeoMesa-NiFi community module has added support to integrate with this API. Using this integration, NiFi users can create and manage mappings from NiFi RecordReader services to SimpleFeatures. For simple conversion needs, this mapping should let users create flows even more quickly to integrate new data sources into the enterprise.
Monitoring GeoServer access with Elasticsearch
GeoServer’s monitor extension provides a flexible API for capturing how users interact with the various REST and OGC web services provided by GeoServer. The GeoMesa-GeoServer community module holds code for the various integrations that the GeoMesa team has written to leverage and work with GeoServer capabilities.
New with this GeoMesa release, the GeoMesa-GeoServer community module offers an implementation of the GeoServer monitoring extension’s API that writes to Elasticsearch. You can use Kibana to build system usage dashboards and drill down into changes in user access. Using this information, system administrators can tune the GeoMesa instance appropriately.
Build More resilient database systems with the new Query Guard API
All database systems have limitations, and often users can readily overwhelm a system when they are not familiar with those limitations. A GeoMesa database typically contains at least one very large table (which can consist of trillions of records) among other tables which may contain a very small amount of contextual data. For small tables, it may be acceptable to run an inefficient query that inspects all the data. On the other hand, doing so on one of the larger tables may cause the whole system to slow down.
In previous releases, GeoMesa added timeout parameters and settings to stop queries that were taking too long. While that is useful, in GeoMesa 3.1, the new Query Guard API provides more options to address this. For each table (feature type), a collection of Query Guards can be configured. These guards run after the logical plan for the query has been calculated; each guard can stop a bad query execution plan before it is started.
GeoMesa 3.1 includes three of these guards. The Full Table Scan Query Guard prevents query plans that read all the data. The Temporal Query Guard prevents queries which have long time ranges. Lastly, the Graduated Query Guard allows for a more complex configuration that applies different time restrictions based on the spatial size of the query.
Looking to the Future
GeoMesa 3.1 delivers new minor features to the 3.x line, and all of these will benefit Optix, GA-CCRI’s enterprise offering of GeoMesa, that offers GeoSpatial Big Data management as a service. As we move forward, we have more projects in the works aimed at improving streaming processing and analysis using Apache Spark. If you are interested in contributing to GeoMesa or have questions about it, feel free to join the GeoMesa mailing list or ask questions on Gitter. To learn more about how your enterprise can leverage GA-CCRi’s expertise to manage your data and empower you to focus on your mission, drop us a note at info@ccri.com.