Skip To Content

Manage spatiotemporal big data sources

A spatiotemporal big data source can be created to store high-volume observation data. Use feature services to access the data and map services to visualize the data. Use the data source settings to manage read and write permissions, while also ensuring resiliency, redundancy, and more.

Create a spatiotemporal big data source

The spatiotemporal big data source settings are described in the tables below. Review each before creating a spatiotemporal big data source.

To create a spatiotemporal big data source, complete the following steps:

  1. Open the GeoEvent Server login page and enter your credentials.
  2. Click Site on the ribbon and click GeoEvent > Spatiotemporal Big Data Stores > Create Data Source.

    The Create data source window appears.

  3. Set the data source parameter configurations.
  4. Click the Advanced tab to configure advanced settings.
  5. Click the Service tab to configure service settings.
  6. Click the Aggregation Styles tab to configure aggregation style setting.
  7. Click the Aggregation Rendering tab to configure aggregation rendering settings.
  8. Click the Feature Rendering tab to configure feature rendering settings.
  9. Click the Time Info tab to configure time information settings.
  10. Click the Z Info tab to configure z information settings.
  11. Click Publish.

A new spatiotemporal big data source is created.

Data source configurations

The required parameters for configuring a data source are described below.

ParameterDescription
Name

The name of the data source. The name cannot be modified after the data source is created.

GeoEvent Definition

The event data definition used to define the schema of the data source. The GeoEvent Definition parameter cannot be modified after the data source is created.

Note:

Event definitions that include Group fields and fields with a cardinality of Many are not supported.

Geometry Type

The type of geometry the data source will store. Available geometry types include point, polygon, polyline, multipoint, and no geometry. The geometry type cannot be modified after the data source is created.

Max Record Count

The maximum number of features that will be rendered or queried. The max record count can be modified after the data source is created.

Advanced settings

The parameters for the Advanced tab are described below.

ParameterDescription
Replication Factor

The number of replicas, or copies, for each record to ensure resiliency. Specifying a replication factor of 0 creates only one copy, no replication. The default of 1 specifies that a single replica of the record is created. Data replication is only recommended when multiple spatiotemporal big data store nodes are available running on multiple servers for fault-tolerant operation.

Number of Shards

A data source is partitioned, or split, into multiple shards where each shard holds a portion of the data. The number of shards defines how many partitions of the data are created. Increasing the number of shards can increase indexing performance, but too many shards can reduce query performance.

Note:

You can set this parameter to Auto so the system determines the appropriate number of shards. When set to Auto, the number is calculated based on the number of nodes in the spatiotemporal big data store cluster, along with a constant factor based on the specified Rolling Data Option parameter value.

Refresh Interval (seconds)

The time interval in seconds that new or updated events are indexed in the data source. Higher values result in higher indexed events per second, improving write speed performance, but higher values also increase the latency of written events being indexed and available for querying. A value of -1 disables data indexing, for a period of time, until the value is set back to a positive number. Dynamically changing the value to and back to a positive number for a period of time is useful when there is a large number of events, allowing for bulk indexing with better performance.

ObjectID Option

Specifies the type of object identifier generated for feature records in the data source.

  • Int64 ObjectID—Generates 64-bit numeric esriFieldTypeOID object identifiers. 64-bit identifiers are supported by applications built with ArcGIS Maps SDKs for JavaScript or ArcGIS Maps SDKs for Native Apps.
  • Int32 ObjectID—Generates 32-bit numeric esriFieldTypeOID object identifiers. 32-bit identifiers are supported by all existing ArcGIS clients including ArcGIS Pro, ArcGIS Desktop, and others.
  • Unique String Identifier—Generates a string-based esriFieldTypeString identifier. String-based identifiers are supported by JavaScript clients including ArcGIS Online map viewer, custom JavaScript client applications, and more.

ObjectID Block Size

The number of object identifiers to allocate per request.

Data Option

The time after which the index is refreshed and a new index is created. This helps with the distribution of data across many data sources versus having all of the data in one source, and improves searching and querying speeds. The data can be refreshed on a century, decade, yearly, monthly, daily, or hourly basis. If GeoEvent Server is writing and storing large amounts of data to the spatiotemporal big data store, set this parameter to Hourly, Daily, or Monthly to maintain optimal performance of the spatiotemporal big data store.

Shrink Option

Specifies the shrink strategy for data shards that are refreshed. The Shrink Option parameter can be set to Shrink or No Shrink. When Shrink is selected, refreshed data is compressed into fewer shards. When No Shrink is selected, refreshed data shards are not compressed. While shrinking reduces the number of outstanding shards, which improves stability, compressing data reduces query performance. The default is Shrink.

Note:

The Shrink Option value can be updated at any time, however compressed shards cannot be decompressed. Shrinking of data occurs when set to Shrink. Set to No Shrink to avoid compressing the existing shards.

Data Retention Option (Optional)

(Optional)

The length of time data is retained in the data source. After the time has expired, data is deleted from the data source. The minimum time data can be retained is one hour.

Data Retention Where (Optional)

A WHERE clause for keeping or retaining specific features in the data source. Features that meet the WHERE clause are retained when the data retention interval has lapsed.

Service settings

The parameters for the Service tab are described below.

ParameterDescription
Create

Specifies the type of services that will be created to reference the data in the data source. Options are Map Service and Feature Service.

Spatial Reference

The spatial reference well-known ID (WKID) or well-known text (WKT). The default is 4326 (GCS WGS 1984).

Display Field Name

The name of the field that will be used as the primary label field.

Background Color

The background color of the map service. By default, the background color is fully transparent. Full transparency prevents any underlying base map from becoming obscured.

Aggregation styles setting

The parameter for the Aggregation Styles tab is described below.

ParameterDescription
Add Aggregation Style

Add a new aggregation style for the map service, or modify an existing aggregation style. Supported styles are Geohash, Square, Flat Hexagon or Triangle, and Pointy Hexagon or Triangle. For each new style, define a spatial reference and level of detail (scales).

Aggregation rendering settings

The parameters for the Aggregation Rendering tab are described below.

ParameterDescription
Feature Threshold

The default number of features rendered by the map service before switching to an aggregation. For example, setting the feature threshold to 200 means that up to 200 discrete features are rendered by the map service. Any additional discrete features cause the map service to begin rendering data in the aggregation view.

Aggregation Style

The aggregation style used to aggregate features. Options are Geohash, Square, Flat Hexagon, Pointy Hexagon, Flat Triangle, and Pointy Triangle.

Bin Rendering Style

The rendering style of the bin. Options are Grid and Oval. The default is Grid.

Aggregation Method

The method for calculating the aggregated bin value. Count defines the bin value using the raw number of features in each bin. Statistics calculates the value for each bin based on the field value from each feature and the statistic type.

Statistics Field

The field used for calculating the aggregated bin statistic value.

Statistics Type

The statistical method used for calculating the aggregated bin value. The statistics type is applied to values in the Statistics Field parameter.

The data type of the statistics field determines what type of statistical options are available. For example, choosing a string type field for the statistics field provides Count and Count Distinct as options. Choosing an integer type field provides Average, Maximum, Minimum, Standard Deviation, Sum, Variance, as well as Count and Count Distinct as options.

Show Labels

Specifies whether each aggregation bin will be labeled.

Label Style

The rendering style for labels in each aggregation bin including Font, Size, Style, Format, and Color. The default label style uses Arial, 12, plain, ###.##, and black.

Min Bin Size (pixels)

The minimum bin size by pixel count that is used when creating bins for the largest level of detail (scales). The bin pixel size is not smaller than the defined value. The default is 25 pixels.

Level of Detail (LOD) Offset

Specifies an optional offset value that is applied to the level of detail. It can be specified as a positive or negative integer value.

Bin Renderer

The rendering settings used for interpolating the aggregation bin rendering color and style.

  • Range—The minimum and maximum values that apply for the bin renderer. A bin with a value equal to the minimum of the range uses the minimum of the fill color, outline color, and outline specified. The same condition applies to bins with a value equal to the maximum of the range. All other values in-between the range minimum and maximum are weight interpolated. The range is derived from the data if a fixed value is not provided (default).
  • Fill Color—The minimum and maximum fill color of the bin. The color gradient of the bins are interpolated based on the range minimum and maximum.
  • Outline Color—The minimum and maximum outline color of the bin. The color gradient of the bin outlines are interpolated based on the range minimum and maximum.
  • Outline Width—The minimum and maximum outline width of the bin. The width of the outline for the bins is interpolated based on the range minimum and maximum.

Normalize By Bin Area

Specifies whether the bins will be normalized by geographic area.

Bins that appear the same size on a map can cover a different geographically sized area. For example, a bin at the equator covers a larger geographic area than a bin at the north pole, even though both are the same size on the map. Enabling normalization means that a bin at the equator has a lighter color, while a bin at the north pole has a darker color, assuming the fill color the maximum range value uses is a darker color than the minimum range value. Disabling normalization treats all bins as the same size for differently sized geographic areas.

Feature rendering settings

The parameters for the Feature Rendering are described below.

ParameterDescription
Style

Specifies the simple marker symbol style that will be used for features with a point or multipoint geometry. Options are Circle, Cross, Diamond, Square, X, Triangle, and Pointer.

Angle

The number of degrees (0 to 360) the simple marker symbol is rotated for point and multipoint geometry. The rotation is from east in a counterclockwise direction where east is the 0 degree axis.

Color

The color for each feature. It applies to all supported geometry types.

Size

The size of each feature with a point or multipoint geometry.

Width

The width of each feature with a polyline geometry.

Outline Color

The outline color of each feature with a point, polygon, or multipoint geometry.

Outline Width

The outline width of each feature with a point, polygon, or multipoint geometry.

X-OffSet

The value, in pixels, to offset the discrete features on the x-axis.

Y-OffSet

The value, in pixels, to offset the discrete features on the y-axis.

Rotate Symbols (degree)

Dynamically rotate the simple marker symbol for point or multipoint features.

Rotation Type

Specifies the rotation type.

  • Arithmetic—Zero is east and advances counterclockwise.
  • Geographic—Zero is north and advances clockwise.

Rotation Field

The field from the data that has the angle of rotation value in degrees (0-360).

Time information settings

The parameters for the Time Info tab are described below.

ParameterDescription
Time Interval

The time interval value representing a useful time step for the data in the data source. The time interval is used by client applications such as an ArcGIS Online web map that needs to set time-aware control settings. An example of a time-aware control setting is the step size for a time slider. The default is 10.

Time Interval Units

Specifies the units for the Time Interval value. Options are Milliseconds, Seconds, Minutes, Hours, Days, Weeks, Months, Years, Decades, and Centuries. The default is seconds.

Has Live Data

Specifies whether the data source has a live data feed. This parameter is used by client applications that require adjustments for the time-aware control settings.

Z information settings

The parameters for the Z Info tab are described below.

ParameterDescription
Has Z

Specifies whether the services have z-geometry values. The default is false.

Enable Z Defaults

Specifies whether the services have a default z-geometry value. The default is true.

Z Default

The default z-geometry value that will be used. The default is zero.