Elasticsearch

The Elasticsearch Integration allows you to track your performance and health metrics across all clusters and nodes. This integration is useful for visualizing disk read/write operations on a per node basis, system CPU usage, and total available physical and virtual memory.

Prerequisites

1 MW Host Agent

Middleware Host Agent (MW Agent) must be installed on your local machine. To install the MW Agent, see our Installation Guide.

2 Elastic Superuser

Must be an Elastic superuser as defined here.

Setup

Step 1: Create Database Credentials

Create a .yaml file containing database credentials like the below example. If the database is not password protected, the username and password fields can be removed.

/home/ubuntu/elasticsearch-creds.yaml

If you are running the Middleware Docker Agent, keep the YAML file within the /var/log directory. Docker Agents might not support dynamic volume binding.

Step 2: Access Integrations

Log in to Middleware, navigate to the Installations Page in the bottom left corner, select All Integration and click MySQL

Step 3: Enable Integration

Add a host machine from the dropdown list, input the credential path from Step 1, and Save

Visualize Your Data

Default Elasticsearch Dashboard

Quickly access your Elasticsearch data with Middleware's default Elasticsearch dashboard. Navigate to the Dashboard Builder and select the Elasticsearch - Metrics (Runtime) Dashboard

Create Elasticsearch Widgets

Create your own widget from scratch. Navigate to the Dashboard Builder and select the dashboard you would like to create a widget in. Select Add New Widget and choose the elasticsearch data source.

Learn more about creating your own widgets and dashboards in the Dashboard Builder section.

Alerts

Alerts can be configured for any Elasticsearch metrics. When creating a new rule select the Database detection method and Elasticsearch database type for available metrics to appear in the Metrics dropdown list. Select the desired metric and continue configuring the alert conditions.

Metrics Collected

MetricDescription
elasticsearch.breaker.memory.estimatedEstimated memory used for the operation
elasticsearch.breaker.memory.limitMemory limit for circuit breaker
elasticsearch.breaker.trippedTotal number of times the circuit breaker has been triggered and prevented an out of memory error
elasticsearch.node.cache.memory.usageSize in bytes of the cache on a node
elasticsearch.node.cache.evictionsNumber of evictions from the cache on a node
elasticsearch.node.cache.countTotal count of query cache misses across all shards assigned to selected nodes
elasticsearch.node.fs.disk.availableAmount of disk space available to the JVM across all file stores for this node
elasticsearch.node.fs.disk.freeAmount of unallocated disk space across all file stores for this node
elasticsearch.node.fs.disk.totalAmount of disk space across all file stores for this node
elasticsearch.node.disk.io.readTotal number of kilobytes read across all file stores for this node
elasticsearch.node.disk.io.writeTotal number of kilobytes written across all file stores for this node
elasticsearch.node.cluster.ioNumber of bytes sent and received on the network for internal cluster communication
elasticsearch.node.cluster.connectionsNumber of open tcp connections for internal cluster communication
elasticsearch.node.http.connectionsNumber of HTTP connections to the node
elasticsearch.node.operations.completedNumber of operations completed by a node
elasticsearch.node.operations.timeTime spent on operations by a node
elasticsearch.node.shards.sizeSize of shards assigned to this node
elasticsearch.node.shards.data_set.sizeTotal data set size of all shards assigned to the node
elasticsearch.node.shards.reserved.sizePrediction of how much larger the shard stores on this node will eventually grow
elasticsearch.node.translog.operationsNumber of transaction log operations
elasticsearch.node.translog.sizeSize of the transaction log
elasticsearch.node.translog.uncommitted.sizeSize of uncommitted transaction log operations
elasticsearch.node.thread_pool.threadsNumber of threads in the thread pool
elasticsearch.node.thread_pool.tasks.queuedNumber of queued tasks in the thread pool
elasticsearch.node.thread_pool.tasks.finishedNumber of tasks finished by the thread pool
elasticsearch.node.documentsNumber of documents on the node
elasticsearch.node.open_filesNumber of open file descriptors held by the node
jvm.classes.loadedNumber of loaded classes
jvm.gc.collections.countTotal number of garbage collections that have occurred
jvm.gc.collections.elapsedApproximate accumulated collection elapsed time
jvm.memory.heap.maxMaximum amount of memory can be used for the heap
jvm.memory.heap.usedCurrent heap memory usage
jvm.memory.heap.committedAmount of memory that is guaranteed to be available for the heap
jvm.memory.nonheap.usedCurrent non-heap memory usage
jvm.memory.nonheap.committedAmount of memory that is guaranteed to be available for non-heap purposes
jvm.memory.pool.maxMaximum amount of memory can be used for the memory pool
jvm.memory.pool.usedCurrent memory pool memory usage
jvm.threads.countCurrent number of threads
elasticsearch.cluster.pending_tasksNumber of cluster-level changes that have not yet been executed
elasticsearch.cluster.in_flight_fetchNumber of unfinished fetches
elasticsearch.cluster.shardsNumber of shards in the cluster
elasticsearch.cluster.data_nodesNumber of data nodes in the cluster
elasticsearch.cluster.nodesTotal number of nodes in the cluster
elasticsearch.cluster.healthThe health status of the cluster
elasticsearch.os.cpu.usageRecent CPU usage for the whole system, or -1 if not supported
elasticsearch.os.cpu.load_avg.1mOne-minute load average on the system (field is not present if one-minute load average is not available)
elasticsearch.os.cpu.load_avg.5mFive-minute load average on the system (field is not present if five-minute load average is not available)
elasticsearch.os.cpu.load_avg.15mFifteen-minute load average on the system (field is not present if fifteen-minute load average is not available)
elasticsearch.os.memoryAmount of physical memory
elasticsearch.memory.indexing_pressureMemory consumed, in bytes, by indexing requests in the specified stage
elasticsearch.indexing_pressure.memory.total.primary_rejectionsCumulative number of indexing requests rejected in the primary stage
elasticsearch.indexing_pressure.memory.total.replica_rejectionsNumber of indexing requests rejected in the replica stage
elasticsearch.indexing_pressure.memory.limitConfigured memory limit, in bytes, for the indexing requests
elasticsearch.cluster.state_queueNumber of cluster states in queue
elasticsearch.cluster.published_states.fullNumber of published cluster states
elasticsearch.cluster.published_states.differencesNumber of differences between published cluster states
elasticsearch.cluster.state_update.countNumber of cluster state update attempts that changed the cluster state since the node started
elasticsearch.cluster.state_update.timeCumulative amount of time updating the cluster state since the node started
elasticsearch.cluster.indices.cache.evictionsNumber of evictions from the cache for indices in the cluster
elasticsearch.node.ingest.documentsTotal documents ingested during the lifetime of this node
elasticsearch.node.ingest.documents.currentTotal documents currently being ingested
elasticsearch.node.ingest.operations.failedTotal failed ingest operations during the lifetime of this node
elasticsearch.node.pipeline.ingest.documents.preprocessedNumber of documents preprocessed by the ingest pipeline
elasticsearch.node.pipeline.ingest.operations.failedTotal failed operations for the ingest pipeline
elasticsearch.node.pipeline.ingest.documents.currentTotal documents currently being ingested by a pipeline
elasticsearch.node.script.compilationsTotal inline script compilations performed by the node
elasticsearch.node.script.cache_evictionsTotal times the script cache has evicted old data
elasticsearch.node.script.compilation_limit_triggeredTotal times the script compilation circuit breaker has limited inline script compilations
elasticsearch.node.segments.memorySize of memory for segment object of a node
elasticsearch.index.operations.completedNumber of operations completed for an index
elasticsearch.index.operations.timeTime spent on operations for an index
elasticsearch.index.shards.sizeSize of the shards assigned to this index
elasticsearch.process.cpu.usageCPU usage in percent
elasticsearch.node.operations.currentNumber of query operations currently running
elasticsearch.index.documentsNumber of documents for an index
elasticsearch.process.memory.virtualSize of virtual memory that is guaranteed to be available to the running process
elasticsearch.process.cpu.timeCPU time used by the process on which the Java virtual machine is running

Troubleshooting

Missing Integrations Menu

If you do not see the Integrations Menu in Middlware, that means your account has not been granted Installation permissions. Contact your system administrator to add the Installation permission to your user role in Settings.

Next Steps

Need assistance or want to learn more about Middleware? Contact our support team in Slack.