​

Generate Metrics from Ingested Logs

This documentation will guide you through the process of creating custom metrics using middleware Logs. Custom metrics can help you gain deeper insights into your application’s behavior and performance by extracting meaningful information from your log data. Utilizing log-based metrics provides a cost-effective method for condensing log data obtained from the entire ingestion stream.By employing log-based metrics, it becomes possible to produce a count metric for logs that satisfy a specific query or a distribution metric for a numerical value found within the logs, such as the duration of requests.

To generate a new log custom metric, go to the custom metric tab from the logs explorer. There can see the list of all the generated list(if any) of Custom metrics with selected rules.

You can create metrics by selecting the “Generate new metric” button.

Add a new log-based metric

Input a query to filter the log stream: The query syntax is the same as for the Log Explorer Search. Just select the key value pairs from the dropdown to filter the logs based on the queries.

Select the measure/count : Select * to generate a count of all logs or enter a log attribute (for example, host.name) to create its corresponding count and avg aggregated metrics. If the log attribute is a countable measure, the value of the metric is the value of the log attribute.

Add dimensions/Groupings: As default, Groupings can be left empty which will then not aggregate logs based on the dimensions and will show all possible attributes in the metric section which can be further selected to see individual’s metric. If selected then only the dimensions attributes metric will be calculated.

Name your metric: Name of your metric which must not exceed 90 words limit.

Note: Data points for log-based metrics are generated at 60-second intervals. When you create a dashboard graph for log-based metrics, the count unique parameter is based on the values within the 60-second interval.

Explore Generated Metrics:

From the metric list there are 2 major selections which can be made.

On selecting the second option it will take the user to the logs listing page and will apply the attributes selected in the generated metric in queries section. This feature is just like the “Add to context” from the log info panel.

The overall metric can be explored for the selected metric by selecting the first action “See metric”. This will take the user to the metric page.

The metric Section is divided into 4 major sub-sections.

1. Breadcrumbs(Metric and Time Selection): This will show the attributes selection of the selected metric with an option to select all the generated custom metrics by the user. There is an option to select the time for which the metrics can be generated. By default it is set to 15 minutes.

2. Metric Dimension Groups: This shows the timeline and lists along with metric such as Total , Min , Max and AVG of the selected Groupings for the custom metric. This allows user to obtain metric for every dimensions and also to look for the values corresponding to the attribute by selecting one dimension. The rightside view will then be generated based on the selection. (Next Section).

3. Selected Dimension View: This view is dynamically generated based on the dimension selected by the user from the “Metric Dimension Groups” section. This shows the same metrics as before but for the values of the selected dimension along with timeline graph at intervals of 60 seconds.

Since the metrics generated are for attribute values it makes it obvious to check for related logs for that attribute value pairs. So on selection of any of the values the related logs dilogue will be displayed with ingested logs list.

4. Measure/Count Timeline: This view shows the time graph of the count/Measure of the fields selected by the user in the selected custom metric. Just like other timeline graphs this interval is also 60 seconds and shows the count of the measure value as spikes.

These are all of the sections which are currently provided for logs custom metrics. Future releases and upgrades may offer more detailed views and subsections for performing more metric related operations.