To create alerts, users need to create Monitors, Triggers, and Destinations.

1. Login to Kibana dashboard.
2. Click the icon and select the Alerting option.
3. Enter name, type, sender, and recipient details in the Destination tab.
4. Navigate to Monitors tab, create Monitor, and set the interval for the monitor to run.
5. Create trigger with trigger name and severity level.
6. Configure actions and provide Action name, Destination, and message template.

1. Add Destination

You can set the details of the recipients in the Destination field, so that the relevant alert is sent to them.

1. Navigate to the Destination tab, click the Add Destination button.
2. Enter Name of the Alert.
3. Select the Type of the destination from the drop-down, for example, email, slack, etc.
4. Select the Sender from the drop-down. If the sender name is not there, click the Manage senders option.
   1. Click the Add sender option to add new sender.
   2. Enter sender name, Email address, host, and port details.
   3. Click the Save button.
5. Select the Recipients from the drop-down. If recipients are not added, click the Manage email groups option.
   1. Click the Add email group option to add new email address/ email group address.
   2. Enter details of Email group and required email IDs.
   3. Click the Save button.

2. Create Monitor and Schedule

1. Navigate to Monitors tab.
2. Click the Create Monitor button.
3. Enter the Monitor name, for example, JIFFY Task Failure Alert.
4. Select Define using extraction query option from the Method of definition drop-down.
5. Select index as jiffy.* to scan all jiffy application logs.
6. Type the query in the Define extraction query field.
7. Enter the interval for the monitor to run in the Monitor schedule field.
8. Click the Create button.

The following query is for Task Execution Failed/Execution Time exceeded.

{
"version": true,
"size": 500,
"sort": [
{
"@timestamp": {
"order": "desc",
"unmapped_type": "boolean" } ],
"aggs": {
"2": {
"date_histogram": {
"field": "@timestamp",
"fixed_interval": "3h",
"time_zone": "Asia/Calcutta",
"min_doc_count": 1 } } },
"stored_fields": [
"*" ],
"script_fields": {},
"docvalue_fields": [
{
"field": "@timestamp",
"format": "date_time" } ],
"_source": {
"excludes": []
},
"query": {
"bool": {
"must": [],
"filter": [
{
"match_all": {}
},
{
"bool": {
"should": [
{
"match_phrase": {
"message": "Task Execution Failed" } }, {
"match_phrase": {
"message": "Execution time exceeded" } } ],
"minimum_should_match": 1
}
},
{
"bool": {
"should": [ {
"match_phrase": {
"level": "WARN" } }, {
"match_phrase": {
"level": "ERROR" } } ],
"minimum_should_match": 1 }
},{
"range": {
"@timestamp": {
"from": "now-30h",
"to": "now",
"include_lower": true,
"include_upper": false,
"boost": 1 } } } ],
"should": [],
"must_not": [] } } }

3. Trigger Alerts

All the monitors created are listed under the Trigger Section.

1. Select Name of the monitor for which you want to trigger the Alert, from the Triggers drop-down.
2. Click the Create button.
3. Enter the Action name, Destination, Message subject, and Message in the Configure Actions field.

If Conditions specified in the Monitor section are met, Alerts are triggered.

4. Configure Actions

1. Click the Add Actions option in the Configure Actions field.
2. Enter the Action Name, Destination, and message.

Sample Message Email Template

{{#ctx.results.0.hits.hits}}Monitor just entered alert status.Please investigate the issue.
LEVEL : {{_source.level}}
TAG: {{_source.TAG}}
ERROR MESSAGE: {{_source.message}}
CORRELATION ID: {{_source.correlationId}}
INDEX NAME : {{_index}}
TIME : {{_source.@timestamp}}
{{/ctx.results.0.hits.hits}}

Sample Message Teams Template

{
"text" : "{{#ctx.results.0.hits.hits}}Monitor just entered alert status.Please investigate the issue.
LEVEL : {{_source.level}}
TAG : {{_source.TAG}}
ERROR MESSAGE : {{_source.message}}
CORRELATION ID : {{_source.correlationId}}
INDEX NAME : {{_index}}
TIME : {{_source.@timestamp}}

{{/ctx.results.0.hits.hits}}"
}

When an alert is triggered, the information related to the issue is sent to the recipients specified in the Destination.

To know more about Alerting in Kibana, click here.