Skip to content

Using elastalert with appmetrics-elk #28

@ineshll1993

Description

@ineshll1993

I got this error when using elastalert with appmetrics-elk:
the command elastalert-test-rule example_rules/nodejs.yaml returns:
Successfully loaded Metrics nodejs application, Metric aggregation Rule

WARNING:elasticsearch:GET http://ip:9200/nodemetrics/_search?ignore_unavailable=true&size=1 [status:400 request:0.023s]
Error running your filter:
RequestError(400, u'search_phase_execution_exception', {u'status': 400, u'error': {u'failed_shards': [{u'node': u'qzWI8cb2RViQxqa3FYOIeA', u'index': u'nodemetrics', u'reason': {u'index_uuid': u'xHZOA681QgyxSBSljHlqyA', u'index': u'nodemetrics', u'reason': u'No mapping found for [@timestamp] in order to sort on', u'type': u'query_shard_exception'}, u'shard': 0}], u'root_cause': [{u'index_uuid': u'xHZOA681QgyxSBSljHlqyA', u'index': u'nodemetrics', u'reason': u'No mapping found for [@timestamp] in order to sort on', u'type': u'query_shard_exception'}], u'grouped': True, u'reason': u'all shards failed', u'phase': u'query', u'type': u'search_phase_execution_exception'}})
INFO:elastalert:Note: In debug mode, alerts will be logged to console but NOT actually sent. To send them, use --verbose.
WARNING:elasticsearch:GET http://ip:9200/nodemetrics/cpu/_search?ignore_unavailable=true&size=0 [status:400 request:0.005s]
ERROR:root:Error running query: TransportError(400, u'search_phase_execution_exception', u'Fielddata is disabled on text fields by default. Set fielddata=true on [hostName] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory. Alternatively use a keyword field instead.')

Would have written the following documents to writeback index (default is elastalert_status):

elastalert_error - {'message': "Error running query: TransportError(400, u'search_phase_execution_exception', u'Fielddata is disabled on text fields by default. Set fielddata=true on [hostName] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory. Alternatively use a keyword field instead.')", 'traceback': ['Traceback (most recent call last):', ' File "/usr/local/lib/python2.7/dist-packages/elastalert/elastalert.py", line 512, in get_hits_aggregation', " res = self.current_es.search(index=index, doc_type=rule.get('doc_type'), body=query, size=0, ignore_unavailable=True)", ' File "/usr/local/lib/python2.7/dist-packages/elasticsearch/client/utils.py", line 73, in _wrapped', ' return func(*args, params=params, **kwargs)', ' File "/usr/local/lib/python2.7/dist-packages/elasticsearch/client/init.py", line 623, in search', " doc_type, '_search'), params=params, body=body)", ' File "/usr/local/lib/python2.7/dist-packages/elasticsearch/transport.py", line 312, in perform_request', ' status, headers, data = connection.perform_request(method, url, params, body, ignore=ignore, timeout=timeout)', ' File "/usr/local/lib/python2.7/dist-packages/elasticsearch/connection/http_requests.py", line 90, in perform_request', ' self._raise_error(response.status_code, raw_data)', ' File "/usr/local/lib/python2.7/dist-packages/elasticsearch/connection/base.py", line 125, in _raise_error', ' raise HTTP_EXCEPTIONS.get(status_code, TransportError)(status_code, error_message, additional_info)', "RequestError: TransportError(400, u'search_phase_execution_exception', u'Fielddata is disabled on text fields by default. Set fielddata=true on [hostName] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory. Alternatively use a keyword field instead.')"], 'data': {'rule': 'Metrics nodejs application, Metric aggregation Rule'}}


Here is the content of my config file for this rule nodejs.yaml

name: Metrics nodejs application, Metric aggregation Rule
type: metric_aggregation

index: nodemetrics

buffer_time:
minutes: 1

metric_agg_key: process.cpu
metric_agg_type: avg
query_key: hostName
doc_type: cpu

bucket_interval:
minutes: 1

sync_bucket_interval: true
#allow_buffer_time_overlap: true
#use_run_every_query_size: true

max_threshold: 0.8

filter:

  • term:
    _type: "cpu"

(Required)

The alert is use when a match is found

alert:

  • "debug"

I think that is because _type: "cpu" is a text field.
I don't want to activate fielddata.
Please, can you help me to find another solution ?

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions