DNS logstash filter is slow

Issue description

 

I've used the DNS filter on the logstash, but i can clearly see that the indexing speed has decreased by adding resolve.
Does it have to be so slow?

Logstash config from documentation:

filter {
  dns {
    reverse => [ "source_host", "field_with_address" ]
    resolve => [ "field_with_fqdn" ]
    action => "replace"
  }
}

Issue solution

 

In older versions of logstash (2018 *), after using the cache_size / failed_cache_size directive, there was a bug that prevented parallel cache polling.

A very nice analysis with performance graphs was carried out by the git user named robcowart:
https://github.com/logstash-plugins/logstash-filter-dns/pull/42

A ready config to use below - please note that full performance is obtained when the cache is full with data.
It's also worth using fast dns, e.g. 1.1.1.1/1.0.0.1

filter{
  # dns resolve
  dns {
    reverse => [ "hostname" ]
    action => "replace"
    nameserver => ["1.1.1.1", "1.0.0.1"]
    hit_cache_size => 131072
    hit_cache_ttl => 900
    failed_cache_size => 131072
    failed_cache_ttl => 900
  }

  # filter performance
  metrics {
    meter => "events"
    add_tag => "metric"
  }
}

output {
  if "metric" in [tags] {
    stdout {
      codec => line {
        format => "DNS filter rate: %{[events][rate_1m]}"
      }
    }
  }
}