LINUX.ORG.RU
Форум — Admin  

Centos7+Postfix+Filebeat+Elasticsearch. Парсим логи. Краткая инструкция

 , , , ,


0

1

# Руководство по загрузке логов в Elasticsearch

Вводные данные:

  • Сервера Elasticsearch:
  • 192.168.10.1:9200
  • 192.168.10.2:9200
  • Необходимый пакет: filebeat
  • Шаблон pipeline: postfix-pipeline.json

Шаблоны взяты отсюда

Установка filebeat

yum install filebeat
cat > /etc/systemd/system/filebeat@.service <<"EOF
[Unit]
Description=filebeat %I
Documentation=https://www.elastic.co/guide/en/beats/filebeat/current/index.html
Wants=network-online.target
After=network-online.target

[Service]
ExecStart=/usr/share/filebeat/bin/filebeat -c /etc/filebeat/%i.yml -path.home /usr/share/filebeat -path.config /etc/filebeat -path.data /var/lib/filebeat -path.logs /var/log/filebeat/%i.log
Restart=always

[Install]
WantedBy=multi-user.target
EOF

**Загрузка шаблона pipeline на сервера elasticsearch**

cat > /etc/filebeat/postfix-pipeline.json <<"EOF"
{
    "description": "Pipeline for parsing Postfix logs.",
    "processors": [
        {
            "grok": {
                "field": "message",
                "ignore_missing": true,
                "patterns": [
                    "^(<%{NUMBER:syslog_event_id}>)?%{SYSLOGTIMESTAMP} (%{SYSLOGHOST} )?%{DATA:syslog_program}(?:\\[%{POSINT:syslog_pid}\\])?: %{POSTFIX_ANVIL}$",
                    "^(<%{NUMBER:syslog_event_id}>)?%{SYSLOGTIMESTAMP} (%{SYSLOGHOST} )?%{DATA:syslog_program}(?:\\[%{POSINT:syslog_pid}\\])?: %{POSTFIX_BOUNCE}$",
                    "^(<%{NUMBER:syslog_event_id}>)?%{SYSLOGTIMESTAMP} (%{SYSLOGHOST} )?%{DATA:syslog_program}(?:\\[%{POSINT:syslog_pid}\\])?: %{POSTFIX_CLEANUP}$",
                    "^(<%{NUMBER:syslog_event_id}>)?%{SYSLOGTIMESTAMP} (%{SYSLOGHOST} )?%{DATA:syslog_program}(?:\\[%{POSINT:syslog_pid}\\])?: %{POSTFIX_DNSBLOG}$",
                    "^(<%{NUMBER:syslog_event_id}>)?%{SYSLOGTIMESTAMP} (%{SYSLOGHOST} )?%{DATA:syslog_program}(?:\\[%{POSINT:syslog_pid}\\])?: %{POSTFIX_ERROR}$",
                    "^(<%{NUMBER:syslog_event_id}>)?%{SYSLOGTIMESTAMP} (%{SYSLOGHOST} )?%{DATA:syslog_program}(?:\\[%{POSINT:syslog_pid}\\])?: %{POSTFIX_LOCAL}$",
                    "^(<%{NUMBER:syslog_event_id}>)?%{SYSLOGTIMESTAMP} (%{SYSLOGHOST} )?%{DATA:syslog_program}(?:\\[%{POSINT:syslog_pid}\\])?: %{POSTFIX_MASTER}$",
                    "^(<%{NUMBER:syslog_event_id}>)?%{SYSLOGTIMESTAMP} (%{SYSLOGHOST} )?%{DATA:syslog_program}(?:\\[%{POSINT:syslog_pid}\\])?: %{POSTFIX_PICKUP}$",
                    "^(<%{NUMBER:syslog_event_id}>)?%{SYSLOGTIMESTAMP} (%{SYSLOGHOST} )?%{DATA:syslog_program}(?:\\[%{POSINT:syslog_pid}\\])?: %{POSTFIX_PIPE}$",
                    "^(<%{NUMBER:syslog_event_id}>)?%{SYSLOGTIMESTAMP} (%{SYSLOGHOST} )?%{DATA:syslog_program}(?:\\[%{POSINT:syslog_pid}\\])?: %{POSTFIX_POSTDROP}$",
                    "^(<%{NUMBER:syslog_event_id}>)?%{SYSLOGTIMESTAMP} (%{SYSLOGHOST} )?%{DATA:syslog_program}(?:\\[%{POSINT:syslog_pid}\\])?: %{POSTFIX_POSTSCREEN}$",
                    "^(<%{NUMBER:syslog_event_id}>)?%{SYSLOGTIMESTAMP} (%{SYSLOGHOST} )?%{DATA:syslog_program}(?:\\[%{POSINT:syslog_pid}\\])?: %{POSTFIX_QMGR}$",
                    "^(<%{NUMBER:syslog_event_id}>)?%{SYSLOGTIMESTAMP} (%{SYSLOGHOST} )?%{DATA:syslog_program}(?:\\[%{POSINT:syslog_pid}\\])?: %{POSTFIX_SCACHE}$",
                    "^(<%{NUMBER:syslog_event_id}>)?%{SYSLOGTIMESTAMP} (%{SYSLOGHOST} )?%{DATA:syslog_program}(?:\\[%{POSINT:syslog_pid}\\])?: %{POSTFIX_SENDMAIL}$",
                    "^(<%{NUMBER:syslog_event_id}>)?%{SYSLOGTIMESTAMP} (%{SYSLOGHOST} )?%{DATA:syslog_program}(?:\\[%{POSINT:syslog_pid}\\])?: %{POSTFIX_SMTP}$",
                    "^(<%{NUMBER:syslog_event_id}>)?%{SYSLOGTIMESTAMP} (%{SYSLOGHOST} )?%{DATA:syslog_program}(?:\\[%{POSINT:syslog_pid}\\])?: %{POSTFIX_LMTP}$",
                    "^(<%{NUMBER:syslog_event_id}>)?%{SYSLOGTIMESTAMP} (%{SYSLOGHOST} )?%{DATA:syslog_program}(?:\\[%{POSINT:syslog_pid}\\])?: %{POSTFIX_SMTPD}$",
                    "^(<%{NUMBER:syslog_event_id}>)?%{SYSLOGTIMESTAMP} (%{SYSLOGHOST} )?%{DATA:syslog_program}(?:\\[%{POSINT:syslog_pid}\\])?: %{POSTFIX_POSTSUPER}$",
                    "^(<%{NUMBER:syslog_event_id}>)?%{SYSLOGTIMESTAMP} (%{SYSLOGHOST} )?%{DATA:syslog_program}(?:\\[%{POSINT:syslog_pid}\\])?: %{POSTFIX_TLSMGR}$",
                    "^(<%{NUMBER:syslog_event_id}>)?%{SYSLOGTIMESTAMP} (%{SYSLOGHOST} )?%{DATA:syslog_program}(?:\\[%{POSINT:syslog_pid}\\])?: %{POSTFIX_TLSPROXY}$",
                    "^(<%{NUMBER:syslog_event_id}>)?%{SYSLOGTIMESTAMP} (%{SYSLOGHOST} )?%{DATA:syslog_program}(?:\\[%{POSINT:syslog_pid}\\])?: %{POSTFIX_TRIVIAL_REWRITE}$",
                    "^(<%{NUMBER:syslog_event_id}>)?%{SYSLOGTIMESTAMP} (%{SYSLOGHOST} )?%{DATA:syslog_program}(?:\\[%{POSINT:syslog_pid}\\])?: %{POSTFIX_DISCARD}$",
                    "^(<%{NUMBER:syslog_event_id}>)?%{SYSLOGTIMESTAMP} (%{SYSLOGHOST} )?%{DATA:syslog_program}(?:\\[%{POSINT:syslog_pid}\\])?: %{POSTFIX_VIRTUAL}$"
                ],
                "pattern_definitions": {
                    "POSTFIX_QUEUEID": "([0-9A-F]{6,}|[0-9a-zA-Z]{15,})",
                    "POSTFIX_CLIENT_INFO": "%{HOSTNAME:postfix_client_hostname}?\\[%{IP:postfix_client_ip}\\](:%{INT:postfix_client_port})?",
                    "POSTFIX_RELAY_INFO": "%{HOSTNAME:postfix_relay_hostname}?\\[(%{IP:postfix_relay_ip}|%{DATA:postfix_relay_service})\\](:%{INT:postfix_relay_port})?|%{WORD:postfix_relay_service}",
                    "POSTFIX_SMTP_STAGE": "(CONNECT|HELO|EHLO|STARTTLS|AUTH|MAIL( FROM)?|RCPT( TO)?|(end of )?DATA|RSET|UNKNOWN|END-OF-MESSAGE|VRFY|\\.)",
                    "POSTFIX_ACTION": "(accept|defer|discard|filter|header-redirect|reject)",
                    "POSTFIX_STATUS_CODE": "\\d{3}",
                    "POSTFIX_STATUS_CODE_ENHANCED": "\\d\\.\\d\\.\\d",
                    "POSTFIX_DNSBL_MESSAGE": "Service unavailable; .* \\[%{GREEDYDATA:postfix_status_data}\\] %{GREEDYDATA:postfix_status_message};",
                    "POSTFIX_PS_ACCESS_ACTION": "(DISCONNECT|BLACKLISTED|WHITELISTED|WHITELIST VETO|PASS NEW|PASS OLD)",
                    "POSTFIX_PS_VIOLATION": "(BARE NEWLINE|COMMAND (TIME|COUNT|LENGTH) LIMIT|COMMAND PIPELINING|DNSBL|HANGUP|NON-SMTP COMMAND|PREGREET)",
                    "POSTFIX_TIME_UNIT": "%{NUMBER}[smhd]",
                    "POSTFIX_KEYVALUE_DATA": "[\\w-]+=[^;]*",
                    "POSTFIX_KEYVALUE": "%{POSTFIX_QUEUEID:postfix_queueid}: %{POSTFIX_KEYVALUE_DATA:postfix_keyvalue_data}",
                    "POSTFIX_WARNING_LEVEL": "(warning|fatal|info)",
                    "POSTFIX_TLSCONN": "(Anonymous|Trusted|Untrusted|Verified) TLS connection established (to %{POSTFIX_RELAY_INFO}|from %{POSTFIX_CLIENT_INFO}): %{DATA:postfix_tls_version} with cipher %{DATA:postfix_tls_cipher} \\(%{DATA:postfix_tls_cipher_size} bits\\)",
                    "POSTFIX_DELAYS": "%{NUMBER:postfix_delay_before_qmgr}/%{NUMBER:postfix_delay_in_qmgr}/%{NUMBER:postfix_delay_conn_setup}/%{NUMBER:postfix_delay_transmission}",
                    "POSTFIX_LOSTCONN": "(lost connection|timeout|SSL_accept error)",
                    "POSTFIX_LOSTCONN_REASONS": "(receiving the initial server greeting|sending message body|sending end of data -- message may be sent more than once)",
                    "POSTFIX_PROXY_MESSAGE": "(%{POSTFIX_STATUS_CODE:postfix_proxy_status_code} )?(%{POSTFIX_STATUS_CODE_ENHANCED:postfix_proxy_status_code_enhanced})?.*",
                    "POSTFIX_COMMAND_COUNTER_DATA": "(helo=(%{INT:postfix_cmd_helo_accepted}/)?%{INT:postfix_cmd_helo} )?(ehlo=(%{INT:postfix_cmd_ehlo_accepted}/)?%{INT:postfix_cmd_ehlo} )?(starttls=(%{INT:postfix_cmd_starttls_accepted}/)?%{INT:postfix_cmd_starttls} )?(mail=(%{INT:postfix_cmd_mail_accepted}/)?%{INT:postfix_cmd_mail} )?(rcpt=(%{INT:postfix_cmd_rcpt_accepted}/)?%{INT:postfix_cmd_rcpt} )?(data=(%{INT:postfix_cmd_data_accepted}/)?%{INT:postfix_cmd_data} )?(rset=(%{INT:postfix_cmd_rset_accepted}/)?%{INT:postfix_cmd_rset} )?(quit=(%{INT:postfix_cmd_quit_accepted}/)?%{INT:postfix_cmd_quit} )?(unknown=(%{INT:postfix_cmd_unknown_accepted}/)?%{INT:postfix_cmd_unknown} )?commands=(%{INT:postfix_cmd_count_accepted}/)?%{INT:postfix_cmd_count}",
                    "GREEDYDATA_NO_COLON": "[^:]*",
                    "GREEDYDATA_NO_SEMICOLON": "[^;]*",
                    "POSTFIX_WARNING_WITH_KV": "(%{POSTFIX_QUEUEID:postfix_queueid}: )?%{POSTFIX_WARNING_LEVEL:postfix_message_level}: %{GREEDYDATA:postfix_message}; %{POSTFIX_KEYVALUE_DATA:postfix_keyvalue_data}",
                    "POSTFIX_WARNING_WITHOUT_KV": "(%{POSTFIX_QUEUEID:postfix_queueid}: )?%{POSTFIX_WARNING_LEVEL:postfix_message_level}: %{GREEDYDATA:postfix_message}",
                    "POSTFIX_WARNING": "%{POSTFIX_WARNING_WITH_KV}|%{POSTFIX_WARNING_WITHOUT_KV}",
                    "POSTFIX_SMTPD_CONNECT": "connect from %{POSTFIX_CLIENT_INFO}",
                    "POSTFIX_SMTPD_DISCONNECT": "disconnect from %{POSTFIX_CLIENT_INFO}( %{GREEDYDATA:postfix_command_counter_data})?",
                    "POSTFIX_SMTPD_LOSTCONN": "%{POSTFIX_LOSTCONN:postfix_smtpd_lostconn_data}( after %{POSTFIX_SMTP_STAGE:postfix_smtp_stage}( \\(%{INT} bytes\\))?)? from %{POSTFIX_CLIENT_INFO}(: %{GREEDYDATA:postfix_smtpd_lostconn_reason})?",
                    "POSTFIX_SMTPD_NOQUEUE": "NOQUEUE: %{POSTFIX_ACTION:postfix_action}: %{POSTFIX_SMTP_STAGE:postfix_smtp_stage} from %{POSTFIX_CLIENT_INFO}:( %{POSTFIX_STATUS_CODE:postfix_status_code} %{POSTFIX_STATUS_CODE_ENHANCED:postfix_status_code_enhanced})?( <%{DATA:postfix_status_data}>:)? (%{POSTFIX_DNSBL_MESSAGE}|%{GREEDYDATA:postfix_status_message};) %{POSTFIX_KEYVALUE_DATA:postfix_keyvalue_data}",
                    "POSTFIX_SMTPD_PIPELINING": "improper command pipelining after %{POSTFIX_SMTP_STAGE:postfix_smtp_stage} from %{POSTFIX_CLIENT_INFO}: %{GREEDYDATA:postfix_improper_pipelining_data}",
                    "POSTFIX_SMTPD_PROXY": "proxy-%{POSTFIX_ACTION:postfix_proxy_result}: (%{POSTFIX_SMTP_STAGE:postfix_proxy_smtp_stage}): %{POSTFIX_PROXY_MESSAGE:postfix_proxy_message}; %{POSTFIX_KEYVALUE_DATA:postfix_keyvalue_data}",
                    "POSTFIX_CLEANUP_MILTER": "%{POSTFIX_QUEUEID:postfix_queueid}: milter-%{POSTFIX_ACTION:postfix_milter_result}: %{GREEDYDATA:postfix_milter_message}; %{GREEDYDATA_NO_COLON:postfix_keyvalue_data}(: %{GREEDYDATA:postfix_milter_data})?",
                    "POSTFIX_QMGR_REMOVED": "%{POSTFIX_QUEUEID:postfix_queueid}: removed",
                    "POSTFIX_QMGR_ACTIVE": "%{POSTFIX_QUEUEID:postfix_queueid}: %{POSTFIX_KEYVALUE_DATA:postfix_keyvalue_data} \\(queue active\\)",
                    "POSTFIX_QMGR_EXPIRED": "%{POSTFIX_QUEUEID:postfix_queueid}: from=<%{DATA:postfix_from}>, status=%{WORD:postfix_status}, returned to sender",
                    "POSTFIX_PIPE_ANY": "%{POSTFIX_QUEUEID:postfix_queueid}: %{POSTFIX_KEYVALUE_DATA:postfix_keyvalue_data}, status=%{WORD:postfix_status} \\(%{GREEDYDATA:postfix_pipe_response}\\)",
                    "POSTFIX_ERROR_ANY": "%{POSTFIX_QUEUEID:postfix_queueid}: %{POSTFIX_KEYVALUE_DATA:postfix_keyvalue_data}, status=%{WORD:postfix_status} \\(%{GREEDYDATA:postfix_error_response}\\)",
                    "POSTFIX_DISCARD_ANY": "%{POSTFIX_QUEUEID:postfix_queueid}: %{POSTFIX_KEYVALUE_DATA:postfix_keyvalue_data} status=%{WORD:postfix_status} %{GREEDYDATA}",
                    "POSTFIX_POSTSUPER_ACTIONS": "(removed|requeued|placed on hold|released from hold)",
                    "POSTFIX_POSTSUPER_ACTION": "%{POSTFIX_QUEUEID:postfix_queueid}: %{POSTFIX_POSTSUPER_ACTIONS:postfix_postsuper_action}",
                    "POSTFIX_POSTSUPER_SUMMARY_ACTIONS": "(Deleted|Requeued|Placed on hold|Released from hold)",
                    "POSTFIX_POSTSUPER_SUMMARY": "%{POSTFIX_POSTSUPER_SUMMARY_ACTIONS:postfix_postsuper_summary_action}: %{NUMBER:postfix_postsuper_summary_count} messages?",
                    "POSTFIX_PS_CONNECT": "CONNECT from %{POSTFIX_CLIENT_INFO} to \\[%{IP:postfix_server_ip}\\]:%{INT:postfix_server_port}",
                    "POSTFIX_PS_ACCESS": "%{POSTFIX_PS_ACCESS_ACTION:postfix_postscreen_access} %{POSTFIX_CLIENT_INFO}",
                    "POSTFIX_PS_NOQUEUE": "%{POSTFIX_SMTPD_NOQUEUE}",
                    "POSTFIX_PS_TOOBUSY": "NOQUEUE: reject: CONNECT from %{POSTFIX_CLIENT_INFO}: %{GREEDYDATA:postfix_postscreen_toobusy_data}",
                    "POSTFIX_PS_DNSBL": "%{POSTFIX_PS_VIOLATION:postfix_postscreen_violation} rank %{INT:postfix_postscreen_dnsbl_rank} for %{POSTFIX_CLIENT_INFO}",
                    "POSTFIX_PS_CACHE": "cache %{DATA} full cleanup: retained=%{NUMBER:postfix_postscreen_cache_retained} dropped=%{NUMBER:postfix_postscreen_cache_dropped} entries",
                    "POSTFIX_PS_VIOLATIONS": "%{POSTFIX_PS_VIOLATION:postfix_postscreen_violation}( %{INT})?( after %{NUMBER:postfix_postscreen_violation_time})? from %{POSTFIX_CLIENT_INFO}(( after %{POSTFIX_SMTP_STAGE:postfix_smtp_stage})?(: %{GREEDYDATA:postfix_postscreen_data})?| in tests (after|before) SMTP handshake)",
                    "POSTFIX_DNSBLOG_LISTING": "addr %{IP:postfix_client_ip} listed by domain %{HOSTNAME:postfix_dnsbl_domain} as %{IP:postfix_dnsbl_result}",
                    "POSTFIX_TLSPROXY_CONN": "(DIS)?CONNECT( from)? %{POSTFIX_CLIENT_INFO}",
                    "POSTFIX_ANVIL_CONN_RATE": "statistics: max connection rate %{NUMBER:postfix_anvil_conn_rate}/%{POSTFIX_TIME_UNIT:postfix_anvil_conn_period} for \\(%{DATA:postfix_service}:%{IP:postfix_client_ip}\\) at %{SYSLOGTIMESTAMP:postfix_anvil_timestamp}",
                    "POSTFIX_ANVIL_CONN_CACHE": "statistics: max cache size %{NUMBER:postfix_anvil_cache_size} at %{SYSLOGTIMESTAMP:postfix_anvil_timestamp}",
                    "POSTFIX_ANVIL_CONN_COUNT": "statistics: max connection count %{NUMBER:postfix_anvil_conn_count} for \\(%{DATA:postfix_service}:%{IP:postfix_client_ip}\\) at %{SYSLOGTIMESTAMP:postfix_anvil_timestamp}",
                    "POSTFIX_SMTP_DELIVERY": "%{POSTFIX_KEYVALUE} status=%{WORD:postfix_status}( \\(%{GREEDYDATA:postfix_smtp_response}\\))?",
                    "POSTFIX_SMTP_CONNERR": "connect to %{POSTFIX_RELAY_INFO}: (Connection timed out|No route to host|Connection refused|Network is unreachable)",
                    "POSTFIX_SMTP_LOSTCONN": "%{POSTFIX_QUEUEID:postfix_queueid}: %{POSTFIX_LOSTCONN:postfix_smtp_lostconn_data} with %{POSTFIX_RELAY_INFO}( while %{POSTFIX_LOSTCONN_REASONS:postfix_smtp_lostconn_reason})?",
                    "POSTFIX_SMTP_TIMEOUT": "%{POSTFIX_QUEUEID:postfix_queueid}: conversation with %{POSTFIX_RELAY_INFO} timed out( while %{POSTFIX_LOSTCONN_REASONS:postfix_smtp_lostconn_reason})?",
                    "POSTFIX_SMTP_RELAYERR": "%{POSTFIX_QUEUEID:postfix_queueid}: host %{POSTFIX_RELAY_INFO} said: %{GREEDYDATA:postfix_smtp_response} \\(in reply to %{POSTFIX_SMTP_STAGE:postfix_smtp_stage} command\\)",
                    "POSTFIX_MASTER_START": "(daemon started|reload) -- version %{DATA:postfix_version}, configuration %{PATH:postfix_config_path}",
                    "POSTFIX_MASTER_EXIT": "terminating on signal %{INT:postfix_termination_signal}",
                    "POSTFIX_BOUNCE_NOTIFICATION": "%{POSTFIX_QUEUEID:postfix_queueid}: sender (non-delivery|delivery status|delay) notification: %{POSTFIX_QUEUEID:postfix_bounce_queueid}",
                    "POSTFIX_SCACHE_LOOKUPS": "statistics: (address|domain) lookup hits=%{INT:postfix_scache_hits} miss=%{INT:postfix_scache_miss} success=%{INT:postfix_scache_success}%",
                    "POSTFIX_SCACHE_SIMULTANEOUS": "statistics: max simultaneous domains=%{INT:postfix_scache_domains} addresses=%{INT:postfix_scache_addresses} connection=%{INT:postfix_scache_connection}",
                    "POSTFIX_SCACHE_TIMESTAMP": "statistics: start interval %{SYSLOGTIMESTAMP:postfix_scache_timestamp}",
                    "POSTFIX_SMTPD": "%{POSTFIX_SMTPD_CONNECT}|%{POSTFIX_SMTPD_DISCONNECT}|%{POSTFIX_SMTPD_LOSTCONN}|%{POSTFIX_SMTPD_NOQUEUE}|%{POSTFIX_SMTPD_PIPELINING}|%{POSTFIX_TLSCONN}|%{POSTFIX_WARNING}|%{POSTFIX_SMTPD_PROXY}|%{POSTFIX_KEYVALUE}",
                    "POSTFIX_CLEANUP": "%{POSTFIX_CLEANUP_MILTER}|%{POSTFIX_WARNING}|%{POSTFIX_KEYVALUE}",
                    "POSTFIX_QMGR": "%{POSTFIX_QMGR_REMOVED}|%{POSTFIX_QMGR_ACTIVE}|%{POSTFIX_QMGR_EXPIRED}|%{POSTFIX_WARNING}",
                    "POSTFIX_PIPE": "%{POSTFIX_PIPE_ANY}",
                    "POSTFIX_POSTSCREEN": "%{POSTFIX_PS_CONNECT}|%{POSTFIX_PS_ACCESS}|%{POSTFIX_PS_NOQUEUE}|%{POSTFIX_PS_TOOBUSY}|%{POSTFIX_PS_CACHE}|%{POSTFIX_PS_DNSBL}|%{POSTFIX_PS_VIOLATIONS}|%{POSTFIX_WARNING}",
                    "POSTFIX_DNSBLOG": "%{POSTFIX_DNSBLOG_LISTING}|%{POSTFIX_WARNING}",
                    "POSTFIX_ANVIL": "%{POSTFIX_ANVIL_CONN_RATE}|%{POSTFIX_ANVIL_CONN_CACHE}|%{POSTFIX_ANVIL_CONN_COUNT}",
                    "POSTFIX_SMTP": "%{POSTFIX_SMTP_DELIVERY}|%{POSTFIX_SMTP_CONNERR}|%{POSTFIX_SMTP_LOSTCONN}|%{POSTFIX_SMTP_TIMEOUT}|%{POSTFIX_SMTP_RELAYERR}|%{POSTFIX_TLSCONN}|%{POSTFIX_WARNING}",
                    "POSTFIX_DISCARD": "%{POSTFIX_DISCARD_ANY}|%{POSTFIX_WARNING}",
                    "POSTFIX_LMTP": "%{POSTFIX_SMTP}",
                    "POSTFIX_PICKUP": "%{POSTFIX_KEYVALUE}",
                    "POSTFIX_TLSPROXY": "%{POSTFIX_TLSPROXY_CONN}|%{POSTFIX_WARNING}",
                    "POSTFIX_MASTER": "%{POSTFIX_MASTER_START}|%{POSTFIX_MASTER_EXIT}|%{POSTFIX_WARNING}",
                    "POSTFIX_BOUNCE": "%{POSTFIX_BOUNCE_NOTIFICATION}",
                    "POSTFIX_SENDMAIL": "%{POSTFIX_WARNING}",
                    "POSTFIX_POSTDROP": "%{POSTFIX_WARNING}",
                    "POSTFIX_SCACHE": "%{POSTFIX_SCACHE_LOOKUPS}|%{POSTFIX_SCACHE_SIMULTANEOUS}|%{POSTFIX_SCACHE_TIMESTAMP}",
                    "POSTFIX_TRIVIAL_REWRITE": "%{POSTFIX_WARNING}",
                    "POSTFIX_TLSMGR": "%{POSTFIX_WARNING}",
                    "POSTFIX_LOCAL": "%{POSTFIX_KEYVALUE}",
                    "POSTFIX_VIRTUAL": "%{POSTFIX_SMTP_DELIVERY}",
                    "POSTFIX_ERROR": "%{POSTFIX_ERROR_ANY}",
                    "POSTFIX_POSTSUPER": "%{POSTFIX_POSTSUPER_ACTION}|%{POSTFIX_POSTSUPER_SUMMARY}"
                }
            }
        },
        {
            "remove": {
                "field": "message"
            }
        },
        {
            "date": {
                "field": "postfix.log.timestamp",
                "target_field": "@timestamp",
                "formats": [
                    "yyyy-MM-dd HH:mm:ss.SSS"
                ],
                "ignore_failure": true
            }
        }
    ],
    "on_failure": [
        {
            "set": {
                "field": "error.message",
                "value": "{{ _ingest.on_failure_message }}"
            }
        }
    ]
}
EOF

curl -H 'Content-Type: application/json' -XPUT 'http://192.168.10.1:9200/_ingest/pipeline/postfix-pipeline' -d @/etc/filebeat/postfix-pipeline.json

curl -H 'Content-Type: application/json' -XPUT 'http://192.168.10.2:9200/_ingest/pipeline/postfix-pipeline' -d @/etc/filebeat/postfix-pipeline.json

Настройка службы filebeat@postfix.service

cat > /etc/filebeat/postfix.yml<<"EOF"
filebeat.prospectors:
- input_type: log
  paths: [ "/var/log/maillog" ]
  exclude_lines: ['lost connection after EHLO from postfix-', 'disconnect from postfix-']
  close.removed: true
  close.renamed: true
output.elasticsearch:
  hosts: ["192.168.10.1:9200", "192.168.10.2:9200"]
  protocols: http
  pipeline: "postfix-pipeline"
  index: "postfix-"
EOF

systemctl enable filebeat@postfix.service

systemctl start filebeat@postfix.service

Настройка службы filebeat@syslog.service

cat >/etc/filebeat/syslog.yml<<"EOF"
filebeat.prospectors:
- input_type: log
  close_renamed: true
  close_removed: true
  paths: [ "/var/log/boot.log" ]
  fields:
    filetype: bootlog
- input_type: log
  close_renamed: true
  close_removed: true
  paths: [ "/var/log/cron" ]
  fields:
    filetype: cron
- input_type: log
  close_renamed: true
  close_removed: true
  paths: [ "/var/log/dmesg" ]
  fields:
    filetype: dmesg
- input_type: log
  close_renamed: true
  close_removed: true
  paths: [ "/var/log/dovecot.log" ]
  fields:
    filetype: dovecot
- input_type: log
  close_renamed: true
  close_removed: true
  paths: [ "/var/log/fail2ban" ]
  fields:
    filetype: fail2ban
- input_type: log
  close_renamed: true
  close_removed: true
  paths: [ "/var/log/messages" ]
  fields:
    filetype: syslog
- input_type: log
  close_renamed: true
  close_removed: true
  paths: [ "/var/log/secure" ]
  fields:
    filetype: secure
- input_type: log
  close_renamed: true
  close_removed: true
  paths: [ "/var/log/spooler" ]
  fields:
    filetype: spooler
- input_type: log
  close_renamed: true
  close_removed: true
  paths: [ "/var/log/vault.log" ]
  fields:
    filetype: vault
- input_type: log
  close_renamed: true
  close_removed: true
  paths: [ "/var/log/yum.log" ]
  fields:
    filetype: yum
- input_type: log
  close_renamed: true
  close_removed: true
  paths: [ "/var/log/zabbix/zabbix_agentd.log" ]
  fields:
    filetype: zabbix
output.elasticsearch:
  hosts: ["192.168.10.1:9200", "192.168.10.2:9200"]
  protocols: http
EOF

systemctl enable filebeat@syslog.service

systemctl start filebeat@syslog.service
★★★

Последнее исправление: Nurmukh (всего исправлений: 2)
← matrix-synapse без «federation»
Миграция с HDD на SSD →
Ответ на: комментарий от CHIPOK

Перед ТС стояла задача запостить сюда простыню с целью набить скор и он с ней справился.

Siado ★★★★★
()
Ответ на: комментарий от CHIPOK

Ну я столкнулся с проблемой, что нигде нет инструкции как прикрутить постфикс к еластиксерчу. ну вот я решил, что я должен это сделать.

потому что:

Не спрашивай, что ЛОР может сделать для тебя, спроси, что ты можешь сделать для ЛОРа (с) Джон Фитцжеральд Кеннеди.

Nurmukh ★★★
() автор топика
Ответ на: комментарий от Nurmukh

Лучше бы запилил цикл статей по настройке телефонии на asterisk, cisco и freeswitch. Чтобы было от А до Я.

anonymous
()
Ответ на: комментарий от anonymous

Сейчас не могу этого сделать. Направление деятельности поменялось. не до астериска. До НГ так будет.

Nurmukh ★★★
() автор топика
Вы не можете добавлять комментарии в эту тему. Тема перемещена в архив.
← matrix-synapse без «federation»
Admin
Миграция с HDD на SSD →