新钛云服-云和安全管理服务专家-助力企业数字化转型-实战

实战 | ELK6.8.12环境部署

新钛云服 2020/10/15

简介信息

ELK是一款开源的海量日志搜索分析平台，能够完美的解决我们上述的问题，对日志进行集中采集和实时索引，提供实时搜索、分析、可视化、报警等功能，帮助企业在统一平台实时管理日志数据，进行线上业务实时监控、异常原因定位、数据统计分析。

Elasticsearch是实时全文搜索和分析引擎，提供搜集、分析、存储数据三大功能；是一套开放REST和JAVA API等结构提供高效搜索功能，可扩展的分布式系统；它构建于Apache Lucene搜索引擎库之上
Logstash是一个用来搜集、分析、过滤日志的工具。它支持几乎任何类型的日志，包括系统日志、错误日志和自定义应用程序日志。
Kibana是一个基于Web的图形界面，用于搜索、分析和可视化存储在 Elasticsearch指标中的日志数据。
Filebeat是一个轻量型日志采集器，用于转发和汇总日志与文件。

基础环境

服务器基础信息

服务器信息	系统版本	组件信息
10.20.21.21 (Master)	Centos7.6	Elasticsearch、Logstash、Kibana、Redis
10.20.21.46 (Slave)	Centos7.6	Elasticsearch、Logstash、Kibana、Redis
10.20.21.33	Windows2012	Filebeat

Elasticsearch配置文件路径/usr/local/src/tyuninstall/elasticsearch/config/elasticsearch.yml
Logstash的配置文件路径信息/usr/local/src/tyuninstall/logstash/conf.d/和/usr/local/src/tyuninstall/logstash/config/logstash.yml
Kibana配置文件路径/usr/local/src/tyuninstall/kibana/config/kibana.yml
Redis 配置文件路径信息/etc/redis/6379.conf

系统常用初始化

yum环境包安装mv /etc/yum.repos.d/CentOS-Base.repo /etc/yum.repos.d/CentOS-Base.repo.backup mv /etc/yum.repos.d/epel.repo /etc/yum.repos.d/epel.repo.bak wget -O /etc/yum.repos.d/CentOS-Base.repo https://mirrors.aliyun.com/repo/Centos-7.repo wget -O /etc/yum.repos.d/epel.repo https://mirrors.aliyun.com/repo/epel-7.repo yum install -y net-tools vim lrzsz tree screen lsof wget chrony telnet gcc gcc-c++ tcl iptables-services nmap sysstat dos2unix bash-completion.noarch curl ruby ruby-devel rubygems rpm-build make jemalloc-devel automake autoconf libtool tcl binutils glibc glibc-kernheaders glibc-common glibc-devel
防火墙systemctl stop firewalld systemctl disable firewalld iptables -L iptables -F iptables -X vim /etc/sysconfig/iptables -A INPUT -p tcp -m state --state NEW -m tcp --dport 22 -j ACCEPT -A INPUT -p tcp -m state --state NEW -m tcp --dport 5601 -j ACCEPT -A INPUT -p tcp -m state --state NEW -m tcp --dport 6379 -j ACCEPT -A INPUT -p tcp -m state --state NEW -m tcp --dport 9200 -j ACCEPT -A INPUT -p tcp -m state --state NEW -m tcp --dport 9300 -j ACCEPT systemctl restart iptables.service systemctl enable iptables.service
Selinux setenforce 0 临时生效selinux sed -i 's/^SELINUX=.*$/SELINUX=disabled/' /etc/selinux/config
主机名称hostnamectl set-hostname 10-20-21-21 bash
zabbix监控rpm -Uvh https://repo.zabbix.com/zabbix/4.0/rhel/7/x86_64/zabbix-release-4.0-2.el7.noarch.rpm vim /etc/yum.repos.d/zabbix.repo [zabbix] name=Zabbix Official Repository - $basearch baseurl=https://mirrors.aliyun.com/zabbix/zabbix/4.0/rhel/7/$basearch/ enabled=1 gpgcheck=1 gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-ZABBIX-A14FE591 [zabbix-non-supported] name=Zabbix Official Repository non-supported - $basearch baseurl=https://mirrors.aliyun.com/zabbix/non-supported/rhel/7/$basearch/ enabled=1 gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-ZABBIX gpgcheck=1 yum install -y zabbix-agent service zabbix-agent restart systemctl enable zabbix-agent
环境变量JAVA下载 https://www.oracle.com/java/technologies/javase/javase-jdk8-downloads.html#license-lightbox tar zxf jdk-8u261-linux-x64.tar.gz -C /usr/local/src/tyuninstall/ cd /usr/local/src/tyuninstall/ mv jdk1.8.0_261/ jdkvim /etc/profile export PATH USER LOGNAME MAIL HOSTNAME HISTSIZE HISTCONTROL #第53行下面添加 export JAVA_HOME=/usr/local/src/tyuninstall/jdk export CLASSPATH=.:$JAVA_HOME/jre/lib/rt.jar:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar export PATH=$PATH:$JAVA_HOME/bin source /etc/profile java -version 版本查看 java version "1.8.0_261"
时间同步systemctl start chronyd && systemctl enable chronyd timedatectl set-timezone Asia/Shanghai && timedatectl set-ntp yes
hosts文件修改echo "10.20.21.21 10-20-21-21">> /etc/hosts echo "10.20.21.46 10-20-21-46">> /etc/hosts

系统优化

内核配置vim /etc/sysctl.conf net.ipv6.conf.all.disable_ipv6 = 1 net.ipv6.conf.default.disable_ipv6 = 1 vm.max_map_count = 655360 net.core.somaxconn=65535 net.ipv4.ip_forward = 1 fs.file-max=1645037 vm.swappiness=1 sysctl -p
文件限制vim /etc/security/limits.conf * soft nofile 65536 * hard nofile 65536 * soft nproc 65536 * hard nproc 65536 * soft memlock unlimited * hard memlock unlimited egrep -v "^$|#" /etc/security/limits.conf ulimit -SHn 65536 ulimit -avim /etc/security/limits.d/20-nproc.conf * soft nproc 4096

环境包下载

软件目录与软件官网地址软件下载目录`/usr/local/src/tiopspage/` 软件安装目录 `/usr/local/src/tyuninstall/` mkdir -p /usr/local/src/{tiopspage,tyuninstall}

官方下载地址 https://www.elastic.co/cn/downloads/

Elasticsearch 6.8.12 https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-6.8.12.tar.gz

Logstash6.8.12 https://artifacts.elastic.co/downloads/logstash/logstash-6.8.12.tar.gz

Kibana6.8.12 https://artifacts.elastic.co/downloads/kibana/kibana-6.8.12-linux-x86_64.tar.gz

Filebeat6.8.12 https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-6.8.12-windows-x86_64.zip

Redis5.0.9 https://download.redis.io/releases/redis-5.0.9.tar.gz

应用部署搭建

Redistar -zxvf /usr/local/software/redis-5.0.9.tar.gz -C /usr/local/src/tyuninstall/ cd /usr/local/src/tyuninstall/ mv redis-5.0.9/ redis cd /usr/local/src/tyuninstall/redis make && make install PREFIX=/etc/redis cp /usr/local/src/tyuninstall/redis/redis.conf /etc/redis/6379.confvim /etc/redis/6379.conf bind 127.0.0.1 10.20.21.21 protected-mode no daemonize yes pidfile /var/run/redis_6379.pid logfile /var/log/redis_6379.log save "" dir /etc/redis/ requirepass TYUN2020 maxmemory 16Gvim /etc/init.d/redis # chkconfig: 2345 10 90 # description: Start and Stop redis PATH=/usr/local/bin:/sbin:/usr/bin:/bin REDISPORT=6379 EXEC=/etc/redis/bin/redis-server REDIS_CLI=/etc/redis/bin/redis-cli PIDFILE=/var/run/redis_6379.pid CONF="/etc/redis/6379.conf" case "$1" in start) if [ -f $PIDFILE ] then echo "$PIDFILE exists, process is already running or crashed." else echo "Starting Redis server..." $EXEC $CONF fi if [ "$?"="0" ] then echo "Redis is running..." fi ;; stop) if [ ! -f $PIDFILE ] then echo "$PIDFILE exists, process is not running." else PID=$(cat $PIDFILE) echo "Stopping..." $REDIS_CLI -p $REDISPORT -a TYUN2020 SHUTDOWN while [ -x $PIDFILE ] do echo "Waiting for Redis to shutdown..." sleep 1 done echo "Redis stopped" fi ;; restart|force-reload) ${0} stop ${0} start ;; *) echo "Usage: /etc/init.d/redis {start|stop|restart|force-reload}" >&2 exit 1 esacchmod 755 /etc/init.d/redis #设置文件redis的权限 chkconfig redis on #开启服务自启动 chkconfig --list #查看所有注册的脚本文件 service redis start #启动redis service redis stop #关闭redis ps aux|grep redis #查看redis进程vim /etc/profile export PATH=$PATH:$JAVA_HOME/bin:/etc/redis/bin source /etc/profileredis连接测试 redis-cli -h 10.20.21.21 -a TYUN2020
Elasticsearchcd /usr/local/src/tiopspage/ tar -xf elasticsearch-6.8.12.tar.gz -C ./../tyuninstall/ cd ./../tyuninstall/ mv elasticsearch-6.8.12 elasticsearch mkdir -p /tyun/elastic/{data,logs} useradd elk chown -Rf elk:elk /tyun/elastic/ chown -Rf elk:elk /usr/local/src/tyuninstall/elasticsearch vim /usr/local/src/tyuninstall/elasticsearch/config/jvm.options -Xms16g -Xmx16g Master机器 vim /usr/local/src/tyuninstall/elasticsearch/config/elasticsearch.yml cluster.name: TYUN88 #集群名称 node.name: 10-20-21-21 #节点名称 path.data: /tyun/elastic/data #索引数据存储位置 path.logs: /tyun/elastic/logs #日志路径 network.host: 10.20.21.21 #当前es节点绑定的ip地址 http.port: 9200 #启动的es对外访问的http端口，默认9200 transport.tcp.compress: true #设置是否压缩tcp传输时的数据 transport.tcp.port: 9300 #设置节点之间交互的tcp端口 node.master: true #指定该节点是否有资格被选举成为node node.data: true #指定该节点是否存储索引数据 discovery.zen.ping.unicast.hosts: ["10.20.21.21","10.20.21.46"] #设置集群中master节点的初始列表，可以通过这些节点来自动发现新加入集群的节点。 discovery.zen.minimum_master_nodes: 1 #设置这个参数来保证集群中的节点可以知道其它N个有master资格的节点。 http.cors.enabled: true #允许集群外的http请求 http.cors.allow-origin: "*" # *号表示接收任意ip的http请求 http.cors.allow-headers: Authorization,X-Requested-With,Content-Length,Content-Type #跨域允许设置的头信息 xpack.security.enabled: true #配置表示开启xpack认证机制 xpack.security.transport.ssl.enabled: true Slave机器 vim /usr/local/src/tyuninstall/elasticsearch/config/elasticsearch.yml cluster.name: TYUN88 node.name: 10-20-21-46 path.data: /tyun/elastic/data path.logs: /tyun/elastic/logs bootstrap.memory_lock: true network.host: 10.20.21.46 http.port: 9200 transport.tcp.compress: true transport.tcp.port: 9300 node.master: false node.data: true discovery.zen.ping.unicast.hosts: ["10.20.21.21","10.20.21.46"] discovery.zen.minimum_master_nodes: 1 http.cors.enabled: true http.cors.allow-origin: "*" http.cors.allow-headers: Authorization,X-Requested-With,Content-Length,Content-Type xpack.security.enabled: true xpack.security.transport.ssl.enabled: trueElasticsearch启动脚本切换到elk用户 vim /usr/local/src/tyuninstall/elasticsearch/TyunES.sh #!/bin/bash #extract ES pid id=`ps -elf | grep -i elasticsearch | grep 'Xmx' | awk '{print $4}'` if [ $# -eq 1 ];then case $1 in start) if [ -z "$id" ];then echo -e '\033[32mes starting....\033[0m' nohup su - elk -c "/usr/local/src/tyuninstall/elasticsearch/bin/elasticsearch" & > /dev/null sleep 5 echo -e '\033[32mes started successfully\033[0m' else echo -e "\033[31mes Already started!,pid:${id}\033[0m" fi ;; restart) if [ -z "$id" ];then echo -e '\033[32mes restart....\033[0m' nohup su - elk -c "/usr/local/src/tyuninstall/elasticsearch/bin/elasticsearch" & > /dev/null sleep 5 echo -e '\033[32mes restarted successfully\033[0m' else echo -e '\033[32mes restart....\033[0m' kill -9 $id &> /dev/null nohup su - elk -c "/usr/local/src/tyuninstall/elasticsearch/bin/elasticsearch" & > /dev/null sleep 5 echo -e '\033[32mes restarted successfully\033[0m' fi ;; status) if [ -z "$id" ];then echo -e '\033[32mes Not started!\033[0m' else echo -e "\033[32mes running,pid:${id}\033[0m" fi ;; stop) if [ -z "$id" ];then echo -e '\033[31mes Not started!\033[0m' else echo -e '\033[32mes Stopping....\033[0m' kill -9 $id &> /dev/null sleep 2 echo -e '\033[32mes stopped successfully\033[0m' fi ;; *) echo -e '\033[31mstart|restart|status|stop\033[0m' esac else echo -e '\033[31mstart|restart|status|stop\033[0m' fichmod +x /usr/local/src/tyuninstall/elasticsearch/TyunES.sh /usr/local/src/tyuninstall/elasticsearch/TyunES.sh start #启动Elasticsearch服务 ss -tunl | grep -E '9200|9300' #查看端口Elasticsearch 账户密码登录切换到elk用户 /usr/local/src/tyuninstall/elasticsearch/bin/elasticsearch-setup-passwords auto Please confirm that you would like to continue [y/N]y Changed password for user apm_system PASSWORD apm_system = lYb3KJTp7h8AdDWWhf2g Changed password for user kibana PASSWORD kibana = awkVszgqiyvfvHEynWHR Changed password for user logstash_system PASSWORD logstash_system = kSypksQqLku0xzqojdTH Changed password for user beats_system PASSWORD beats_system = cj7QYJV8Dhl97jjJjHV9 Changed password for user remote_monitoring_user PASSWORD remote_monitoring_user = vM3ikyjluHUFQGEN7Eeo Changed password for user elastic PASSWORD elastic = 5tVF53HD4TyqeLlflR0q `浏览器输入地址` https://10.20.21.21:9200/ `或者是谷歌浏览器安装插件` ElasticSearch Head
Logstashcd /usr/local/src/tiopspage/ tar -xf logstash-6.8.12.tar.gz -C ./../tyuninstall/ #解压文件 cd ./../tyuninstall/ mv logstash-6.8.12 logstashvim /usr/local/src/tyuninstall/logstash/config/logstash.yml path.data: /tyun/logstash/data #数据存储路径 path.logs: /tyun/logstash/logs #日志输出路径 path.config: /usr/local/src/tyuninstall/logstash/conf.d/*.conf #配置文件目录mkdir -p /usr/local/src/tyuninstall/logstash/{conf.d,geoip} mkdir -p /tyun/logstash/{data,logs} https://www.maxmind.com/en/account/login 注册一个账户便于下载地图文件 https://dev.maxmind.com/geoip/geoip2/geolite2/ `网站下载`GeoLite2-City.mmdb `文件` 文件指引下载：输入地址使用账户密码登录后进入页面 Download Files模块找到GeoLite2 City 下载文件 GeoLite2-City_20200811.tar.gz 把文件放到/usr/local/src/tyuninstall/logstash/geoip/下面解压 cd /usr/local/src/tyuninstall/logstash/geoip tar -xf GeoLite2-City_20200811.tar.gz cp GeoLite2-City_20200811/GeoLite2-City.mmdb . rm -rf GeoLite2-City_20200811 GeoLite2-City_20200811.tar.gz ll /usr/local/src/tyuninstall/logstash/geoip/GeoLite2-City.mmdbLogstash启动脚本 vim /usr/local/src/tyuninstall/logstash/logstash.sh #!/bin/bash #extractlogstash pid id=`ps -elf | grep -i logstash | grep 'Xms' | awk '{print $4}'` if [ $# -eq 1 ];then case $1 in start) if [ -z "$id" ];then echo -e '\033[32mlogstash starting....\033[0m' nohup sh /usr/local/src/tyuninstall/logstash/bin/logstash & > /dev/null sleep 5 echo -e '\033[32mlogstash started successfully\033[0m' else echo -e "\033[31mlogstash Already started!,pid:${id}\033[0m" fi ;; restart) if [ -z "$id" ];then echo -e '\033[32mlogstash restart.....\033[0m' nohup sh /usr/local/src/tyuninstall/logstash/bin/logstash & > /dev/null sleep 5 echo -e '\033[32mlogstash restarted successfully\033[0m' else echo -e '\033[32mlogstash restart.....\033[0m' kill -9 $id &> /dev/null nohup sh /usr/local/src/tyuninstall/logstash/bin/logstash & > /dev/null sleep 5 echo -e '\033[32mlogstash restarted successfully\033[0m' fi ;; status) if [ -z "$id" ];then echo -e '\033[32mlogstash Not started!\033[0m' else echo -e "\033[32mlogstash running,pid:${id}\033[0m" fi ;; stop) if [ -z "$id" ];then echo -e '\033[31mlogstash Not started!\033[0m' else echo -e '\033[32mlogstash Stopping....\033[0m' kill -9 $id &> /dev/null sleep 2 echo -e '\033[32mlogstash stopped successfully\033[0m' fi ;; *) echo -e '\033[31mstart|restart|status|stop\033[0m' esac else echo -e '\033[31mstart|restart|status|stop\033[0m' fichmod +x /usr/local/src/tyuninstall/logstash/logstash.sh /usr/local/src/tyuninstall/logstash/logstash.sh start #启动Logstash服务如果发现启动失败的话是因为 No config files found in path {:path=>"/usr/local/src/tyuninstall/logstash/conf.d/*.conf"} 下面还没有文件（把iis.conf放conf.d下面就可以启动成功） ss -tunl | grep :9600
Kibanacd /usr/local/src/tiopspage/ tar -xf kibana-6.8.12-linux-x86_64.tar.gz -C ./../tyuninstall/ cd ./../tyuninstall/ mv kibana-6.8.12-linux-x86_64 kibanavim /usr/local/src/tyuninstall/kibana/config/kibana.yml elasticsearch.requestTimeout: 60000 #Elasticsearch后端响应请求的超时时间 elasticsearch.shardTimeout: 50000 #Elasticsearch等待分片响应的超时时间 server.port: 5601 #kibana服务端口 server.host: "10.20.21.21" #kibana服务地址 server.name: "Kibana" #kibana服务名称 elasticsearch.hosts: ["https://10.20.21.21:9200"] #查询的ES实例的url elasticsearch.username: "elastic" elasticsearch.password: "5tVF53HD4TyqeLlflR0q"Kibana启动脚本 vim /usr/local/src/tyuninstall/kibana/kibana.sh #!/bin/bash #extract kibana pid id=`ps -elf | grep -i kibana | grep 'max-http-header-size' | awk '{print $4}'` if [ $# -eq 1 ];then case $1 in start) if [ -z "$id" ];then echo -e '\033[32mkibana starting.....\033[0m' nohup sh /usr/local/src/tyuninstall/kibana/bin/kibana & > /dev/null sleep 5 echo -e '\033[32mkibana started successfully\033[0m' else echo -e "\033[31mkibana Already started!,pid:${id}\033[0m" fi ;; restart) if [ -z "$id" ];then echo -e '\033[32mkibana restart....\033[0m' nohup sh /usr/local/src/tyuninstall/kibana/bin/kibana & > /dev/null sleep 5 echo -e '\033[32mkibana restarted successfully\033[0m' else echo -e '\033[32mkibana restart....\033[0m' kill -9 $id &> /dev/null nohup sh /usr/local/src/tyuninstall/kibana/bin/kibana & > /dev/null sleep 5 echo -e '\033[32mkibana restarted successfully\033[0m' fi ;; status) if [ -z "$id" ];then echo -e '\033[32mkibana Not started!\033[0m' else echo -e "\033[32mkibana running,pid:${id}\033[0m" fi ;; stop) if [ -z "$id" ];then echo -e '\033[31mkibana Not started!\033[0m' else echo -e '\033[32mkibana Stopping....\033[0m' kill -9 $id &> /dev/null sleep 2 echo -e '\033[32mkibana stopped successfully\033[0m' fi ;; *) echo -e '\033[31mstart|restart|status|stop\033[0m' esac else echo -e '\033[31mstart|restart|status|stop\033[0m' fichmod +x /usr/local/src/tyuninstall/kibana/kibana.sh /usr/local/src/tyuninstall/kibana/kibana.sh start 启动Kibana服务 ss -tunl | grep -E '5601'
Filebeat解压的文件安放的位置 C:\Program Files\ 以管理员运行PowerShell PS C:\Users\Administrator> cd 'C:\Program Files\Filebeat' PS C:\Program Files\Filebeat> .\install-service-filebeat.ps1 注：如果有报错的话请在此目录下面执行命令来安装 set-ExecutionPolicy RemoteSigned PS C:\Program Files\Winlogbeat> Start-Service filebeat 查看服务的启动 services.msc 命令编辑文件filebeat.yml filebeat.inputs: - type: log encoding: GB2312 paths: - C:\inetpub\logs\LogFiles\*\*.log tags: ["iislog"] ignore_older: "72h" fields: list: iislog #fields_under_root: true output.redis: # Array of hosts to connect to. hosts: ["10.20.21.21:6379","10.20.21.46:6379"] db: 8 password: "TYUN2020" key: "%{[fields.list]}" loadbalance: true worker: 2 #timeout: 5

日志采集

grok 在线调试器，匹配日志字段URL调式地址https://grokdebug.herokuapp.com/
IIS日志内容2020-08-11 14:29:58 10.20.21.33 GET /Maintain.html - 8088 - 114.114.100.117 Mozilla/5.0+(Windows+NT+10.0;+Win64;+x64)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/84.0.4147.125+Safari/537.36 200 0 0 536
Logstash收集IIS日志编写vim /usr/local/src/tyuninstall/logstash/conf.d/iis.conf input { redis { host => "10.20.21.21" port => "6379" key => "iislog" data_type => "list" type => "log" db => "8" password => "TYUN2020" threads => 1 batch_count => 1 } } filter{ if "iislog" in [tags]{ if [message] =~ "^#" {drop {}} grok { match => ["message", "%{TIMESTAMP_ISO8601:log_timestamp} %{IPORHOST:site} %{WORD:method} %{URIPATH:page} %{NOTSPACE:querystring} %{NUMBER:port} %{NOTSPACE:username} %{IPORHOST:clienthost} %{NOTSPACE:useragent} %{NUMBER:response} %{NUMBER:subresponse} %{NUMBER:scstatus} %{NUMBER:time_taken}"] } geoip{ source=>"clienthost" target => "geoip" database => "/usr/local/src/tyuninstall/logstash/geoip/GeoLite2-City.mmdb" add_field => [ "[geoip][coordinates]", "%{[geoip][longitude]}" ] add_field => [ "[geoip][coordinates]", "%{[geoip][latitude]}" ] } mutate { convert => [ "[geoip][coordinates]", "float"] } date{ match => [ "log_timestamp", "yyyy-MM-dd HH:mm:ss" ] #timezone => "Asia/Shanghai" timezone => "+00:00" target => "@timestamp" } mutate { remove_field => ["log_timestamp"] } ruby{ code => "event.timestamp.time.localtime" } mutate { convert => { "port" => "integer" "response" => "integer" "time_taken" => "integer" } } if [useragent] == "ChinaCache" {drop {}} if [useragent] != "-" and [useragent] != "" { useragent{ add_tag => "ua" source=>"useragent" } if "ua" in [tags]{ if [device] == "iPhone" and [os] == "Other"{ mutate { update => { "os" => "IOS" "os_name" => "IOS" } } } if [device] == "iPad" and [os] == "Other"{ mutate{ update =>{ "os" => "IOS" "os_name" => "IOS" } } } if [os_name] == "iOS"{ mutate{ update =>{ "os" => "IOS" "os_name" => "IOS" } } } if [device] == "Spider"{ mutate{ update =>{ "os" => "Windows" "os_name" => "Windows" } } } if [os_name] == "Other"{ mutate{remove_field => "os_name"}} if [device] == "Other"{ mutate{remove_field => "device"}} if [name] == "Other"{ mutate{remove_field => "name"}} if [os] == "Other"{ mutate{remove_field => "os"}} if [os] =~ "Android*" or [os] =~ "IOS*"{ mutate{ add_field => { "platform" => "mobile"} } } else{ mutate{ add_field => { "platform" => "pc" } } } } } } } output { if "iislog" in [tags]{ elasticsearch { hosts => "10.20.21.21:9200" user => "elastic" password => "5tVF53HD4TyqeLlflR0q5tVF53HD4TyqeLlflR0q" index => "logstash-%{+YYYY.MM.dd}" #manage_template => false #document_type => "iislog" } #stdout {codec => rubydebug} } }