對于fluentd這個組件來(lai)說(shuo)，你(ni)是負責(ze)抓取日(ri)志的(de)，它可以從docker的(de)控(kong)制(zhi)臺(tai)里(li)抓取，也可以從指定文(wen)(wen)件夾(jia)里(li)抓取，對于文(wen)(wen)件夾(jia)里(li)存(cun)儲的(de)日(ri)志文(wen)(wen)件，我們(men)需(xu)要(yao)先(xian)配置(zhi)logback，然后再進行fluentd的(de)configmap的(de)配置(zhi)，這樣才(cai)能把(ba)持久化的(de)日(ri)志抓取出來(lai)，并推(tui)送到elastic這種(zhong)存(cun)儲介(jie)質里(li)。

logback控制存儲位置

<?xml version="1.0" encoding="UTF-8"?>
<configuration>
    <property name="logPath" value="/var/log/"/>
    <springProperty scope="context" name="springAppName" source="spring.application.name"/>
    <appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
        <encoder>
            <pattern>%date-%level-%X{X-B3-TraceId:-}-%X{X-B3-SpanId:-}-[%file:%line]-%msg%n</pattern>
        </encoder>
    </appender>

    <appender name="fileInfoLog" filePermissions="rw-r--r--" class="ch.qos.logback.core.rolling.RollingFileAppender">
        <encoder class="net.logstash.logback.encoder.LoggingEventCompositeJsonEncoder">
            <providers class="net.logstash.logback.composite.loggingevent.LoggingEventJsonProviders">
                <pattern>
                    <pattern>
                        {
                        "level": "%level",
                        "application": "${springAppName:-}",
                        "trace": "%X{X-B3-TraceId:-}",
                        "span": "%X{X-B3-SpanId:-}",
                        "exportable": "%X{X-Span-Export:-}",
                        "pid": "${PID:-}",
                        "thread": "%thread",
                        "class": "%logger{40}",
                        "message": "%message"
                        }
                    </pattern>
                </pattern>
            </providers>
        </encoder>
        <!--滾動策略-->
        <rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
            <!--路徑-->
            <fileNamePattern>${logPath}/info.%d.log</fileNamePattern>
            <maxHistory>7</maxHistory>
        </rollingPolicy>
    </appender>
    <root level="INFO">
        <appender-ref ref="STDOUT"/>
        <appender-ref ref="fileInfoLog"/>
   </root>

fluentd以sidecar邊車方法注冊到pod里

這種(zhong)sidecar設(she)計(ji)主(zhu)要為(wei)了解耦，它與pod里(li)的容器共享存儲卷(juan)，事實上(shang)就是(shi)(shi)讀取容器產生的日志，然后把日志推送(song)到(dao)存儲介質里(li)，本例是(shi)(shi)推送(song)到(dao)elastic里(li)，通(tong)過kibana進行查詢和分(fen)析,k8s的yaml部署腳(jiao)本如下

kind: Deployment
apiVersion: apps/v1
metadata:
  name: hello-world-deployment
  namespace: saas
  labels:
    app: hello-world
spec:
  replicas: 1
  selector:
    matchLabels:
      app: hello-world
  template:
    metadata:
      labels:
        app: hello-world
    spec:
      containers:
        - name: hello-world
          image: 172.17.0.22:8888/saas/hello-world:latest
          imagePullPolicy: Always
          ports:
            - containerPort: 9001
          env:
            - name: spring.profiles.active
              value: prod
          volumeMounts:
            - name: varlog
              mountPath: /var/log
        - name: fluent-sidecar
          image: registry.cn-beijing.aliyuncs.com/k8s-mqm/fluentd-elasticsearch:v2.1.0
          env:
            - name: FLUENTD_ARGS
              value: -c /etc/fluentd-config/fluentd.conf
          volumeMounts:
            - name: varlog
              mountPath: /var/log
            - name: config-volume
              mountPath: /etc/fluentd-config
      volumes:
        - name: varlog
          emptyDir: {}
        - name: config-volume
          configMap:
            name: fluentd-config

最后是(shi)，為fluentd添(tian)加配置，就是(shi)k8s里的configmap，注意(yi)，它(ta)是(shi)針(zhen)對(dui)某個(ge)namespace來說(shuo)的，這(zhe)個(ge)configmap不能(neng)跨namespace訪問(wen)。

一(yi)(yi)般(ban)來說，一(yi)(yi)個(ge)namespace在kibana里(li)可以(yi)對應一(yi)(yi)個(ge)索引，你在配置fluentd時，可以(yi)使用tag和(he)logstash_prefix來實現索引前綴的(de)功能

<source>
type tail
format json
path /var/log/*.log
pos_file /var/log/*.log.pos
tag test.*
</source>

<match **>
@id elasticsearch
@type elasticsearch
@log_level debug
type_name fluentd
host elasticsearch.elk
port 9200
logstash_format true
logstash_prefix test #表示索引的前綴，對應source里的tag，一個namespace可以是一個，對應一組微服務，方便進行日志追蹤
flush_interval 10s
</match> 	  	

最后在kibana里建立索引

Management->create index，選擇(ze)test-*，保存即(ji)可