loki日志收集

# loki服务搭建

[【附件】loki-linux-amd64.zip](/media/attachment/2022/05/loki-linux-amd64.zip)
直接二进制文件加配置启动

```shell
curl -O -L "https://github.com/grafana/loki/releases/download/v1.5.0/loki-linux-amd64.zip"
unzip loki-linux-amd64.zip
chmod a+x loki-linux-amd64
#指定配置文件启动
nohup ./loki-linux-amd64 -config.file=/data/loki/loki.yml  >loki.log 2>&1 &   
```

loki 配置文件

```yaml
auth_enabled: false

server:
  http_listen_port: 3100 #服务监听端口

ingester:
  lifecycler:
    address: 127.0.0.1
    ring:
      kvstore:
        store: inmemory
      replication_factor: 1
    final_sleep: 0s
  chunk_idle_period: 5m #块被刷新后在内存中的保留时间（在刷新块之后，它们应该在内存中保留多长时间。）
  chunk_retain_period: 30s #多久块未被刷新，在内存中存留 （在之前没有更新的情况下，块应该在内存中保留多长时间）
  max_transfer_retries: 0

schema_config: #数据库设置
  configs:
    - from: 2020-07-01
      store: boltdb
      object_store: filesystem
      schema: v11
      index:
        prefix: index_
        period: 168h

storage_config:
  boltdb:
    directory: /tqls_system/apps/loki/index

filesystem:
    directory: /tqls_system/apps/loki/chunks

limits_config:
  enforce_metric_name: false
  reject_old_samples: true
  reject_old_samples_max_age: 168h  # 接受可摄入的最大样本年龄
  ingestion_rate_mb: 15  #每个用户摄入速率限制，即 每秒 样本量，默认值为4M
  ingestion_burst_size_mb: 30   #每用户摄入速率限制， 突发 样本量，默认值为6M

chunk_store_config:
  max_look_back_period: 0s

table_manager:
  retention_deletes_enabled: true #日志保留周期开关，默认为false
  retention_period: 336h #日志保留周期，必须为index period 的倍数 336h为半个月
```

# 应用接入loki实现日志收集

[【附件】promtail-linux-amd64.zip](/media/attachment/2022/05/promtail-linux-amd64.zip)
loki可以使用promtail搜集日志文件

以二进制文件加配置文件启动promtail

```shell
curl -O -L "https://github.com/grafana/loki/releases/download/v1.5.0/promtail-linux-amd64.zip"
unzip promtail-linux-amd64.zip

chmod a+x promtail-linux-amd64
#指定配置文件启动
nohup ./promtail-linux-amd64 -config.file=/data/promtail/promtail.yml  >promtail.log 2>&1 &   
```

promtail配置文件

```yaml
# Promtail Server Config
server:
  http_listen_port: 9080
  grpc_listen_port: 0

# Positions
positions:
  filename: /tmp/positions.yaml

# Loki服务器的地址
clients:
  - url: http://10.2.8.217:3100/loki/api/v1/push #配置loki服务ip

scrape_configs:
- job_name: efsapi
    static_configs:
      - targets:
          - localhost
        labels:
          job: api
          host: localhost
          __path__:/tqls_system/logs/api.log #一个job_name表示一个任务，可以写多个job_name监听多个文件
          
- job_name: efsdatabus
    static_configs:
      - targets:
          - localhost
        labels:
          job: databus
          host: localhost
          __path__:/tqls_system/logs/databus.log

- job_name: efsureport
    static_configs:
      - targets:
          - localhost
        labels:
          job: ureport
          host: localhost
          __path__:/tqls_system/logs/ureport.log
```

# 查看收集的日志

使用grafana查询loki收集的日志

## 在Configuration中添加数据源

![image-20220509151453367](https://tqls-mrdoc.oss-cn-beijing.aliyuncs.com/mr-doc/image-20220509151453367.png)

选中loki

![image-20220509151626223](https://tqls-mrdoc.oss-cn-beijing.aliyuncs.com/mr-doc/image-20220509151626223.png)

写入loki服务地址，再测试一下

![image-20220509151726775](https://tqls-mrdoc.oss-cn-beijing.aliyuncs.com/mr-doc/image-20220509151726775.png)

## 查看日志

![image-20220509152324330](https://tqls-mrdoc.oss-cn-beijing.aliyuncs.com/mr-doc/image-20220509152324330.png)

![image-20220509152258918](https://tqls-mrdoc.oss-cn-beijing.aliyuncs.com/mr-doc/image-20220509152258918.png)

实时查看日志

![image-20220509152501364](https://tqls-mrdoc.oss-cn-beijing.aliyuncs.com/mr-doc/image-20220509152501364.png)

搜索日志

![image-20220509152607489](https://tqls-mrdoc.oss-cn-beijing.aliyuncs.com/mr-doc/image-20220509152607489.png)

可以查看上下几行日志

![image-20220509152646336](https://tqls-mrdoc.oss-cn-beijing.aliyuncs.com/mr-doc/image-20220509152646336.png)