Graylog 安裝架設
這是一套免費開源Log收集分析系統，可以自訂過濾器、報表，並且發送警告通知mail，以下記錄相關安裝設定方式
作業系統 Ubuntu 22.04
# apt-get update && sudo apt-get upgrade
安裝 Java 11
# apt-get install apt-transport-https openjdk-11-jre-headless uuid-runtime pwgen
安裝 MongoDB 建議安裝 6.0 版比較不會有問題
# curl -fsSL https://www.mongodb.org/static/pgp/server-6.0.asc | sudo gpg -o /usr/share/keyrings/mongodb-server-6.0.gpg --dearmor
# touch /etc/apt/sources.list.d/mongodb-org-6.0.list
# echo "deb [ arch=amd64,arm64 signed-by=/usr/share/keyrings/mongodb-server-6.0.gpg ] https://repo.mongodb.org/apt/ubuntu jammy/mongodb-org/6.0 multiverse" | sudo tee /etc/apt/sources.list.d/mongodb-org-6.0.list
# apt-get update
# apt-get install -y mongodb-org
# systemctl start mongod.service
# systemctl enable mongod.service
初始化系統
# ps --no-headers -o comm 1
使用 mongo shell 測試
# mongosh
test> show dbs
test> use config
config>exit
裝 Elasticsearch (不支援最新 8.13.4)
警告： Graylog 不支援 Elasticsearch 7.11 或更高版本。不要升級 Elasticsearch，因為這樣做會破壞您的執行個體！
# wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-7.12.1-amd64.deb
# dpkg -i elasticsearch-7.12.1-amd64.deb
關閉 SSL 連線 (for 8.x 版本)
# vi /etc/elasticsearch/elasticsearch.yml
xpack.security.enabled: false
xpack.security.http.ssl:
enabled: false
啟動 Elasticsearch 服務
# systemctl start elasticsearch.service
設置 Elasticsearch 開機啟動
# systemctl enable elasticsearch.service
設置 Elasticsearch 查看服務
# systemctl status elasticsearch.service
檢查 java 版本，至少要11版以上
# java -version
測試連線資訊
# curl -X GET http://localhost:9200
出現下段
{
"name" : "ubu22",
"cluster_name" : "elasticsearch",
"cluster_uuid" : "dTg337rQREaNF5_xMPc3yw",
"version" : {
"number" : "7.12.1",
"build_flavor" : "default",
"build_type" : "deb",
"build_hash" : "3e5a16cfec50876d20ea77b075070932c6464c7d",
"build_date" : "2021-03-06T05:54:38.141101Z",
"build_snapshot" : false,
"lucene_version" : "8.7.0",
"minimum_wire_compatibility_version" : "6.8.0",
"minimum_index_compatibility_version" : "6.0.0-beta1"
},
"tagline" : "You Know, for Search"
}
安裝 Graylog 6.0
# wget https://packages.graylog2.org/repo/packages/graylog-6.0-repository_latest.deb
# dpkg -i graylog-6.0-repository_latest.deb
# apt-get update
# apt-get install graylog-server
生成一個安全的密碼
# pwgen -N 1 -s 96
u7M4C7lQjNQNUS6OsBbiFpxcAGAvhwRfF49yGvQr8xcKdldzUJUCUQLZzuY0V2D0f
生產 sha256 hash 值
# echo -n "Enter Password: " && head -1 </dev/stdin | tr -d '\n' | sha256sum | cut -d" " -f1
Enter Password: admin
8c6976e5b5410415bde908bd4dee15dfb167a9c873fc4bb8a81f6f
# vi /etc/graylog/server/server.conf
http_bind_address = 0.0.0.0:9000
root_timezone = Asia/Taipei
password_secret = u7M4C7lQjNQNUS6OsBbiFpxcAGAvhwRfF49yGvQr8xcKdldzUJUCUQLZzuY0V2D0fk8CCAZkMD5qxXKy9UA8BFDhj8LWq2hb
root_password_sha2 = 8c6976e5b5410415bde908bd4dee15dfb167a9c873fc4bb8a81f6f2ab448a918
# Email transport
transport_email_enabled = true
transport_email_hostname = 10.1.1.125
transport_email_port = 25
transport_email_use_auth = false
transport_email_from_email = graylog@domain.com.tw
transport_email_socket_connection_timeout = 10s
transport_email_socket_timeout = 10s
transport_email_use_tls = false
transport_email_use_ssl = false
transport_email_web_interface_url = http://graylog.domain.com.tw
存檔
# systemctl start graylog-server.service
# systemctl enable graylog-server.service
# systemctl status graylog-server.service
檢查 9000 port 是否 open
# netstat -tunlp
查看 log ，會有寫 admim 的密碼
# vi /var/log/graylog-server/server.log
http://10.1.1.123:9000
初始化設定，建立CA憑證，選擇1年並按下 Skip provisioning
完成後，登入畫面：帳號/密碼預設為 admin
System → Inputs → 選擇 Syslog UDP
Global 打勾
Title：syslog-udp-input
Bind address：0.0.0.0
Port：514
可以查看是否有open udp 514
# netstat -tunlp
接下來就可以開始設定Client端，將 Log 導入到 Graylog Server
# vi /etc/rsyslog.conf
# 導入 Graylog Server
*.* @10.1.1.123:514
# systemctl restart rsyslog.service
---------------------------------------------------------------------------------------------
System → Inputs → 選擇 Beats
Global 打勾
Title：Sidecar
Bind address：0.0.0.0
Port：5044
System → Sidecars → Create or reuse a token for the graylog-sidecar user
Token Name：test win10
把產生的 Token 複製出來，可以給多台電腦安裝使用
1cg2lir8nk24r5m5gl81d718doacgf8kqftickpdsgde9dv
---------------------------------------------------------------------------------------------
System → Sidecars → Configuration → winlogbeat-default → Edit
Configuration 內容修改如下：
# Needed for Graylog
fields_under_root: true
fields.collector_node_id: ${sidecar.nodeName}
fields.gl2_source_collector: ${sidecar.nodeId}
output.logstash:
hosts: ["${user.graylog_host}:5044"]
path:
data: ${sidecar.spoolDir!"C:\\Program Files\\Graylog\\sidecar\\cache\\winlogbeat"}\data
logs: ${sidecar.spoolDir!"C:\\Program Files\\Graylog\\sidecar"}\logs
tags:
- windows
winlogbeat:
event_logs:
- name: Application
level: critical, error, warning # 告警等級
ignore_older: 48h # 忽略48小時以前的event
- name: System
ignore_older: 48h
- name: Security
event_id: 4624, 4625, 4740 # 只收集特定事件，如果要排除4735，請下 -4735
ignore_older: 48h
- name: Setup
ignore_older: 48h
- name: ForwardedEvents
forwarded: true
ignore_older: 48h
- name: Microsoft-Windows-Windows Defender/Operational
ignore_older: 48h
- name: Microsoft-Windows-Sysmon/Operational
event_id: 22
ignore_older: 48h
- name: Microsoft-Windows-TerminalServices-LocalSessionManager/Operational
ignore_older: 48h
- name: Microsoft-Windows-PowerShell/Operational
ignore_older: 48h
- name: windows PowerShell
ignore_older: 48h
---------------------------------------------------------------------------------------------
下載 Sidecar 安裝程式 for Windows 、Linux 平台
https://github.com/Graylog2/collector-sidecar/releases/
Windows 安裝過程輸入
server_url：http://10.1.1.123:9000/api
node_name：Windows (如果空白，將使用主機名稱)
server_api_token：1h0br6kc5cknsv9qo84vvtcgv75nldd6r8950s0tuodqjiksbqoa
設定檔位置
C:\Program Files\Graylog\sidecar\sidecar.yml
---------------------------------------------------------------------------------------------
檢查版本 ( Ubuntu、Rcoky Linux )
# dpkg -l | grep -E ".(elasticsearch|graylog|mongo)."
# dnf list installed | grep -E ".*(elasticsearch|graylog|mongo).* "
---------------------------------------------------------------------------------------------
Graylog Marketplace 是 Graylog 附加元件的中心資源
尋找、探索並嘗試由 Graylog 社群成員和愛好者創建的 Graylog 附加元件。
提供Plugins插件、Extractors提取器、Content packs內容包和 GELF 庫以及指南和文件。
Content packs 推薦：
Windows 10/11 和 Windows Server 安全性日誌 ( 客製化 Dashboards )
https://community.graylog.org/t/windows-10-11-windows-server-security-log/30969
---------------------------------------------------------------------------------------------
Linux Agent 安裝： (不建議安裝，請使用系統的 rsyslog 套件)
# wget https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-7.2.1-x86_64.rpm
# wget https://github.com/Graylog2/collector-sidecar/releases/download/1.5.0/graylog-sidecar-1.5.0-1.x86_64.rpm
# rpm -ivh filebeat-7.2.1-x86_64.rpm
# rpm -ivh graylog-sidecar-1.5.0-1.x86_64.rpm
# vi /etc/graylog/sidecar/sidecar.yml
# graylog-sidecar -service install
# systemctl start graylog-sidecar
# systemctl enable graylog-sidecar
# systemctl start filebeat
# systemctl enable filebeat
--------------------------------------------------------------------------------------------
SuSE 設定： (透過 rsyslog 套件，且僅倒出 sshd log)
# vi /etc/rsyslog.d/remote.conf
authpriv.* @10.1.1.123:514
# systemctl restart rsyslog.service
# systemctl status rsyslog.service
---------------------------------------------------------------------------------------------
Rocky Linux 設定： (透過 rsyslog 套件，且僅倒出 sshd log)
# vi /etc/rsyslog.conf
authpriv.* @10.1.1.123:514
# systemctl restart rsyslog.service
# systemctl status rsyslog.service
---------------------------------------------------------------------------------------------
Ubuntu 設定： (透過 rsyslog 套件，且僅倒出 sshd log)
# vi /etc/rsyslog.d/50-default.conf
authpriv.* @10.1.1.123:514
# systemctl restart rsyslog.service
# systemctl status rsyslog.service
---------------------------------------------------------------------------------------------
版本 Release 變更日誌：
https://go2docs.graylog.org/current/changelogs/changelog.html
---------------------------------------------------------------------------------------------
e-mail 通知設定
Alerts → Notifications → Create Notification
Notification Type： Email Notification
Sender (Optional)： graylog@domain.com.tw
Email recipient(s) (Optional)： abc@domain.com.tw
Time zone for date/time values (Optional)： Taipei
---------------------------------------------------------------------------------------------
資料庫占用空間位置
[root@graylog mongodb]# pwd
/var/lib/mongodb
---------------------------------------------------------------------------------------------
預設情況下 inactive sidecar 未活動用戶端，它們會在 14 天後自動過期刪除
System → Configurations → Sidecars → Expiration threshold:P14D
預設 log 保留天數
System → Indices → Default index set → 可在上方點 Enable stats 會顯示 Log 占用容量
Max. in storage: 40 days 最大保存天數
Min. in storage: 30 days 最小保存天數
---------------------------------------------------------------------------------------------
Search Queries Syntax 搜尋查詢語法 ( AND、OR、NOT )
https://go2docs.graylog.org/current/making_sense_of_your_log_data/writing_search_queries.html
---------------------------------------------------------------------------------------------
User 帳號登入本機成功
event_code:4624 AND winlog_event_data_LogonType:2 AND winlog_event_data_ProcessName:C\:\\Windows\\System32\\svchost.exe
---------------------------------------------------------------------------------------------
帳戶登入失敗 (包含遠端桌面連線)
event_code:4625
---------------------------------------------------------------------------------------------
開機 及 上次關機時間 Event ID：1、6005、7001
message:"系統已從低電源狀態回復" OR message:"事件記錄檔服務已啟動" OR message:"客戶經驗改進計畫的使用者登入通知"
---------------------------------------------------------------------------------------------
SSH 、FTP 登入失敗 ( Search 指令)
message:"failure for" OR message:"Failed" OR message:"failure;"
message:"pam_unix(sshd:auth): authentication failure;" NOT source:Cupid
message:"pam_unix(vsftpd:auth): authentication failure;" NOT source:Cupid
---------------------------------------------------------------------------------------------
AD 帳號新增、刪除、啟用、停用
event_code:4720 event_code:4726 event_code:4722 event_code:4725 AND source:AD
---------------------------------------------------------------------------------------------
Alerts → Notifications
Sender GrayLog <graylog@domain.com.tw>
Subject Graylog 事件通知： ${event_definition_title} - ${foreach backlog message}${message.fields.winlog_event_data_TargetUserName}${end}
Email Recipients abc@domain.com.tw
Time Zone Asia/Taipei
Email Body：
--- [Event Definition] ---------------------------
標題： ${event_definition_title}
--- [Event] --------------------------------------
告警連結: http://graylog.domain.com.tw:9000/alerts/${event.id}/replay-search
時間戳記: ${event.timestamp}
${foreach event.fields field} ${field.key}: ${field.value}
${end}
${if backlog}
--- [Backlog] ------------------------------------
${foreach backlog message}
登入主機: ${message.source}
主機 IP: ${message.fields.gl2_remote_ip}
來源 IP: ${message.fields.winlog_event_data_IpAddress}
登入帳號： ${message.fields.winlog_event_data_TargetUserName}
完整訊息： ${message}
${end}
${end}
Email HTML Body：
<table width="100%" border="0" cellpadding="10" cellspacing="0" style="background-color:#f9f9f9;border:none;line-height:1.2"><tbody>
<tr style="line-height:1.5"><th colspan="2" style="background-color:#e6e6e6">事件定義</th></tr>
<tr><td width="200px">標題</td><td>${event_definition_title}</td></tr>
</tbody></table>
<br /><table width="100%" border="0" cellpadding="10" cellspacing="0" style="background-color:#f9f9f9;border:none;line-height:1.2"><tbody>
<tr><th colspan="2" style="background-color:#e6e6e6;line-height:1.5">事件</th></tr>
<tr><td>告警連結</td><td>http://graylog.domain.com.tw:9000/alerts/${event.id}/replay-search</td></tr>
<tr><td width="200px">時間戳記</td><td>${event.timestamp}</td></tr>
</tbody></table>
${if backlog}
<br /><table width="100%" border="0" cellpadding="10" cellspacing="0" style="background-color:#f9f9f9;border:none;line-height:1.2"><tbody>
<tr><th style="background-color:#e6e6e6;line-height:1.5">訊息</th></tr>
${foreach backlog message}
<tr><td>登入主機： ${message.source}</td></tr>
<tr><td>主機 IP： ${message.fields.gl2_remote_ip}</td></tr>
<tr><td>來源 IP： ${message.fields.winlog_event_data_IpAddress}</td></tr>
<tr><td>登入帳號： ${message.fields.winlog_event_data_TargetUserName}</td></tr>
<tr><td>完整訊息： ${message}</td></tr>
${end}
</tbody></table>
${end}
---------------------------------------------------------------------------------------------
Graylog 儲存的所有日誌訊息都會在 Elasticsearch 中建立索引
存放路徑：
/var/lib/elasticsearch/nodes/0
-------------------------------------------------------------------------------------------
另一套免費收集 Log 軟體
1. 安裝 nxlog-ce-3.2.2329.msi
2. 將 nxlog.conf 覆蓋到 C:\Program Files\nxlog\conf\
3. 開啟 services.msc ，重新啟動 nxlog ，將啟動類型改成「自動」
4. 檢查 C:\Program Files\nxlog\data\nxlog.log 紀錄是否有錯誤
--------------------------------------------------------------------------------------------
以下是 nxlog.conf 內容：
Panic Soft
#NoFreeOnExit TRUE
define ROOT C:\Program Files\nxlog
define CERTDIR %ROOT%\cert
define CONFDIR %ROOT%\conf\nxlog.d
define LOGDIR %ROOT%\data
include %CONFDIR%\\*.conf
define LOGFILE %LOGDIR%\nxlog.log
LogFile %LOGFILE%
Moduledir %ROOT%\modules
CacheDir %ROOT%\data
Pidfile %ROOT%\data\nxlog.pid
SpoolDir %ROOT%\data
<Extension syslog>
Module xm_syslog
</Extension>
<Extension _gelf>
Module xm_gelf
</Extension>
<Input in>
Module im_msvistalog
ReadFromLast TRUE
SavePos TRUE
Query <QueryList>\
<Query Id="0">\
<Select Path="Security">*[System[(EventID='4625')]]</Select>\
<Select Path="Security">*[System[(EventID='4740')]]</Select>\
<Select Path="Security">*[System[(EventID='4720')]]</Select>\
<Select Path="Security">*[System[(EventID='4726')]]</Select>\
<Select Path="Security">*[System[(EventID='4723')]]</Select>\
<Select Path="System">*[System[(EventID='1')]]</Select>\
<Select Path="System">*[System[(EventID='6005')]]</Select>\
<Select Path="System">*[System[(EventID='7001')]]</Select>\
</Query>\
</QueryList>
</Input>
<Output out>
Module om_udp
Host 10.1.1.123
Port 514
# OutputType GELF 開啟：收到的Log會變成亂碼
# Exec to_syslog_snare(); 開啟：收到的Log會連在一起，source 欄位顯示電腦名稱
</Output>
<Route 1>
Path in => out
</Route>
-----------------------------------------------------------------------------------------------
如要稽核應用程式 或 Sysmon，可加入如下：
<Select Path="Application">*</Select>\
<Select Path="Microsoft-Windows-Sysmon/Operational">*[System[(EventID='22')]]</Select>\
-----------------------------------------------------------------------------------------------
另一套免費收集 Log 軟體 Winlogbeat (比較推薦)
1. 下載 Winlogbeat (選擇 Windows ZIP x86_64)
https://www.elastic.co/downloads/beats/winlogbeat
2. 將 zip 解壓縮到 C:\Program Files\
並將 winlogbeat-8.14.3-windows 資料夾名稱更改為 winlogbeat
3. 覆蓋 winlogbeat.yml 到 C:\Program Files\winlogbeat
4. 以系統管理員身分執行 Windows PowerShell
5. 輸入 cd 'C:\Program Files\winlogbeat\'
安裝 .\install-service-winlogbeat.ps1
6. 測試 winlogbeat.yml 是否正確，輸入 .\winlogbeat.exe test config
7. 啟動服務，執行 services.msc 尋找 winlogbeat 進行啟動 (自動：延遲啟動)
-------------------------------------------------------------------------------------------
winlogbeat.yml 修改內容：
winlogbeat.event_logs:
- name: Application
level: critical, error, warning
ignore_older: 48h
- name: System
event_id: 1, 41, 6005, 7001
ignore_older: 48h
- name: Security
event_id: 4625, 4648, 4740, 4720, 4726, 4723
ignore_older: 48h
# - name: Microsoft-Windows-Sysmon/Operational
# event_id: 22
# - name: Windows PowerShell
# event_id: 400, 403, 600, 800
# - name: Microsoft-Windows-PowerShell/Operational
# event_id: 4103, 4104, 4105, 4106
# - name: ForwardedEvents
# tags: [forwarded]
# 關閉 elasticsearch 輸出
#output.elasticsearch:
# Array of hosts to connect to.
# hosts: ["localhost:9200"]
# pipeline: "winlogbeat-%{[agent.version]}-routing"
# 啟動 logstash 輸出
output.logstash:
# The Logstash hosts
hosts: ["graylog.domain.com.tw:5044"]
------------------------------------------------------------------------------------------
Windows 事件識別碼
應用程式：
1000：應用程式當機事件。失敗的應用程式名稱：
11707：安裝軟體
11724：移除軟體
1033：安裝軟體 (紀錄名稱、版本、語言、製造商)
1034：移除軟體 (紀錄名稱、版本、語言、製造商)
安全性：
4625：帳戶無法登入
4648：使用明確宣告的認證嘗試登入
4720：已建立使用者帳戶
4723：嘗試變更帳戶的密碼
4724：嘗試重設帳戶的密碼
4726：使用者帳戶已刪除
4740：已鎖定使用者帳戶
6416：系統已識別新的外接式裝置
系統：
1：系統已從低電源狀態回復
41：系統已重新啟動，但未先完全關閉。 如果系統停止回應、損毀或意外中斷電源，可能會造成此錯誤。
45：系統無法成功載入故障轉儲驅動程式
46：損毀傾印初始化失敗
6005：事件記錄檔服務已啟動
6008：系統上次發生意外的關機
7001：客戶經驗改進計畫的使用者登入通知
留言