2022年7月15日 星期五

將 Oxidized 外掛上 Librenms

 最近又有人在問我如何做 Cisco 或 FortiGate 設定組態 (configuration)

如何自動備份?

N年前我就有寫過類似的東西,只不過在當時我是用 Perl + telnet DIY小程式去做.

當然也是改成 Perl + ssh DIY 的小程式去做.但 N 年過去了,

目前已有太多工具可以逹成這個目地.

接下來的筆記是記錄,我是如何透過 Oxidized  去做這件事,

順便將其整合進 LibreNMS內,

當然只使用 Oxidized 也是 OK 的.

參考文件

http://blog.jason.tools/2021/02/librenms-oxidized.html

官方 URL

https://docs.librenms.org/Extensions/Oxidized/

https://github.com/ytti/oxidized

之前寫的東西

http://xrcd2.blogspot.com/2013/02/cisco-show-run-config-perl.html

自動備份 cisco 設備 show run 的 config ( 使用 perl )

http://xrcd2.blogspot.com/2016/01/cisco-ios-configuration-cisco-config.html

Cisco IOS 設備 configuration 自動版本管理的方式 ( cisco config auto 2 svn )

------------------------------------------------------------------------

安裝 Oxidized (OS CentOS 7.9)

yum install -y centos-release-scl-rh

yum install -y rh-ruby24 rh-ruby24-ruby-devel

yum install make cmake which sqlite-devel openssl-devel libssh2-devel ruby gcc ruby-devel libicu-devel gcc-c++

scl enable rh-ruby24 bash

Install the gems: 

gem install oxidized oxidized-web

===============
git 設定

git config --global user.name "cisco"
git config --global user.email "cisco@cisco.lab.local"

===============

[root@centos7 ~]# pwd
/root
[root@centos7 ~]# 

[root@centos7 ~]# oxidized
edit ~/.config/oxidized/config
[root@centos7 ~]# oxidized
edit ~/.config/oxidized/router.db 

設定檔參考如下:

[root@centos7 oxidized]# pwd
/root/.config/oxidized
[root@centos7 oxidized]# cat config
---
username: username
password: password
model: ios
resolve_dns: false
interval: 300
log: /root/.config/oxidized/logs/oxidized.log
use_syslog: false
debug: false
threads: 30
timeout: 20
retries: 3
prompt: !ruby/regexp /^([\w.@-]+[#>]\s?)$/
rest: 192.168.100.105:8888
next_adds_job: false
vars: {}
groups: {}
models: {}
pid: "/root/.config/oxidized/pid"
crash:
  directory: "/root/.config/oxidized/crashes"
  hostnames: false
stats:
  history_size: 10
input:
  default: ssh, telnet
  debug: false
  ssh:
    secure: false
  ftp:
    passive: true
  utf8_encoded: true
output:
  default: git
  git:
    user: cisco
    email: cisco@cisco.lab.local
    repo: "/root/.config/oxidized/configs/configs.git"
source:
  default: csv
  csv:
    file: "/root/.config/oxidized/router.db"
    delimiter: !ruby/regexp /:/
    map:
      name: 0
      model: 1
      ip: 2
      username: 3
      password: 4
      group: 5
    gpg: false
model_map:
  juniper: junos
  cisco: ios
============================
[root@centos7 oxidized]# cat router.db 

csr1000v:ios:192.168.100.222:cisco:cisco:configs

[root@centos7 oxidized]# 

=============================
整合進 librenms
===================

[root@centos7 oxidized]# su - librenms
Last login: Sat Jul 16 12:39:31 CST 2022 on pts/0
-bash-4.2$ lnms config:set oxidized.enabled true
-bash-4.2$ lnms config:set oxidized.url http://192.168.100.105:8888
-bash-4.2$ lnms config:set oxidized.features.versioning true
-bash-4.2$ lnms config:set oxidized.group_support true
-bash-4.2$ lnms config:set oxidized.default_group default
-bash-4.2$ lnms config:set oxidized.reload_nodes true
-bash-4.2$ 


=======================
Cisco CSR 1000v  show run 
==================

csr1000v#sh run 
Building configuration...

Current configuration : 1204 bytes
!
! Last configuration change at 13:29:30 TPE Sat Jul 16 2022 by cisco
!
version 15.5
service timestamps debug datetime localtime
service timestamps log datetime localtime
no platform punt-keepalive disable-kernel-core
platform console auto
!
hostname csr1000v
!
boot-start-marker
boot-end-marker
!
!
!
no aaa new-model
clock timezone TPE 8 0
!
!
!
!
!
!
!
!
!



!
!
!
!
!
!
!
!
!
!         
subscriber templating
!
multilink bundle-name authenticated
!
!
!
license udi pid CSR1000V sn XXXXXXXXXXXXX
license boot level ax
spanning-tree extend system-id
!
username cisco privilege 15 secret 5 $1$7wax$evNlQZGH2VorRL3bm/SRV0
!
redundancy
!
!
!
!
!
!
! 
!
!
!
!
!
!
!
!
!
!
!
!
! 
! 
!
interface GigabitEthernet1
 ip address 192.168.100.222 255.255.255.0
 negotiation auto
!
interface GigabitEthernet2
 no ip address
 shutdown
 negotiation auto
!
interface GigabitEthernet3
 no ip address
 shutdown
 negotiation auto
!
!
virtual-service csr_mgmt
 ip shared host-interface GigabitEthernet1
!
ip forward-protocol nd
!
no ip http server
ip http secure-server
!
!
snmp-server community cisco RO
!
!
control-plane
!
!
line con 0
 stopbits 1
line vty 0 4
 login local
 transport input ssh
!
ntp server 168.95.195.12
!
end

csr1000v# 

=====================

將  oxidized 設定成服務

====================

vi /etc/profile.d/rh-ruby24.sh
#!/bin/bash
source /opt/rh/rh-ruby24/enable
export X_SCLS="`scl enable rh-ruby24 'echo $X_SCLS'`"
export PATH=$PATH:/opt/rh/rh-ruby24/root/usr/bin/ruby


ln -s /opt/rh/rh-ruby24/root/usr/local/bin/oxidized /usr/local/bin/oxidized



====================


vi  /lib/systemd/system/oxidized.service
 
# /lib/systemd/system/oxidized.service
[Unit]
Description=Oxidized - Network Device Configuration Backup Tool
After=network-online.target multi-user.target
Wants=network-online.target
 
[Service]
ExecStart=/usr/local/bin/oxidized
KillSignal=SIGKILL
User=root
 
[Install]
WantedBy=multi-user.target


===========================

 
chmod +x /lib/systemd/system/oxidized.service
 
vi /etc/ld.so.conf

#增加 /opt/rh/rh-ruby24/root/usr/lib64
 
ldconfig
ldconfig -v

#檢查 ruby ldconf
 
chmod +x /lib/systemd/system/oxidized.service
 
systemctl enable oxidized.service
systemctl start oxidized.service
systemctl status oxidized.service


====================

Demo















2022年6月2日 星期四

CentOS 7 安裝 NGINX ModSecurity WAF 筆記

 [root@centos7 yum.repos.d]# vi epel.repo 

[epel]

name=Extra Packages for Enterprise Linux 7 - $basearch

#baseurl=http://download.fedoraproject.org/pub/epel/7/$basearch

metalink=https://mirrors.fedoraproject.org/metalink?repo=epel-7&arch=$basearch

failovermethod=priority

enabled=1

gpgcheck=1

gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL-7


[epel-debuginfo]

name=Extra Packages for Enterprise Linux 7 - $basearch - Debug

#baseurl=http://download.fedoraproject.org/pub/epel/7/$basearch/debug

metalink=https://mirrors.fedoraproject.org/metalink?repo=epel-debug-7&arch=$basearch

failovermethod=priority

enabled=0

gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL-7

gpgcheck=1


[epel-source]

name=Extra Packages for Enterprise Linux 7 - $basearch - Source

#baseurl=http://download.fedoraproject.org/pub/epel/7/SRPMS

metalink=https://mirrors.fedoraproject.org/metalink?repo=epel-source-7&arch=$basearch

failovermethod=priority

enabled=0

gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL-7

gpgcheck=1


====================================

[root@centos7 /]# vi /etc/yum.repos.d/nginx.repo 

[nginx-stable]

name=nginx stable repo

baseurl=http://nginx.org/packages/centos/$releasever/$basearch/

gpgcheck=1

enabled=1

gpgkey=https://nginx.org/keys/nginx_signing.key

module_hotfixes=true


[nginx-mainline]

name=nginx mainline repo

baseurl=http://nginx.org/packages/mainline/centos/$releasever/$basearch/

gpgcheck=1

enabled=0

gpgkey=https://nginx.org/keys/nginx_signing.key

module_hotfixes=true


======================================

[root@centos7 yum.repos.d]# vi remi.repo 



# Repository: http://rpms.remirepo.net/

# Blog:       http://blog.remirepo.net/

# Forum:      http://forum.remirepo.net/


[remi]

name=Remi's RPM repository for Enterprise Linux 7 - $basearch

#baseurl=http://rpms.remirepo.net/enterprise/7/remi/$basearch/

#mirrorlist=https://rpms.remirepo.net/enterprise/7/remi/httpsmirror

mirrorlist=http://cdn.remirepo.net/enterprise/7/remi/mirror

enabled=1

gpgcheck=1

gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-remi


[remi-php55]

name=Remi's PHP 5.5 RPM repository for Enterprise Linux 7 - $basearch

#baseurl=http://rpms.remirepo.net/enterprise/7/php55/$basearch/

#mirrorlist=https://rpms.remirepo.net/enterprise/7/php55/httpsmirror

mirrorlist=http://cdn.remirepo.net/enterprise/7/php55/mirror

# NOTICE: common dependencies are in "remi-safe"

enabled=1

gpgcheck=1

gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-remi


=================================

參考 URL


https://github.com/SpiderLabs/ModSecurity/


https://github.com/SpiderLabs/ModSecurity-nginx


https://github.com/SpiderLabs/owasp-modsecurity-crs


===================================


cd /opt

git clone https://github.com/SpiderLabs/ModSecurity

cd ModSecurity

git checkout -b v3/master origin/v3/master

sh build.sh

git submodule init

git submodule update

./configure

make

make install

cd ..

git clone https://github.com/SpiderLabs/ModSecurity-nginx.git modsecurity-nginx

wget https://nginx.org/download/nginx-1.22.0.tar.gz

tar -zvxf nginx-1.22.0.tar.gz 

cd nginx-1.22.0/


build a dynamic module


./configure --with-compat --add-dynamic-module=/opt/modsecurity-nginx

make modules

cd objs/

cp ngx_http_modsecurity_module.so /etc/nginx/modules/

cp /opt/ModSecurity/modsecurity.conf-recommended /etc/nginx/modsecurity.conf

在 /etc/nginx/nginx.conf 放入 load_module

load_module modules/ngx_http_modsecurity_module.so;

在 /etc/nginx/conf.d/default.conf(或其他 ) 的 server 內放 

    modsecurity on;

    modsecurity_rules_file /etc/nginx/modsecurity.conf;


===========================

cp /opt/ModSecurity/unicode.mapping /etc/nginx

sed -ie 's/SecRuleEngine DetectionOnly/SecRuleEngine On/g' /etc/nginx/modsecurity.conf


安裝 OWASP  rules

cd /opt

git clone https://github.com/SpiderLabs/owasp-modsecurity-crs.git

cd /opt/owasp-modsecurity-crs/rules

[root@centos7 rules]# pwd

/opt/owasp-modsecurity-crs/rules

cat *.conf > /etc/nginx/csr.conf

cp /opt/owasp-modsecurity-crs/crs-setup.conf.example /etc/nginx/crs-setup.conf

cd /etc/nginx 

cat modsecurity.conf crs-setup.conf csr.conf > rules.conf

 cp *.data  /etc/nginx/


======================

[root@centos7 conf.d]# cat /etc/nginx/nginx.conf 


user  nginx;

worker_processes  auto;


error_log  /var/log/nginx/error.log notice;

pid        /var/run/nginx.pid;



load_module modules/ngx_http_modsecurity_module.so;


events {

    worker_connections  1024;

}



http {

    include       /etc/nginx/mime.types;

    default_type  application/octet-stream;


    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '

                      '$status $body_bytes_sent "$http_referer" '

                      '"$http_user_agent" "$http_x_forwarded_for"';


    access_log  /var/log/nginx/access.log  main;


    sendfile        on;

    #tcp_nopush     on;


    keepalive_timeout  65;


    #gzip  on;


    include /etc/nginx/conf.d/*.conf;

}

[root@centos7 conf.d]# 



===========================


[root@centos7 conf.d]# cat default.conf 

server {

    listen       80;

    server_name  localhost;

    modsecurity on;

    modsecurity_rules_file /etc/nginx/modsecurity.conf;

    # or  use OWASP  rules modsecurity_rules_file /etc/nginx/rules.conf;

    #access_log  /var/log/nginx/host.access.log  main;


    location / {

        root   /usr/share/nginx/html;

        index  index.html index.htm;


        #modsecurity_rules_file rules.conf;


    }


    #error_page  404              /404.html;


    # redirect server error pages to the static page /50x.html

    #

    error_page   500 502 503 504  /50x.html;

    location = /50x.html {

        root   /usr/share/nginx/html;

    }


    # proxy the PHP scripts to Apache listening on 127.0.0.1:80

    #

    #location ~ \.php$ {

    #    proxy_pass   http://127.0.0.1;

    #}


    # pass the PHP scripts to FastCGI server listening on 127.0.0.1:9000

    #

    location ~ \.php$ {

        root           /usr/share/nginx/html;

        fastcgi_pass   127.0.0.1:9000;

        fastcgi_index  index.php;

        #fastcgi_param  SCRIPT_FILENAME  /scripts$fastcgi_script_name;

fastcgi_param  SCRIPT_FILENAME $document_root$fastcgi_script_name;  

        include        fastcgi_params;

    }


    # deny access to .htaccess files, if Apache's document root

    # concurs with nginx's one

    #

    #location ~ /\.ht {

    #    deny  all;

    #}

}


[root@centos7 conf.d]# 


vi /etc/php.ini


; cgi.fix_pathinfo provides *real* PATH_INFO/PATH_TRANSLATED support for CGI.  PHP's

; http://php.net/cgi.fix-pathinfo

;cgi.fix_pathinfo=1

cgi.fix_pathinfo=0




vi /etc/php-fpm.d/www.conf 



; RPM: apache user chosen to provide access to the same directories as httpd

user = nginx

; RPM: Keep a group allowed to write in log dir.

group = nginx




vi /etc/nginx/conf.d/default.conf


    location ~ \.php$ {

        root           /usr/share/nginx/html;

        fastcgi_pass   127.0.0.1:9000;

        fastcgi_index  index.php;

        #fastcgi_param  SCRIPT_FILENAME  /scripts$fastcgi_script_name;

        fastcgi_param  SCRIPT_FILENAME $document_root$fastcgi_script_name;

        include        fastcgi_params;

    }






2022年5月22日 星期日

Fortigate Log monitoring with Wazuh-manager

 參考自 https://github.com/wazuh/wazuh-kibana-app/issues/1884

 

架構說明

FTG-VM(.254)(syslog)

                                 --->Wazuh(.160)(Rsyslog)

                                                                      --->Wazuh-Agent(localfile)

                                                                                                             --->Wazuh(WEB UI)


==================================

vi /etc/rsyslog.conf 

# Provides UDP syslog reception

$ModLoad imudp

$UDPServerRun 514


if $fromhost-ip startswith '192.168.100.254' then /tmp/forti/syslog.log

& ~

==================================

vi /var/ossec/etc/ossec.conf 

  <localfile>

    <log_format>syslog</log_format>

    <location>/tmp/forti/syslog.log</location>

  </localfile>

===================================


Demo

















2022年5月21日 星期六

Graylog 4.2 on Rocky Linux

 安裝 SOP 參考自 https://tech.davidfield.co.uk/graylog-4-x-on-rhel-8-for-log-monitoring/

使用軟體如下:


[root@rocky8 ~]# cat /etc/redhat-release 

Rocky Linux release 8.6 (Green Obsidian)

[root@rocky8 ~]# 



[root@rocky8 ~]# rpm -qa | grep elasticsearch 

elasticsearch-oss-7.10.2-1.x86_64


[root@rocky8 ~]# rpm -qa | grep mongodb

mongodb-org-server-4.2.20-1.el8.x86_64

mongodb-org-shell-4.2.20-1.el8.x86_64

mongodb-org-4.2.20-1.el8.x86_64

mongodb-org-tools-4.2.20-1.el8.x86_64

mongodb-org-mongos-4.2.20-1.el8.x86_64


[root@rocky8 ~]# rpm -qa | grep graylog

graylog-integrations-plugins-4.3.0-8.noarch

graylog-server-4.3.0-8.noarch


===============================================

測試架構

FTG-VM(192.168.100.254)--->RockyLinux(.160)---->GrayLog 4.2

syslog UDP 514------------------>Rsyslog UDP 514---->Graylog Syslog 8888


 vi /etc/rsyslog.conf


# Provides UDP syslog reception

# for parameters see http://www.rsyslog.com/doc/imudp.html

module(load="imudp") # needs to be done just once

input(type="imudp" port="514")


*.* @192.168.100.160:8888;RSYSLOG_SyslogProtocol23Format


=====================

# You MUST set a secret to secure/pepper the stored user passwords here. Use at least 64 characters.

# Generate one by using for example: pwgen -N 1 -s 96

# ATTENTION: This value must be the same on all Graylog nodes in the cluster.

# Changing this value after installation will render all user sessions and encrypted values in the database invalid. (e.g. encrypted access tokens)

password_secret = XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX


# Create one by using for example: echo -n yourpassword | shasum -a 256

# and put the resulting hash value into the following line

root_password_sha2 = XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX


vi /etc/graylog/server/server.conf

root_timezone = Asia/Taipei

http_bind_address = 0.0.0.0:9000

password_secret =

root_password_sha2 =


DEMO





卡關處-待研究

Marketplace

https://github.com/seanthegeek/graylog-fortigate-cef

以前的 Lab http://xrcd2.blogspot.com/2016/07/graylog2.html


2022年5月3日 星期二

CentOS 7 XFS Disk Extending

 yum install cloud-utils-growpart gdisk -y



growpart /dev/sdb 1


xfs_growfs /bricks/brick1



===============================



[root@fsb ~]# yum install cloud-utils-growpart gdisk -y

.........

Installed:

  cloud-utils-growpart.noarch 0:0.29-5.el7                                                               


Complete!

[root@fsb ~]# fdisk -l 


Disk /dev/sda: 64.4 GB, 64424509440 bytes, 125829120 sectors

Units = sectors of 1 * 512 = 512 bytes

Sector size (logical/physical): 512 bytes / 512 bytes

I/O size (minimum/optimal): 512 bytes / 512 bytes

Disk label type: dos

Disk identifier: 0x000646df


   Device Boot      Start         End      Blocks   Id  System

/dev/sda1   *        2048     2099199     1048576   83  Linux

/dev/sda2         2099200   125583359    61742080   8e  Linux LVM


Disk /dev/sdb: 8589 MB, 8589934592 bytes, 16777216 sectors

Units = sectors of 1 * 512 = 512 bytes

Sector size (logical/physical): 512 bytes / 512 bytes

I/O size (minimum/optimal): 512 bytes / 512 bytes

Disk label type: dos

Disk identifier: 0x329dce28


   Device Boot      Start         End      Blocks   Id  System

/dev/sdb1            2048     8388607     4193280   83  Linux


Disk /dev/mapper/centos-root: 59.1 GB, 59055800320 bytes, 115343360 sectors

Units = sectors of 1 * 512 = 512 bytes

Sector size (logical/physical): 512 bytes / 512 bytes

I/O size (minimum/optimal): 512 bytes / 512 bytes



Disk /dev/mapper/centos-swap: 4160 MB, 4160749568 bytes, 8126464 sectors

Units = sectors of 1 * 512 = 512 bytes

Sector size (logical/physical): 512 bytes / 512 bytes

I/O size (minimum/optimal): 512 bytes / 512 bytes


[root@fsb ~]# fdisk -l /dev/sdb


Disk /dev/sdb: 8589 MB, 8589934592 bytes, 16777216 sectors

Units = sectors of 1 * 512 = 512 bytes

Sector size (logical/physical): 512 bytes / 512 bytes

I/O size (minimum/optimal): 512 bytes / 512 bytes

Disk label type: dos

Disk identifier: 0x329dce28


   Device Boot      Start         End      Blocks   Id  System

/dev/sdb1            2048     8388607     4193280   83  Linux

[root@fsb ~]# fdisk -l /dev/sdb1


Disk /dev/sdb1: 4293 MB, 4293918720 bytes, 8386560 sectors

Units = sectors of 1 * 512 = 512 bytes

Sector size (logical/physical): 512 bytes / 512 bytes

I/O size (minimum/optimal): 512 bytes / 512 bytes


[root@fsb ~]# lsblk /dev/sdb

NAME   MAJ:MIN RM SIZE RO TYPE MOUNTPOINT

sdb      8:16   0   8G  0 disk 

└─sdb1   8:17   0   4G  0 part /bricks/brick1


[root@fsb ~]# growpart /dev/sdb 1

CHANGED: partition=1 start=2048 old: size=8386560 end=8388608 new: size=16775135 end=16777183

[root@fsb ~]# df

Filesystem              1K-blocks    Used Available Use% Mounted on

devtmpfs                  1996168       0   1996168   0% /dev

tmpfs                     2013060       0   2013060   0% /dev/shm

tmpfs                     2013060   12284   2000776   1% /run

tmpfs                     2013060       0   2013060   0% /sys/fs/cgroup

/dev/mapper/centos-root  57643520 5305684  52337836  10% /

/dev/sdb1                 4183040   33232   4149808   1% /bricks/brick1

/dev/sda1                 1038336  299836    738500  29% /boot

fsb:/glustervol1.tcp      4183040   75064   4107976   2% /gluster/lock

tmpfs                      402612       0    402612   0% /run/user/0

[root@fsb ~]# df -h

Filesystem               Size  Used Avail Use% Mounted on

devtmpfs                 2.0G     0  2.0G   0% /dev

tmpfs                    2.0G     0  2.0G   0% /dev/shm

tmpfs                    2.0G   12M  2.0G   1% /run

tmpfs                    2.0G     0  2.0G   0% /sys/fs/cgroup

/dev/mapper/centos-root   55G  5.1G   50G  10% /

/dev/sdb1                4.0G   33M  4.0G   1% /bricks/brick1

/dev/sda1               1014M  293M  722M  29% /boot

fsb:/glustervol1.tcp     4.0G   74M  4.0G   2% /gluster/lock

tmpfs                    394M     0  394M   0% /run/user/0

[root@fsb ~]# xfs_growfs /bricks/brick1

meta-data=/dev/sdb1              isize=512    agcount=4, agsize=262080 blks

         =                       sectsz=512   attr=2, projid32bit=1

         =                       crc=1        finobt=0 spinodes=0

data     =                       bsize=4096   blocks=1048320, imaxpct=25

         =                       sunit=0      swidth=0 blks

naming   =version 2              bsize=4096   ascii-ci=0 ftype=1

log      =internal               bsize=4096   blocks=2560, version=2

         =                       sectsz=512   sunit=0 blks, lazy-count=1

realtime =none                   extsz=4096   blocks=0, rtextents=0

data blocks changed from 1048320 to 2096891

[root@fsb ~]# df -h

Filesystem               Size  Used Avail Use% Mounted on

devtmpfs                 2.0G     0  2.0G   0% /dev

tmpfs                    2.0G     0  2.0G   0% /dev/shm

tmpfs                    2.0G   12M  2.0G   1% /run

tmpfs                    2.0G     0  2.0G   0% /sys/fs/cgroup

/dev/mapper/centos-root   55G  5.1G   50G  10% /

/dev/sdb1                8.0G   33M  8.0G   1% /bricks/brick1

/dev/sda1               1014M  293M  722M  29% /boot

fsb:/glustervol1.tcp     8.0G  115M  7.9G   2% /gluster/lock

tmpfs                    394M     0  394M   0% /run/user/0

[root@fsb ~]# 


2022年4月29日 星期五

GlusterFS + Samba

 最近有人問我如何建 DRDB ,但我之前沒留筆記,加這之前透過 google 很快的就建好了 DRDB

我只記得當初那個專案的內容為二台 CentOS 透過 Keepalived 去做 HA,DRDB的重點則用在 PostgreSQL 的 HA.

簡單的說就是做出 PostgreSQL的 HA (Active/Standby),供 Zabbix 使用.

而我最近則是使用 MariaDB Cluster去做 MySQL 的 HA (Active/Active),供 Librenms 使用,

HA 的 VIP 一樣是用 Keepalived.

但他的需求不是用在 DB 上,而是想去做 Samba 的 HA,但我想到的一個問題是,如果透過 DRBD 去做 HA 的話,會受限 DRDB 的機制,必然是一個  A/S 的架構,另一個問題是它是用  Block level disk replication,平時無法驗證檔案是否真的有複寫成功,除非 Remount replicated disk才可去驗證,或做 HA 切換演練去驗證.另一個問題則是 Active GG 後, A/S 的角色互換問題與重建.

假設 Active/Standby 的機器沒在監控它的 DRDB 機制是否運作正常?會不會有一天, HA 切換後才發現 Samba 的東西短少了!?  


經過 goolge 後,發現 GlusterFS +  Samba (CTDB) 這個組合比 DRDB + Samba 還要優一些,而且是 A/A 的架構

Byte-level file replication VS Block-level disk replication 我個人是比較推崇 GlusterFS 的.


以下為個人做的建制小筆記.


架構說明:


(1) fsa 192.168.100.111

(2) fsb 192.168.100.112

(3) vip 192.168.100.100


CentOS 7.9 + Clusterfs 9.5



參考 URL

https://www.server-world.info/en/note?os=CentOS_7&p=glusterfs9&f=1

https://www.server-world.info/en/note?os=CentOS_7&p=glusterfs9&f=5

https://wiki.samba.org/index.php/CTDB_and_Clustered_Samba


=========================================================

vi /etc/hosts

yum install centos-release-gluster

yum install centos-release-gluster glusterfs-server centos-release-samba samba samba-vfs-glusterfs

yum -y install centos-release-samba411

yum --enablerepo=centos-samba411 -y install samba ctdb samba-vfs-glusterfs


fdisk -l

fdisk /dev/sdb

mkfs.xfs /dev/sdb1

mkdir -p /bricks/brick1

mount /dev/sdb1 /bricks/brick1

vi /etc/fstab 

systemctl enable glusterd.service

systemctl start glusterd.service

gluster peer probe fsb

gluster peer status

gluster volume create glustervol1 replica 2 transport tcp fsa:/bricks/brick1/brick fsb:/bricks/brick1/brick

gluster volume start glustervol1

gluster volume info all

gluster volume stop glustervol1

gluster volume set glustervol1 user.smb enable

gluster volume set glustervol1 performance.write-behind off

gluster volume set glustervol1 group samba

vi /var/lib/glusterd/hooks/1/start/post/S29CTDBsetup.sh

vi /var/lib/glusterd/hooks/1/stop/pre/S29CTDB-teardown.sh

gluster volume start glustervol1

df -h /gluster/lock

tail -1 /etc/fstab

vi /etc/ctdb/nodes

ip add

vi /etc/ctdb/public_addresses

systemctl enable --now ctdb

ctdb status

ctdb ip

mount -t glusterfs fsa:/glustervol1 /mnt

mkdir /mnt/smbshare

groupadd smbgroup

chgrp smbgroup /mnt/smbshare

chmod 770 /mnt/smbshare

umount /mnt

vi /etc/samba/smb.conf

systemctl enable --now smb

useradd cent

smbpasswd -a cent

usermod -aG smbgroup cent

systemctl restart smb








Gluster Replication + CTDB







Architecture






2022年3月11日 星期五

Zabbix 6.0 試裝

 安裝 postgresql server 14 


https://www.postgresql.org/download/linux/redhat/



# Install the repository RPM:


sudo dnf install -y https://download.postgresql.org/pub/repos/yum/reporpms/EL-8-x86_64/pgdg-redhat-repo-latest.noarch.rpm


# Disable the built-in PostgreSQL module:

sudo dnf -qy module disable postgresql


# Install PostgreSQL:

sudo dnf install -y postgresql14-server


# Optionally initialize the database and enable automatic start:

sudo /usr/pgsql-14/bin/postgresql-14-setup initdb

sudo systemctl enable postgresql-14

sudo systemctl start postgresql-14



安裝 zabbix 6.0 + Rocky Linux release 8.5 (Green Obsidian) + Apache + Postgresq 14 + Timescaledb-2



https://www.zabbix.com/download?zabbix=6.0&os_distribution=centos&os_version=8&db=postgresql&ws=apache




安裝 timescaledb-2-postgresql-14


export SPLUNK_HOME=/opt/splunk


PATH=$PATH:/usr/pgsql-14/bin/

export PATH

PGDATA=/var/lib/pgsql/14/data

export PGDATA



Timescale_timescaledb 參考文件及設定


https://docs.timescale.com/install/latest/self-hosted/installation-redhat/#install-self-hosted-timescaledb-on-red-hat-based-systems



Create the Timescale repository:


tee /etc/yum.repos.d/timescale_timescaledb.repo <<EOL

[timescale_timescaledb]

name=timescale_timescaledb

baseurl=https://packagecloud.io/timescale/timescaledb/el/$(rpm -E %{rhel})/\$basearch

repo_gpgcheck=1

gpgcheck=0

enabled=1

gpgkey=https://packagecloud.io/timescale/timescaledb/gpgkey

sslverify=1

sslcacert=/etc/pki/tls/certs/ca-bundle.crt

metadata_expire=300

EOL



[root@rocky ~]# timescaledb-tune

Using postgresql.conf at this path:

/var/lib/pgsql/14/data/postgresql.conf


Is this correct? [(y)es/(n)o]: y

Writing backup to:

/tmp/timescaledb_tune.backup202203121248


shared_preload_libraries needs to be updated

Current:

#shared_preload_libraries = ''

Recommended:

shared_preload_libraries = 'timescaledb'

Is this okay? [(y)es/(n)o]: y

success: shared_preload_libraries will be updated


Tune memory/parallelism/WAL and other settings? [(y)es/(n)o]: y

Recommendations based on 7.75 GB of available memory and 4 CPUs for PostgreSQL 14


Memory settings recommendations

Current:

shared_buffers = 128MB

#effective_cache_size = 4GB

#maintenance_work_mem = 64MB

#work_mem = 4MB

Recommended:

shared_buffers = 1983MB

effective_cache_size = 5950MB

maintenance_work_mem = 1015528kB

work_mem = 5077kB

Is this okay? [(y)es/(s)kip/(q)uit]: y

success: memory settings will be updated


Parallelism settings recommendations

Current:

missing: timescaledb.max_background_workers

#max_worker_processes = 8

#max_parallel_workers_per_gather = 2

#max_parallel_workers = 8

Recommended:

timescaledb.max_background_workers = 8

max_worker_processes = 15

max_parallel_workers_per_gather = 2

max_parallel_workers = 4

Is this okay? [(y)es/(s)kip/(q)uit]: y

success: parallelism settings will be updated


WAL settings recommendations

Current:

#wal_buffers = -1

min_wal_size = 80MB

Recommended:

wal_buffers = 16MB

min_wal_size = 512MB

Is this okay? [(y)es/(s)kip/(q)uit]: y

success: WAL settings will be updated


Miscellaneous settings recommendations

Current:

#default_statistics_target = 100

#random_page_cost = 4.0

#checkpoint_completion_target = 0.9

#max_locks_per_transaction = 64

#autovacuum_max_workers = 3

#autovacuum_naptime = 1min

#effective_io_concurrency = 1

Recommended:

default_statistics_target = 500

random_page_cost = 1.1

checkpoint_completion_target = 0.9

max_locks_per_transaction = 64

autovacuum_max_workers = 10

autovacuum_naptime = 10

effective_io_concurrency = 256

Is this okay? [(y)es/(s)kip/(q)uit]: y

success: miscellaneous settings will be updated

Saving changes to: /var/lib/pgsql/14/data/postgresql.conf

[root@rocky ~]#


Zabbix 啟用  Timescaledb


https://www.zabbix.com/documentation/current/en/manual/appendix/install/timescaledb


[root@rocky tmp]# echo "CREATE EXTENSION IF NOT EXISTS timescaledb CASCADE;" | sudo -u postgres psql zabbix

WARNING:

WELCOME TO

 _____ _                               _     ____________

|_   _(_)                             | |    |  _  \ ___ \

  | |  _ _ __ ___   ___  ___  ___ __ _| | ___| | | | |_/ /

  | | | |  _ ` _ \ / _ \/ __|/ __/ _` | |/ _ \ | | | ___ \

  | | | | | | | | |  __/\__ \ (_| (_| | |  __/ |/ /| |_/ /

  |_| |_|_| |_| |_|\___||___/\___\__,_|_|\___|___/ \____/

               Running version 2.6.0

For more information on TimescaleDB, please visit the following links:


 1. Getting started: https://docs.timescale.com/timescaledb/latest/getting-started

 2. API reference documentation: https://docs.timescale.com/api/latest

 3. How TimescaleDB is designed: https://docs.timescale.com/timescaledb/latest/overview/core-concepts


Note: TimescaleDB collects anonymous reports to better understand and assist our users.

For more information and how to disable, please see our docs https://docs.timescale.com/timescaledb/latest/how-to-guides/configuration/telemetry.


CREATE EXTENSION


=====================


匯入 Zabbix  schema 等資料



[root@rocky tmp]# zcat /usr/share/doc/zabbix-sql-scripts/postgresql/server.sql.gz | sudo -u zabbix psql zabbix


[root@rocky tmp]# cat /usr/share/doc/zabbix-sql-scripts/postgresql/timescaledb.sql | sudo -u zabbix psql zabbix


先前的 LAB https://xrcd2.blogspot.com/search/label/Zabbix






2021年9月26日 星期日

LibreNMS平均CPU使用率監控功能

  最近有一個需求是要去CPU的監控,但使用的方式為 

processors.processor_usage >= processor_perc_warn

這個方式去監控是依據每一顆CPU去做監控的.如果該主機有 8 顆CPU,

其中有 1 顆 超過告警值 即會發出告警,但其它 7 顆可能沒那麼重的 loading .

這種方式的監控似乎不是那麼合理.理想的情境應是依 這 8 顆的平均值去做告警才是.

在 官方 https://docs.librenms.org/Alerting/Rules/ 的 Advanced 處,

有看到一個 SQL 語法.


SELECT *,AVG(processors.processor_usage) as cpu_avg FROM devices,processors WHERE (devices.device_id = ? AND devices.device_id = processors.device_id) AND (devices.status = 1 && (devices.disabled = 0 && devices.ignore = 0)) = 1 HAVING AVG(processors.processor_usage) > 10


後來試一下真的可以用.

壓測方式

[root@librenms CPULoadGenerator]# python3.6 -m cpu_load_generator -l 0.8 -d 600 -c -1

使用工具

https://pypi.org/project/cpu-load-generator/


Project description


CPU Load Generator


This package allows to generate a fixed CPU load for a finite time period. 

The script takes in input the desired CPU load, the duration of the experiment 

and the CPU core or all cores on which the load has to be generated.


Python versions This master branch refers to Python versions 3.x


Dependencies Installment of psutil is required 

(it is already set as dependency in the setup.py)


Install psutil:


pip install psutil

Other dependencies for unit tests and linters:


pip install mock pytest flake8 tox

Usage To generate 20% of load on core 0 for 20 seconds run:


python -m cpu_load_generator -l 0.2 -d 20 -c 0

To generate 50% of load on all logical cores for 20 seconds run:


python -m cpu_load_generator -l 0.5 -d 20 -c -1

There is an option to run CPU load based on profile file. An exemplary profile 

is under load_profiles/default_profile.json. In order to run generator based on

a profile issue the following command:


python -m cpu_load_generator -p <path_to_profile_json>

To use the package features from python code:


Install the package from PyPi by issuing the following command:


python -m pip instal cpu-load-generator

To use its features from your code:


from cpu_load_generator import load_single_core, load_all_cores, from_profile


load_single_core(core_num=0, duration_s=20, target_load=0.4)  # generate load on single core (0)

load_all_cores(duration_s=30, target_load=0.2)  # generates load on all cores

from_profile(path_to_profile_json=r"c:\profiles\profile1.json")


DEMO














2021年9月25日 星期六

LibreNMS 外掛 Service 監控功能

源自 Nagios Plugins - Services 

https://docs.librenms.org/Extensions/Services/

Demo




















Setting

======================================================


Service


{{ $alert->title }}

@if ($alert->faults)

@foreach ($alert->faults as $key => $value) {{ $value['service_desc'] }} - {{ $value['service_type'] }}

{{ $value['service_message'] }}

@endforeach

@endif


Error=========

Service: {{ $value['service_name'] }} {{ $value['service_type'] }}

=============

=========================================================

Other

SELECT * FROM devices,services WHERE (devices.device_id = ? AND devices.device_id = services.device_id) AND services.service_status != 0 AND (devices.status = 1 && (devices.disabled = 0 && devices.ignore = 0)) = 1