Linux Centos7.4--apache日志分割，日志管理分析

Apache日志分割

日志分割有两种方式，第一个是apache自带的rotatelogs分割工具实现，第二个是第三方的工具cronnolog分割

apache自带的分割工具rotatelogs

[root@client ~]# yum install bind httpd -y[root@client ~]# cd /usr/sbin/[root@client sbin]# ls rotat*rotatelogs//安装好apache才能在系统能使用的命令底下看到

配置apache主配置文件，开启服务

[root@client sbin]# vim /etc/httpd/conf/httpd.conf Listen 192.168.136.128:80   //监听你本地的地址#Listen 80  //把ipv6的监听端口注释掉#If your host doesn't have a registered DNS name, enter its IP address here.ServerName www.kgc.com:80  //把原本的改成你定义的域名，并开启[root@client httpd]# systemctl stop firewalld.service [root@client httpd]# setenforce 0[root@client httpd]# systemctl start httpd[root@client httpd]# ls  //日志文件在服务开启的时候才有access_log  error_log[root@client httpd]# cat access_log   //访问日志是空的[root@client httpd]# cat error_log    //错误日志文件，这些是PID的进程使用，不代表又一些错误[Wed Oct 23 21:18:24.917418 2019] [core:notice] [pid 4429] SELinux policy enabled; httpd running as context system_u:system_r:httpd_t:s0[Wed Oct 23 21:18:24.918499 2019] [suexec:notice] [pid 4429] AH01232: suEXEC mechanism enabled (wrapper: /usr/sbin/suexec)[Wed Oct 23 21:18:24.938959 2019] [lbmethod_heartbeat:notice] [pid 4429] AH02282: No slotmem from mod_heartmonitor[Wed Oct 23 21:18:24.941240 2019] [mpm_prefork:notice] [pid 4429] AH00163: Apache/2.4.6 (CentOS) configured -- resuming normal operations[Wed Oct 23 21:18:24.941268 2019] [core:notice] [pid 4429] AH00094: Command line: '/usr/sbin/httpd -D FOREGROUND'

去客户段去测试一下

回到服务器当中看一下访问日志

如果来访者越来越多，日志就越来越多，我们人看不过来，我们就需要进行分割

[root@client httpd]# cat access_log 192.168.136.134 - - [23/Oct/2019:21:24:55 +0800] "GET /favicon.ico HTTP/1.1" 404 209 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/42.0.2311.135 Safari/537.36 Edge/12.10240"192.168.136.134 - - [23/Oct/2019:21:24:55 +0800] "GET / HTTP/1.1" 403 4897 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/42.0.2311.135 Safari/537.36 Edge/12.10240"192.168.136.134 - - [23/Oct/2019:21:24:56 +0800] "GET /noindex/css/bootstrap.min.css HTTP/1.1" 200 19341 "http://192.168.136.128/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/42.0.2311.135 Safari/537.36 Edge/12.10240"192.168.136.134 - - [23/Oct/2019:21:24:56 +0800] "GET /images/apache_pb.gif HTTP/1.1" 200 2326 "http://192.168.136.128/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/42.0.2311.135 Safari/537.36 Edge/12.10240"192.168.136.134 - - [23/Oct/2019:21:24:56 +0800] "GET /noindex/css/open-sans.cs

日志分割配置

[root@client httpd]# vim /etc/httpd/conf/httpd.conf #ErrorLog "logs/error_log"  //我们把原来的注释掉，线网上的技巧，万一错了可以恢复。ErrorLog "| /usr/sbin/rotatelogs -l logs/www.kgc.com.error_%Y%m%dlog 86400" //找到这一行，/输入管道符号"|" 跟你系统apache命令的绝对路径，起个名字www.kgc.vom，%Y%m%d固定格式代表年月日，86400代表一天的时间86400秒    CustomLog "| /usr/sbin/rotatelogs -l logs/www.kgc.com.access_%Y%m%dlog 86400" combined        //这边也是一样，找到这一行

验证日志分割

[root@client httpd]# systemctl stop httpd
[root@client httpd]# systemctl start httpd
[root@client httpd]# ls
access_log error_log www.kgc.com.error_20191023log //我们当天的日志就被分割出来了
[root@client httpd]# date -s 10/24
2019年 10月 24日 星期四 00:00:00 CST
[root@client httpd]# systemctl stop httpd
[root@client httpd]# systemctl start httpd
[root@client httpd]# ls
access_log error_log www.kgc.com.error_20191023log www.kgc.com.error_20191024log //我们改了一下时间，日志分割出来了

第三方工具日志分割，我这有包，要的私信我

[root@localhost httpd]# smbclient -L //192.168.100.3  //记得你的宿主机Vmnet1网卡要设置成192.168.100.3Enter SAMBA\root's password: OS=[Windows 10 Enterprise 17763] Server=[Windows 10 Enterprise 6.3]    Sharename       Type      Comment    ---------       ----      -------    ADMIN$          Disk      远程管理    C$              Disk      默认共享    D$              Disk      默认共享    E$              Disk      默认共享    F$              Disk      默认共享    G$              Disk      默认共享    IPC$            IPC       远程 IPC    LAMP            Disk          LAMP-C7         Disk          share           Disk          Users           Disk      Connection to 192.168.100.3 failed (Error NT_STATUS_RESOURCE_NAME_NOT_FOUND)NetBIOS over TCP disabled -- no workgroup available[root@localhost httpd]# cd ~[root@localhost ~]# mkdir /abc  //创建挂载点[root@localhost ~]# mount.cifs //192.168.100.3/LAMP-C7 /abc/  //把宿主机的文件挂载到我们的挂载点中Password for root@//192.168.100.3/LAMP-C7:  [root@localhost ~]# cd /abc/[root@localhost abc]# lsapr-1.6.2.tar.gz                  Discuz_X2.5_SC_UTF8.zip  mysql-5.6.26.tar.gzapr-util-1.6.0.tar.gz             fiddler.exe              php-5.6.11.tar.bz2awstats-7.6.tar.gz                httpd-2.4.29.tar.bz2cronolog-1.6.2-14.el7.x86_64.rpm  LAMP-php5.6.txt[root@localhost abc]# rpm -ivh cronolog-1.6.2-14.el7.x86_64.rpm //安装这个第三方分割工具包警告：cronolog-1.6.2-14.el7.x86_64.rpm: 头V3 RSA/SHA256 Signature, 密钥 ID 352c64e5: NOKEY准备中...                          ################################# [100%]正在升级/安装...   1:cronolog-1.6.2-14.el7            ################################# [100%][root@localhost abc]# cd /usr/sbin/[root@localhost sbin]# ls cronolog*  //到系统命令中查看有没有这个命令cronolog

第三方工具日志分割

[root@localhost sbin]# vim /etc/httpd/conf/httpd.conf ErrorLog "| /usr/sbin/cronolog logs/www.kgc.comerror_%Y%m%dlog"  //注意不要加-l 也没有86400CustomLog "| /usr/sbin/ cronolog logs/www.kgc.comaccess_%Y%m%dlog" combined[root@localhost sbin]# systemctl stop httpd[root@localhost sbin]# systemctl start httpd[root@localhost sbin]# ls /var/log/httpd/access_log  error_log  www.kgc.comerror_20191023log[root@localhost sbin]# date -s 10/242019年 10月 24日 星期四 00:00:00 CST[root@localhost sbin]# ls /var/log/httpd/access_log  www.kgc.comerror_20191023logerror_log   www.kgc.comerror_20191024log

日志分析

我们换一台虚拟机做，配置DNS主配置文件

[root@localhost ~]# yum install bind httpd -y //安装DNS,Apache软件包[root@localhost ~]# vim /etc/named.conf options {        listen-on port 53 { any; };  //改成any        listen-on-v6 port 53 { ::1; };        directory       "/var/named";        dump-file       "/var/named/data/cache_dump.db";        statistics-file "/var/named/data/named_stats.txt";        memstatistics-file "/var/named/data/named_mem_stats.txt";        recursing-file  "/var/named/data/named.recursing";        secroots-file   "/var/named/data/named.secroots";        allow-query     { any; };  //改成any

配置DNS区域配置文件

[root@localhost ~]# vim /etc/named.rfc1912.zones
zone "kgc.com" IN { //定义域名
type master;
file "kgc.com.zone"; //定义区域数据配置文件
allow-update { none; };
};
zone "1.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0

配置DNS区域数据配置文件

[root@localhost ~]# cd /var/named/[root@localhost named]# cp -p named.localhost kgc.com.zone[root@localhost named]# vim kgc.com.zone $TTL 1D@       IN SOA  @ rname.invalid. (                                        0       ; serial                                        1D      ; refresh                                        1H      ; retry                                        1W      ; expire                                        3H )    ; minimum        NS      @        A       127.0.0.1www IN  A       192.168.136.135  //加入你本地地址

配置Apache主配置文件

[root@localhost named]# vim /etc/httpd/conf/httpd.conf Listen 192.168.136.135:80#Listen 80ServerName www.kgc.com:80[root@localhost named]# systemctl stop firewalld.service [root@localhost named]# setenforce 0[root@localhost named]# systemctl start httpd

去客户端测试一下

日志分析又有工具包，需要的私信我，

[root@localhost httpd]# mkdir /abc  //创建挂载点[root@localhost httpd]# mount.cifs //192.168.100.3/LAMP-C7 /abc //挂载到abcPassword for root@//192.168.100.3/LAMP-C7:  [root@localhost httpd]# cd /abc/[root@localhost abc]# lsapr-1.6.2.tar.gz                  Discuz_X2.5_SC_UTF8.zip  mysql-5.6.26.tar.gzapr-util-1.6.0.tar.gz             fiddler.exe              php-5.6.11.tar.bz2awstats-7.6.tar.gz      //这个包就是我们的日志分析工具包          httpd-2.4.29.tar.bz2cronolog-1.6.2-14.el7.x86_64.rpm  LAMP-php5.6.txt

解压，移动，利用我们刚才创建的这个脚本awstats，去配置apache,让它去加载awstats的日志分析于统计模块

[root@localhost abc]# tar zxvf awstats-7.6.tar.gz -C /opt/  //jie'ya解压到OPT下awstats-7.6/awstats-7.6/tools/awstats-7.6/tools/awstats_buildstaticpages.plawstats-7.6/tools/awstats_updateall.pl[root@localhost abc]# cd /opt/[root@localhost opt]# lsawstats-7.6  rh[root@localhost opt]# mv awstats-7.6/ /usr/local/awstats //把这个数据包移动到/usr/local下起个名字awstats[root@localhost opt]# lsrh[root@localhost opt]# cd /usr/local/  //查看[root@localhost local]# lsawstats  bin  etc  games  include  lib  lib64  libexec  sbin  share  src

在工具中做apache的路径和域名

[root@localhost local]# cd awstats/[root@localhost awstats]# lsdocs  README.md  tools  wwwroot[root@localhost awstats]# cd tools/[root@localhost tools]# lsawstats_buildstaticpages.pl  dolibarr            maillogconvert.pl   xsltawstats_configure.pl         geoip_generator.pl  nginxawstats_exportlib.pl         httpd_conf          urlaliasbuilder.plawstats_updateall.pl         logresolvemerge.pl  webmin[root@localhost tools]# ./awstats_configure.pl > /etc/httpd/conf/httpd.conf  //写上apache路径file (required if first install) [y/N] ? y  //要不要建立一个新的apache文件> www.kgc.com  //输入你的域名-----> Define config file pathIn which directory do you plan to store your config file(s) ?Default: /etc/awstats  //跟你这个域名相关的配置文件，会生成到这个路径下Directory path to store config file(s) (Enter for default):>   //直接回车Press ENTER to continue...   //直接回车> http://localhost/awstats/awstats.pl?config=www.kgc.com  //这个路径就是它给你的日志分析网页Press ENTER to finish...  //直接回车

检查，配置apache有没有加载这些模块

[root@localhost tools]# cd /etc/httpd/conf[root@localhost conf]# vim httpd.conf Alias /awstatsclasses "/usr/local/awstats/wwwroot/classes/"Alias /awstatscss "/usr/local/awstats/wwwroot/css/"Alias /awstatsicons "/usr/local/awstats/wwwroot/icon/"ScriptAlias /awstats/ "/usr/local/awstats/wwwroot/cgi-bin/"#This is to permit URL access to scripts/files in AWStats directory.  //目录站点    Options None    AllowOverride None    //相关的权限控制   # Order allow,deny   //把这两行注释掉   # Allow from all     Require all granted   //加入让所有人可以访问

分析出来的数据要放在Apache访问日志中

[root@localhost etc]# cd /etc/awstats/
[root@localhost awstats]# ls
awstats.www.kgc.com.conf //刚才生成的文件就是这个
[root@localhost awstats]# ls /var/log/httpd/ //
access_log error_log
[root@localhost awstats]# vim awstats.www.kgc.com.conf
LogFile="/var/log/httpd/access_log" //把原来的改成access_log，分析这个日志
DirData="/var/lib/awstats" //这个不需要改，我们分析的数据放这个文件下

创建awstats目录才能让系统识别把分析数据放进去

[root@localhost awstats]# cd /var/lib[root@localhost lib]# ls awstats*ls: 无法访问awstats*: 没有那个文件或目录[root@localhost lib]# mkdir awstats[root@localhost lib]# lsAccountsService  dav       hyperv      net-snmp        rpcbind         tpmalsa             dbus      initramfs   NetworkManager  rpm             tunedalternatives     dhclient  ipa-client  nfs             rpm-state       udisks2authconfig       dnsmasq   iscsi       ntp             rsyslog         upowerawstats     //这个就是     flatpak   libvirt    [root@localhost lib]# systemctl restart httpd  //重启服务

去客户端测试一下

这只是一个分析页面

输入这个网址http://www.kgc.com/awstats/awstats.pl?config=www.kgc.com

更新数据才能统计

[root@localhost lib]# cd /usr/local/awstats/[root@localhost awstats]# lsdocs  README.md  tools  wwwroot[root@localhost awstats]# cd tools/[root@localhost tools]# lsawstats_buildstaticpages.pl  dolibarr            maillogconvert.pl   xsltawstats_configure.pl         geoip_generator.pl  nginxawstats_exportlib.pl         httpd_conf          urlaliasbuilder.plawstats_updateall.pl         logresolvemerge.pl  webmin[root@localhost tools]# ./awstats_updateall.pl now //直接执行就行了，记得加上now最新的Running '"/usr/local/awstats/wwwroot/cgi-bin/awstats.pl" -update -config=www.kgc.com -configdir="/etc/awstats"' to update config www.kgc.comCreate/Update database for config "/etc/awstats/awstats.www.kgc.com.conf" by AWStats version 7.6 (build 20161204)From data in log file "/var/log/httpd/access_log"...Phase 1 : First bypass old records, searching new record...Searching new records from beginning of log file...Phase 2 : Now process new records (Flush history on disk after 20000 hosts)...Jumped lines in file: 0Parsed lines in file: 485 Found 0 dropped records, Found 0 comments, Found 0 blank records, Found 1 corrupted records, Found 0 old records, Found 484 new qualified records.

回到客户端测试一下能不能统计数据

所有的数据采集我们每次都要去执行那个脚本，我们这里做个周期性计划性任务让它自动执行

[root@localhost tools]# crontab -e*/5 * * * * /usr/local/awstats/tools/awstats_updateall.pl now//每月每周每天每小时每5分种去执行这个脚本[root@localhost tools]# systemctl start crond //开启

优化网页地址

[root@localhost tools]# cd /var/www//html/您在 /var/spool/mail/root 中有邮件[root@localhost html]# ls[root@localhost html]# vim aws.html          

去客户端测试一下优化过的网址

以上就是我们全部的内容了，谢谢大家收看