GoAccess를 설치하고 로그를 분석하는 방법

GoAccess를 설치하고 로그를 분석하는 방법

goaccess : Real-time web log analyzer and interactive viewer

1. goaccess 설치

$ yum install -y goaccess

$ goaccess --version
GoAccess - 1.5.4.
For more details visit: https://goaccess.io/
Copyright (C) 2009-2020 by Gerardo Orellana

Build configure arguments:
  --enable-debug
  --enable-utf8
  --enable-geoip=legacy
  --with-getline
  --with-openssl

2. 로그 파일 분석

• 아파치 로그

$ pwd
/var/log/httpd

$ ls -l
total 176
-rw-r--r-- 1 root root 100402 Feb 11 11:42 access_log
-rw-r--r-- 1 root root  75748 Feb 11 08:49 error_log

• goaccess 로그 파일 분석

goaccess /var/log/httpd/access_log --log-format=COMBINED

a1

• 아파치 웹 루트 디렉토리 : /var/www/html

goaccess -f /var/log/httpd/access_log --log-format=COMBINED -a -o /var/www/html/index.html

goaccess /var/log/httpd/access_log --log-format=COMBINED -a -o /var/www/html/report.html

goaccess 실시간 로그 분석

• 실시간 분석 대화식

goaccess -f /var/log/httpd/access_log --log-format=COMBINED -a -o /var/www/html/report.html --real-time-html
WebSocket server ready to accept new client connections

• 실시간 분석 daemonize(--real-time-html enabled)

goaccess -f /var/log/httpd/access_log --log-format=COMBINED -a -o /var/www/html/report.html --daemonize

$ ps -ef | grep -v grep | grep goaccess
root      165509  152463 99 12:25 pts/0    00:03:10 goaccess -f \
/var/log/httpd/access_log \
--log-format=COMBINED -a \
-o /var/www/html/report.html \
--daemonize

728x90

Log & Date Format Options

$ goaccess --help

GoAccess - 1.3

Usage: goaccess [filename] [ options ... ] [-c][-M][-H][-S][-q][-d][...]
The following options can also be supplied to the command:

Log & Date Format Options

  --date-format=<dateformat>      - Specify log date format. e.g., %d/%b/%Y
  --log-format=<logformat>        - Specify log format. Inner quotes need to be
                                    escaped, or use single quotes.
  --time-format=<timeformat>      - Specify log time format. e.g., %H:%M:%S

User Interface Options

  -c --config-dialog              - Prompt log/date/time configuration window.
  -i --hl-header                  - Color highlight active panel.
  -m --with-mouse                 - Enable mouse support on main dashboard.
  --color=<fg:bg[attrs, PANEL]>   - Specify custom colors. See manpage for more
                                    details and options.
  --color-scheme=<1|2|3>          - Schemes: 1 => Grey, 2 => Green, 3 => Monokai.
  --html-custom-css=<path.css>    - Specify a custom CSS file in the HTML report.
  --html-custom-js=<path.js>      - Specify a custom JS file in the HTML report.
  --html-prefs=<json_obj>         - Set default HTML report preferences.
  --html-report-title=<title>     - Set HTML report page title and header.
  --json-pretty-print             - Format JSON output w/ tabs & newlines.
  --max-items                     - Maximum number of items to show per panel.
                                    See man page for limits.
  --no-color                      - Disable colored output.
  --no-column-names               - Don't write column names in term output.
  --no-csv-summary                - Disable summary metrics on the CSV output.
  --no-html-last-updated          - Hide HTML last updated field.
  --no-parsing-spinner            - Disable progress metrics and parsing spinner.
  --no-progress                   - Disable progress metrics.
  --no-tab-scroll                 - Disable scrolling through panels on TAB.

Server Options

  --addr=<addr>                   - Specify IP address to bind server to.
  --daemonize                     - Run as daemon (if --real-time-html enabled).
  --fifo-in=<path>                - Path to read named pipe (FIFO).
  --fifo-out=<path>               - Path to write named pipe (FIFO).
  --origin=<addr>                 - Ensure clients send the specified origin header
                                    upon the WebSocket handshake.
  --pid-file=<path>               - Write PID to a file when --daemonize is used.
  --port=<port>                   - Specify the port to use.
  --real-time-html                - Enable real-time HTML output.
  --ssl-cert=<cert.crt>           - Path to TLS/SSL certificate.
  --ssl-key=<priv.key>            - Path to TLS/SSL private key.
  --ws-url=<url>                  - URL to which the WebSocket server responds.

File Options

  -                               - The log file to parse is read from stdin.
  -f --log-file=<filename>        - Path to input log file.
  -S --log-size=<number>          - Specify the log size, useful when piping in logs.
  -l --debug-file=<filename>      - Send all debug messages to the specified
                                    file.
  -p --config-file=<filename>     - Custom configuration file.
  --invalid-requests=<filename>   - Log invalid requests to the specified file.
  --no-global-config              - Don't load global configuration file.

Parse Options

  -a --agent-list                 - Enable a list of user-agents by host.
  -b --browsers-file=<path>       - Use additional custom list of browsers.
  -d --with-output-resolver       - Enable IP resolver on HTML|JSON output.
  -e --exclude-ip=<IP>            - Exclude one or multiple IPv4/6. Allows IP
                                    ranges e.g. 192.168.0.1-192.168.0.10
  -H --http-protocol=<yes|no>     - Set/unset HTTP request protocol if found.
  -M --http-method=<yes|no>       - Set/unset HTTP request method if found.
  -o --output=file.html|json|csv  - Output either an HTML, JSON or a CSV file.
  -q --no-query-string            - Ignore request's query string. Removing the
                                    query string can greatly decrease memory
                                    consumption.
  -r --no-term-resolver           - Disable IP resolver on terminal output.
  --444-as-404                    - Treat non-standard status code 444 as 404.
  --4xx-to-unique-count           - Add 4xx client errors to the unique visitors
                                    count.
  --anonymize-ip                  - Anonymize IP addresses before outputting to report.
  --all-static-files              - Include static files with a query string.
  --crawlers-only                 - Parse and display only crawlers.
  --date-spec=<date|hr>           - Date specificity. Possible values: `date`
                                    (default), or `hr`.
  --double-decode                 - Decode double-encoded values.
  --enable-panel=<PANEL>          - Enable parsing/displaying the given panel.
  --hide-referer=<NEEDLE>         - Hide a referer but still count it. Wild cards
                                    are allowed. i.e., *.bing.com
  --hour-spec=<hr|min>            - Hour specificity. Possible values: `hr`
                                    (default), or `min` (tenth of a min).
  --ignore-crawlers               - Ignore crawlers.
  --ignore-panel=<PANEL>          - Ignore parsing/displaying the given panel.
  --ignore-referer=<NEEDLE>       - Ignore a referer from being counted. Wild cards
                                    are allowed. i.e., *.bing.com
  --ignore-statics=<req|panel>    - Ignore static requests.
                                    req => Ignore from valid requests.
                                    panel => Ignore from valid requests and panels.
  --ignore-status=<CODE>          - Ignore parsing the given status code.
  --num-tests=<number>            - Number of lines to test. >= 0 (10 default)
  --process-and-exit              - Parse log and exit without outputting data.
  --real-os                       - Display real OS names. e.g, Windows XP, Snow
                                    Leopard.
  --sort-panel=PANEL,METRIC,ORDER - Sort panel on initial load. For example:
                                    --sort-panel=VISITORS,BY_HITS,ASC. See
                                    manpage for a list of panels/fields.
  --static-file=<extension>       - Add static file extension. e.g.: .mp3.
                                    Extensions are case sensitive.

GeoIP Options

  -g --std-geoip                  - Standard GeoIP database for less memory
                                    consumption.
  --geoip-database=<path>         - Specify path to GeoIP database file. i.e.,
                                    GeoLiteCity.dat, GeoIPv6.dat ...

Other Options

  -h --help                       - This help.
  -V --version                    - Display version information and exit.
  -s --storage                    - Display current storage method. e.g., B+
                                    Tree, Hash.
  --dcf                           - Display the path of the default config
                                    file when `-p` is not used.

Examples can be found by running `man goaccess`.

For more details visit: http://goaccess.io
GoAccess Copyright (C) 2009-2017 by Gerardo Orellana

Access logs formats

로그 포맷	로그 표시
COMBINED (NCSA Combined Log Format)	%h %^[%d:%t %^] "%r" %s %b "%R" "%u"
VCOMBINED (NCSA Combined Log Format with Virtual Host)	%v:%^ %h %^[%d:%t %^] "%r" %s %b "%R" "%u"
COMMON (Common Log Format (CLF))	%h %^[%d:%t %^] "%r" %s %b
VCOMMON (Common Log Format (CLF) with Virtual Host)	%v:%^ %h %^[%d:%t %^] "%r" %s %b
W3C	%d %t %^ %m %U %q %^ %^ %h %u %R %s %^ %^ %L
SQUID (Squid Native Format)	%^ %^ %^ %v %^: %x.%^ %~%L %h %^/%s %b %m %U

goaccess 웹 접근

http://go.sangchul.kr

a2

 

참고URL

- https://goaccess.io