Elasticsearch – how to parse MySQL general log

Elasticsearch provides native integration using beats or Agent to collect MySQL errors/slow logs, however if you want detailed auditing via MySQL general log, you can parse the log as following:

• Configure a new Pipeline and Processor using GROK, here is the GROK to parse MySQL Logs:

%{TIMESTAMP_ISO8601:transactionDate}   %{INT:LogId} %{WORD:Type}\t%{WORD:Type1} %{GREEDYDATA:Type3}

NOTE: please review the spaces properly , make sure to review it before deployment.

• Create a new file stream integration in Elastic Agent, point the stream to the proper path where MySQL general logs are kept and configure the pipleline to be the newly created custom pipleline.

Have fun.