HyperAIHyperAI
2 months ago

On Automatic Parsing of Log Records

Rand, Jared ; Miranskyy, Andriy
On Automatic Parsing of Log Records
Abstract

Software log analysis helps to maintain the health of software solutions andensure compliance and security. Existing software systems consist ofheterogeneous components emitting logs in various formats. A typical solutionis to unify the logs using manually built parsers, which is laborious. Instead, we explore the possibility of automating the parsing task byemploying machine translation (MT). We create a tool that generates syntheticApache log records which we used to train recurrent-neural-network-based MTmodels. Models' evaluation on real-world logs shows that the models can learnApache log format and parse individual log records. The median relative editdistance between an actual real-world log record and the MT prediction is lessthan or equal to 28%. Thus, we show that log parsing using an MT approach ispromising.