|
Network anomaly detection based on keyword embedding log
|
Authors: Yong Song, Zhiwei Yan, Yukun Qin, Yuchen Xie, Xiaozhou Ye, Ye Ouyang
Status: Final
Date of publication: 22 September 2022
Published in: ITU Journal on Future and Evolving Technologies, Volume 3 (2022), Issue 2, Pages 235-242
Article DOI : https://doi.org/10.52953/ICXQ5497
|
Abstract:
Log anomaly detection is an important and challenging task in the field of Artificial Intelligence for IT Operations (AIOps). Logs that record important runtime information are widely used for troubleshooting purposes. There have been many studies that use log data to construct deep learning methods for detecting system anomalies, which are usually based on log parsing. However, they ignore the effect of keywords that are promising for system status analysis. Here, we propose KELog (Keyword Embedding Log), a novel log anomaly detection approach that utilizes keyword information. We build a keyword library by keyword information extraction and fuse them into log representations. In this way, KELog can raise the reliability of anomaly detection. The experimental results on a real-world log dataset of a communications operator show that the F1 score of our proposed KELog method achieves a maximum increase of 0.341 compared with the commonly used machine learning algorithms (PCA, SVM, Invaiant Mining) and a maximum increase of 0.039 compared with deep learning algorithms (DeepLog, LogBERT) respectively. In 2021, ITU launched the second ITU AI/ML in 5G Challenge. We used KELog to participate in the thematic track of the Artificial Intelligence Innovation and Application Competition in the China Division, and won first place with a full F1 score. |
Keywords: Artificial intelligence for IT operations, deep learning, keyword information, log anomaly detection, transformer
Rights: © International Telecommunication Union, available under the CC BY-NC-ND 3.0 IGO license.
|
|
| ITEM DETAIL | ARTICLE | PRICE | |
|---|
ENGLISH 
Full article (PDF) |
|
| 0
| Free of charge | DOWNLOAD |
|
|
| |
