|
Work item:
|
Q.MMAI
|
|
Subject/title:
|
Methods and metrics for monitoring ML/AI in future networks including IMT-2020
|
|
Status:
|
Under study
|
|
Approval process:
|
AAP
|
|
Type of work item:
|
Recommendation
|
|
Version:
|
New
|
|
Equivalent number:
|
-
|
|
Timing:
|
05-2025 (Medium priority)
|
|
Liaison:
|
ITU-T SG13, ISO/IEC SC42, IEEE
|
|
Supporting members:
|
China Telecom, China Unicom, Zhejiang Lab, Xi'an Jiaotong University
|
|
Summary:
|
Summary (provides a brief overview of the purpose and contents of the Recommendation, thus permitting readers to judge its usefulness for their work):
ML/AI monitoring refers to the ongoing process of tracking and observing an ML/AI system's performance in real-time. ML/AI monitoring evaluates the performance of ML/AI model to determine whether it operates effectively. When the ML model experiences some performance degradation, appropriate maintenance measures should be taken to restore performance.
The ML/AI algorithm predicts the future or optimizes the process based on the data at the time of model establishment. However, the environment in which we live is constantly changing, and the parameter values are also constantly changing. Therefore, the environment and running state of ML/AI model should be monitored in order to determine whether the model should be updated or not.
To overcome various issues that resulted in performance degradation, a set of parameters and events should be defined and monitored. Given the different causes of these issues selecting appropriate monitoring metrics is a key step in identifying the ML/AI performance degradation, but there are no standards that provide a comprehensive guide on various monitoring metrics.
This proposed work item will give a guide and reference of ML/AI monitoring metrics.
|
|
Comment:
|
-
|
|
Reference(s):
|
|
|
Historic references:
|
|
Contact(s):
|
|
| ITU-T A.5 justification(s): |
|
|
|
|
First registration in the WP:
2023-06-01 12:40:24
|
|
Last update:
2024-04-24 14:19:20
|
