On Trusted AI Platforms

Abstract

The development and use of artificial intelligence systems (machine learning) in critical areas (avionics, autonomous movement, etc.) inevitably raise the question of the reliability of the software used. Trusted computing systems have been around for a long time. Their meaning is to allow the execution of only certain applications and guarantee against interference with the work of such applications. Trust in this case is the confidence that the assigned applications work as they did when tested. But in the case of machine learning, this is not enough. The application can work as intended, there is no intervention, but the results cannot be trusted simply because the data has changed. In general, this problem is a consequence of a fundamental point for all machine learning systems - the data at the testing (operation) stage may differ from the same data on which the system was trained. Accordingly, a violation of the machine learning system is possible without targeted actions, simply because we encountered data at the operational stage for which the generalization achieved at the training stage does not work. And there are also attacks, which are understood as special actions on the elements of the machine learning pipeline (training data, the model itself, test data) in order to either achieve the desired behavior of the system or prevent it from working correctly. Today, this problem, which is generally associated with the stability of machine learning systems, is the main obstacle to the use of machine learning in critical applications.