Preferred Benchmarking Criteria for Systematic Taxonomy of Embedded Platforms (STEP) in Human System Interaction Systems

Alicja Kwaśniewska; Sharath Raghava; Carlos Davila; Mikael Sevenier; David Gamba; Jacek Rumiński

doi:10.1109/hsi55341.2022.9869470

Preferred Benchmarking Criteria for Systematic Taxonomy of Embedded Platforms (STEP) in Human System Interaction Systems

Abstract

The rate of progress in the field of Artificial Intelligence (AI) and Machine Learning (ML) has significantly increased over the past ten years and continues to accelerate. Since then, AI has made the leap from research case studies to real production ready applications. The significance of this growth cannot be undermined as it catalyzed the very nature of computing. Conventional platforms struggle to achieve greater performance and efficiency, what causes a surging demand for innovative AI accelerators, specialized platforms and purpose-built computes. At the same time, it is required to provide solutions for assessment of ML platform performance in a reproducible and unbiased manner to be able to provide a fair comparison of different products. This is especially valid for Human System Interaction (HSI) systems that require specific data handling for low latency responses in emergency situations or to improve user experience, as well as for preserving data privacy and security by processing it locally. Taking it into account, this work presents a comprehensive guideline on preferred benchmarking criteria for evaluation of ML platforms that include both lower level analysis of ML models and system-level evaluation of the entire pipeline. In addition, we propose a Systematic Taxonomy of Embedded Platforms (STEP) that can be used by the community and customers for better selection of specific ML hardware consistent with their needs for better design of ML-based HSI solutions.

Citations

1

CrossRef
0

Web of Science
2

Scopus

Authors (6)

Alicja Kwaśniewska dr inż.
Sharath Raghava
Carlos Davila
Mikael Sevenier
David Gamba
Jacek Rumiński prof. dr hab. inż.

Cite as

Full text

full text is not available in portal

full content of the article see on external site open in new tab

Keywords

INDUSTRIES, SYSTEMATICS, TAXONOMY, BENCHMARK TESTING, RELIABILITY ENGINEERING, USER EXPERIENCE, SYSTEM-ON-CHIP

Details

Category:

Conference activity

Type:

publikacja w wydawnictwie zbiorowym recenzowanym (także w materiałach konferencyjnych)

Language:

English

Publication year:

2022

Bibliographic description:

Kwaśniewska A., Raghava S., Davila C., Sevenier M., Gamba D., Rumiński J.: Preferred Benchmarking Criteria for Systematic Taxonomy of Embedded Platforms (STEP) in Human System Interaction Systems// / : , 2022,

DOI: