Scoreboard Architectural Pattern and Integration of Emotion Recognition Results - Publication - Bridge of Knowledge

Your account

login

Search

Scoreboard Architectural Pattern and Integration of Emotion Recognition Results

Abstract

This paper proposes a new design pattern, named Scoreboard , dedicated for applications solving complex, multi-stage, non-deterministic problems. The pattern provides a computational framework for the design and implementation of systems that integrate a large number of diverse specialized modules that may vary in accuracy, solution level, and modality. The Scoreboard is an extension of Blackboard design pattern and comes under behavioral type. The pattern allows for an integration of multimodal results, employing early, and/or late fusion paradigms. Additionally, it provides a framework for the evaluation of the modules, dealing with inconsistency and low accuracy. In this paper, the Scoreboard design pattern is described with the standard meta-data model, followed by a sample implementation. This paper also provides the evaluation results based on experiments and a case study. The evaluation results confirmed the robustness, modularization, ease of integration, efficiency, and adaptability of the solutions with the Scoreboard pattern in comparison with the Blackboard pattern and ‘‘no pattern’’ condition. This paper provides also a case study of Scoreboard application in an integration of emotion recognition results. There are certain complex problems in modern software engineering which require multi-stage, multi-party, multi-modal solutions, and non-deterministic control strategies. Among those are natural language processing, image processing, and emotion recognition, to name just a few. The proposed Scoreboard pattern might be used in the software addressing the problems, especially in research systems that explore large solution spaces and require runtime decisions on execution order.

Citations

  • 1

    CrossRef

  • 0

    Web of Science

  • 1

    Scopus

Authors (2)

Cite as

Full text

download paper
downloaded 23 times
Publication version
Accepted or Published Version
License
Creative Commons: CC-BY open in new tab

Keywords

Details

Category:
Articles
Type:
artykuł w czasopiśmie wyróżnionym w JCR
Published in:
IEEE Access no. 7, pages 7228 - 7249,
ISSN: 2169-3536
Language:
English
Publication year:
2019
Bibliographic description:
Landowska A., Brodny G.: Scoreboard Architectural Pattern and Integration of Emotion Recognition Results// IEEE Access. -Vol. 7, (2019), s.7228-7249
DOI:
Digital Object Identifier (open in new tab) 10.1109/access.2018.2889557
Bibliography: test
  1. C. Alexander, S. Ishikawa, and M. Silverstein, A Pattern Language. Berkeley, CA, USA: Oxford Univ. Press, 1977. open in new tab
  2. E. Gamma, R. Helm, R. Johnson, and J. Vlissides, Design Patterns CD: Elements of Reusable Object-Oriented Software (Addison-Wesley Profes- sional Computing). Reading, MA, USA: Addison-Wesley, 1996. open in new tab
  3. D. Schmidt, M. Stal, H. Rohnert, and F. Buschmann, Pattern-Oriented Software Architecture: A System of Patterns, vol. 2. Hoboken, NJ, USA: Wiley, 2000.
  4. J. Hannemann and G. Kiczales, ''Design pattern implementation in Java and aspectJ,'' ACM SIGPLAN Notices, vol. 37, no. 11, pp. 161-173, 2002. open in new tab
  5. T. Erl, SOA Design Patterns. Upper Saddle River, NJ, USA: Prentice-Hall, 2009.
  6. J. Tidwell, Designing Interfaces: Patterns for Effective Interaction Design. Newton, MA, USA: O'Reilly, 2005.
  7. E. Cambria, Y. Song, H. Wang, and N. Howard, ''Semantic multidimen- sional scaling for open-domain sentiment analysis,'' IEEE Intell. Syst., vol. 29, no. 2, pp. 44-51, Mar./Apr. 2014. open in new tab
  8. S. Dublin et al., ''Natural language processing to identify pneumonia from radiology reports,'' Pharmacoepidemiol. Drug Saf., vol. 22, no. 8, pp. 834-841, Aug. 2013. open in new tab
  9. R. Lienhart, L. Liang, and A. Kuranov, ''A detector tree of boosted classi- fiers for real-time object detection and tracking,'' in Proc. IEEE Int. Conf. Multimedia Expo, Jul. 2003, p. II-277. open in new tab
  10. A. Landowska, ''Emotion monitor-Concept, construction and lessons learned,'' in Proc. Federated Conf. Comput. Sci. Inf. Syst. (FedCSIS), 2015, pp. 75-80. open in new tab
  11. F. Buschmann, R. Meunier, and H. Rohnert, Pattern-Oriented Software Architecture: A System of Patterns. Hoboken, NJ, USA: Wiley, 1996.
  12. T. S. Levitt, ''Choosing uncertainty representations in artificial intelli- gence,'' Int. J. Approx. Reason., vol. 2, no. 3, pp. 217-232, Jul. 1988. open in new tab
  13. M. Mohri, F. Pereira, and M. Riley, ''Weighted finite-state transducers in speech recognition,'' Comput. Speech Lang., vol. 16, no. 1, pp. 69-88, Jan. 2002. open in new tab
  14. A. Norouzian and R. Rose, ''An approach for efficient open vocabulary spoken term detection,'' Speech Commun., vol. 57, pp. 50-62, Feb. 2014. open in new tab
  15. M. O. Rabin and D. Scott, ''Finite automata and their decision problems,'' IBM J. Res. Develop., vol. 3, no. 2, pp. 114-125, 1959. open in new tab
  16. A. Cleeremans, D. Servan-Schreiber, and J. L. McClelland, ''Finite state automata and simple recurrent networks,'' Neural Comput., vol. 1, no. 3, pp. 372-381, Sep. 1989. open in new tab
  17. J. Daciuk, S. Mihov, B. W. Watson, and R. E. Watson, ''Incremental con- struction of minimal acyclic finite-state automata,'' Comput. Linguistics, vol. 26, no. 1, pp. 3-16, Mar. 2000. open in new tab
  18. J. F. Ackermann and M. S. Landy, ''Suboptimal decision criteria are predicted by subjectively weighted probabilities and rewards,'' Attention, Perception, Psychophys., vol. 77, no. 2, pp. 638-658, Feb. 2015. open in new tab
  19. A. Ampatzoglou, S. Charalampidou, and I. Stamelos, ''Research state of the art on GoF design patterns: A mapping study,'' J. Syst. Softw., vol. 86, no. 7, pp. 1945-1964, 2013. open in new tab
  20. V. Lesser, R. Fennell, L. Erman, and D. Reddy, ''Organization of the hearsay II speech understanding system,'' IEEE Trans. Acoust., Speech, Signal Process., vol. ASSP-23, no. 1, pp. 11-24, Feb. 1975. open in new tab
  21. A. Landowska, G. Brodny, and M. R. Wrobel, ''Limitations of emotion recognition from facial expressions in e-learning context,'' in Proc. 9th Int. Conf. Comput. Support. Educ. (CSEDU), vol. 2, 2017, pp. 383-389. open in new tab
  22. S. Poria, E. Cambria, R. Bajpai, and A. Hussain, ''A review of affective computing: From unimodal analysis to multimodal fusion,'' Inf. Fusion, vol. 37, pp. 98-125, Sep. 2017. open in new tab
  23. H. Gunes and B. Schuller, ''Categorical and dimensional affect analysis in continuous input: Current trends and future directions,'' Image Vis. Comput., vol. 31, no. 2, pp. 120-136, Feb. 2013. open in new tab
  24. Z. Zeng, M. Pantic, G. I. Roisman, and T. S. Huang, ''A survey of affect recognition methods: Audio, visual, and spontaneous expressions,'' IEEE Trans. Pattern Anal. Mach. Intell., vol. 31, no. 1, pp. 39-58, Jan. 2009.
  25. T. Partala and A. Kallinen, ''Understanding the most satisfying and unsat- isfying user experiences: Emotions, psychological needs, and context,'' Interact. Comput., vol. 24, no. 1, pp. 25-34, Jan. 2012. open in new tab
  26. R. L. Hazlett and J. Benedek, ''Measuring emotional valence to understand the user's experience of software,'' Int. J. Hum.-Comput. Stud., vol. 65, no. 4, pp. 306-314, Apr. 2007. open in new tab
  27. P. G. Zimmermann, P. Gomez, B. Danuser, and S. G. Schär, ''Extending usability: Putting affect into the user-experience,'' in Proc. NordiCHI, 2006, pp. 27-32.
  28. H. H. Binali, C. Wu, and V. Potdar, ''A new significant area: Emotion detection in E-learning using opinion mining techniques,'' in Proc. IEEE Int. Conf. Digit. Ecosyst. Technol., Jun. 2009, pp. 259-264. open in new tab
  29. A. Landowska, ''Affect-awareness framework for intelligent tutoring systems,'' in Proc. 6th Int. Conf. Hum. Syst. Interact. (HSI), 2013, pp. 540-547. open in new tab
  30. A. Landowska, M. Szwoch, and W. Szwoch, ''Methodology of affective intervention design for intelligent systems,'' Interact. Comput., vol. 28, no. 6, pp. 737-759, 2016. open in new tab
  31. K. Hone, ''Empathic agents to reduce user frustration: The effects of varying agent characteristics,'' Interact. Comput., vol. 18, no. 2, pp. 227-245, 2006. open in new tab
  32. A. Kolakowska, A. Landowska, M. Szwoch, W. Szwoch, and M. R. Wro- bel, ''Emotion recognition and its applications,'' in Human-Computer Systems Interaction: Backgrounds and Applications. Cham, Switzerland: Springer, 2014, pp. 51-62. open in new tab
  33. L. Chittaro and R. Sioni, ''Affective computing vs. Affective placebo: Study of a biofeedback-controlled game for relaxation training,'' Int. J. Hum.-Comput. Stud., vol. 72, pp. 663-673, Aug./Sep. 2014. open in new tab
  34. T. Partala and V. Surakka, ''The effects of affective interventions in human- computer interaction,'' Interact. Comput., vol. 16, no. 2, pp. 295-309, 2004. open in new tab
  35. A. Kołakowska, A. Landowska, W. Szwoch, W. R. Wróbel, and M. Szwoch, ''Emotion recognition and its application in software engineer- ing,'' in Proc. 6th Int. Conf. Hum. Syst. Interact. (HSI), 2013, pp. 532-539. open in new tab
  36. M. R. Wrobel, ''Emotions in the software development process,'' in Proc. 6th Int. Conf. Hum. Syst. Interact., Apr. 2013, pp. 518-523. open in new tab
  37. A. Landowska, ''Emotion monitoring-Verification of physiological char- acteristics measurement procedures,'' Metrol. Meas. Syst., vol. 21, no. 4, pp. 719-732, 2014. open in new tab
  38. H. Gunes and M. Piccardi, ''Affect recognition from face and body: Early fusion vs. Late fusion,'' in Proc. IEEE Int. Conf. Syst. Man Cybern., vol. 4, Oct. 2005, pp. 3437-3443. open in new tab
  39. I. Hupont, S. Ballano, S. Baldassarri, and E. Cerezo, ''Scalable multimodal fusion for continuous affect sensing,'' in Proc. IEEE Symp. Ser. Com- put. Intell. (SSCI), Workshop Affect. Comput. Intell. (WACI), Apr. 2011, pp. 1-8. open in new tab
  40. S. Poria, E. Cambria, and A. Gelbukh, ''Deep convolutional neural network textual features and multiple kernel learning for utterance-level multimodal sentiment analysis,'' in Proc. Conf. Empirical Methods Natural Lang. Process., 2015, pp. 2539-2544. open in new tab
  41. M. Mansoorizadeh and N. M. Charkari, ''Multimodal information fusion application to human emotion recognition from face and speech,'' Multimed. Tools Appl., vol. 49, no. 2, pp. 277-297, Aug. 2010. open in new tab
  42. S. Poria, E. Cambria, N. Howard, G.-Bin Huang, and A. Hussain, ''Fusing audio, visual and textual clues for sentiment analysis from multimodal content,'' Neurocomputing, vol. 174, pp. 50-59, Jan. 2016. open in new tab
  43. M. Wollmer et al., ''YouTube movie reviews: Sentiment analysis in an audio-visual context,'' IEEE Intell. Syst., vol. 28, no. 3, pp. 46-53, May 2013. open in new tab
  44. P. K. Atrey, M. A. Hossain, A. El Saddik, and M. S. Kankanhalli, ''Multi- modal fusion for multimedia analysis: A survey,'' Multimedia Syst., vol. 16, no. 6, pp. 345-379, 2010. open in new tab
  45. Software Engineering-Product Quality-Part 1: Quality Model, Standard ISO/IEC 9126:2001, ISO, 2001. open in new tab
  46. R. G. Dromey, ''A model for software product quality,'' IEEE Trans. Softw. Eng., vol. 21, no. 2, pp. 146-162, Feb. 1995. open in new tab
  47. M. H. Klein, R. Kazman, L. Bass, J. Carriere, M. Barbacci, and H. Lipson, ''Attribute-based architecture styles,'' in Proc. Work. Conf. Softw. Archit., 1999, pp. 225-243. open in new tab
  48. F. Losavio, L. Chirinos, and M. A. Perez, ''Quality models to design software architectures,'' in Proc. Technol. Object-Oriented Lang. Syst. (TOOLS), 2001, pp. 123-135. open in new tab
  49. B. Meyer, Object-Oriented Software Construction. Upper Saddle River, NJ, USA: Prentice-Hall, 1989.
  50. Y. Press and L. Constantine, Structured Design: Fundamentals of a Disci- pline of Computer Program and Systems Design. Upper Saddle River, NJ, USA: Prentice-Hall, 1979.
  51. A. Shaik, B. Manda, C. Prakashini, C. R. K. Reddy, and K. Deepthi, ''Metrics for object oriented design software systems: A survey,'' J. Emerg. Trends Eng. Appl. Sci., vol. 1, no. 2, pp. 190-198, 2010.
  52. M. A. Babar, L. Zhu, and R. Jeffery, ''A framework for classifying and comparing software architecture evaluation methods,'' in Proc. Softw. Eng. Conf., 2004, pp. 309-318. open in new tab
  53. R. Kazman, M. Klein, and P. Clements, ''ATAM: Method for architecture evaluation,'' Softw. Eng. Inst., Carnegie-Mellon Univ, Pittsburgh, PA, USA, Tech. Rep. CMU/SEI-2000-TR-004, 2000. open in new tab
  54. A. Landowska, ''Emotion monitor-concept, construction and lessons learned,'' in Proc. Federated Conf. Comput. Sci. Inf. Syst. (FedCSIS), Oct. 2015, pp. 75-80. open in new tab
  55. R. Van Solingen, V. Basili, G. Caldiera, and H. D. Rombach, ''Goal ques- tion metric (GQM) approach,'' in Encyclopedia of Software Engineering. Hoboken, NJ, USA: Wiley, 2002. open in new tab
  56. T. J. McCabe, ''A complexity measure,'' IEEE Trans. Softw. Eng., vol. SE-2, no. 4, pp. 308-320, Dec. 1976. open in new tab
  57. K. A. M. Ferreira, M. A. S. Bigonha, R. S. Bigonha, L. F. O. Mendes, and H. C. Almeida, ''Identifying thresholds for object-oriented software metrics,'' J. Syst. Softw., vol. 85, no. 2, pp. 244-257, Feb. 2012. open in new tab
  58. T. L. Alves, C. Ypma, and J. Visser, ''Deriving metric thresholds from benchmark data,'' in Proc. IEEE Int. Conf. Softw. Maintenance, Sep. 2010, pp. 1-10. open in new tab
  59. P. Oliveira, M. T. Valente, and F. P. Lima, ''Extracting relative thresh- olds for source code metrics,'' in Proc. IEEE Conf. Softw. Maintenance, Reeng., Reverse Eng. Softw. Evol. Week (CSMR-WCRE), Feb. 2014, pp. 254-263. open in new tab
  60. S. Herbold, J. Grabowski, and S. Waack, ''Calculation and optimization of thresholds for sets of software metrics,'' Empir. Softw. Eng., vol. 16, no. 6, pp. 812-841, Dec. 2011. open in new tab
  61. A. Landowska, ''Web questionnaire as construction method of affect- annotated lexicon-Risks reduction strategy,'' in Proc. Int. Conf. Affect. Comput. Intell. Interact. (ACII), Sep. 2015, pp. 421-427. open in new tab
  62. A. Kolakowska, A. Landowska, M. Szwoch, W. Szwoch, and M. R. Wrobel, ''Evaluation criteria for affect-annotated databases,'' in Beyond Databases, Architectures and Structures (Communications in Computer and Information Science). Cham, Switzerland: Springer, 2015. open in new tab
  63. M. Bradley and P. Lang, ''Affective norms for English words (ANEW): Instruction manual and affective ratings,'' Center Res. Psychophysiol., Univ. Florida, Gainesville, FL, USA, Tech. Rep. C-1, 1999.
  64. G. Brodny and A. Landowska, ''Integration in multichannel emotion recognition,'' in Proc. 11th Int. Conf. Hum. Syst. Interact. (HSI), 2018, pp. 35-41. open in new tab
Sources of funding:
Verified by:
Gdańsk University of Technology

seen 177 times

Recommended for you

Affective Learning Manifesto – 10 Years Later

2014

Investigation of educational processes with affective computing methods

2017

Limitations of Emotion Recognition in Software User Experience Evaluation Context

2016

Mining inconsistent emotion recognition results with the multidimensional model

2021
Meta Tags