MODALITY corpus - SPEAKER 08 - SEQUENCE S1 - Open Research Data - MOST Wiedzy

Wyszukiwarka

MODALITY corpus - SPEAKER 08 - SEQUENCE S1

Opis

The MODALITY corpus is one of the multimodal database of word recordings in English. It consists of over 30 hours of multimodal recordings. The database contains high-resolution, high-framerate stereoscopic video streams and audio signals obtained from a microphone array and a laptop microphone. The corpus can be employed to develop an AVSR system, as every utterance was labelled. Recordings in noisy conditions can be used to test the robustness of speech recognition systems.

The language material was based on a remote control scenario and it includes 231 words -numbers, names of months and days, a set of verbs and nouns related to a computer device control. They were read by speakers as separated words and sequences resulting in a set of 12 recording sessions per speaker. Half of the sessions were recorded in quiet conditions, the other half contained three kinds of intrusive signals (traffic, babble and factory noise).

The corpus includes recordings of 42 speakers (33 male, 9 female). The participants include 20 students and staff of Multimedia Systems Department of the Gdańsk University of Technology, 5 students of the Institute of English and American Studies of the University of Gdańsk, and 17 native English speakers.

The dataset consist of recordings and visual features for SPEAKER 08:

  • sex: man
  • native speaker: no
  • age: 28

The test material: SEQUENCE S1

All recordings for all speakers are available at http://www.modality-corpus.org/

Ilustracja publikacji

Sample still from the corpus
(SPEAKER 08)

Due to the size of the corpus (approx. 2.5 TB of data), every speaker’s recording was placed in a separate zip file of the size approx. 4-7 GB each.

The recordings were organized according to the speakers’ language skills. The group A (17 speakers) consists of native-speakers. Non-native speakers recordings (Polish nationals) were placed in the Group B (25 speakers).

The audio files use the Waveform Audio File Format (.wav), and contain a single PCM audio stream sampled at 44.1 kSa/s with 16-bit depth. The video files utilize the Matroska Multimedia Container Format (.mkv) in which a video stream in 1080p resolution, captured at 100 fps was placed after being compressed with h.265 codec (using High 4:4:4 profile). The ‘.lab’ files are text files containing the information on word positions in audio files, and follow the HTK label format. Each line of a ‘.lab’ file contains the actual label preceded by start and end times (in 100 ns units) e.g. : 1239620000 1244790000 FIVE which denotes the word “five”, occurring between the 123.962 s and 124.479 s of audio.
Word-accurate SNR values calculated for every recording are also included in the ZIP file.

Plik z danymi badawczymi

SP08_SEQUENCE1.ZIP
514.2 MB, S3 ETag 60969559339b432d37c86dbeb8d196a3-2, pobrań: 16
Hash pliku liczony jest ze wzoru
hexmd5(md5(part1)+md5(part2)+...)-{parts_count} gdzie pojedyncza część pliku jest wielkości 512 MB

Przykładowy skrypt do wyliczenia:
https://github.com/antespi/s3md5
pobierz plik SP08_SEQUENCE1.ZIP

Informacje szczegółowe o pliku

Licencja:
Niestandardowa
przeczytaj

Informacje szczegółowe

Rok publikacji:
2016
Data zatwierdzenia:
2021-03-19
Język danych badawczych:
angielski
Dyscypliny:
  • Informatyka techniczna i telekomunikacja (Dziedzina nauk inżynieryjno-technicznych)
  • Językoznawstwo (Dziedzina nauk humanistycznych)
DOI:
Identyfikator DOI 10.34808/mb6b-j908 otwiera się w nowej karcie
Seria:
Weryfikacja:
Politechnika Gdańska

Słowa kluczowe

Powiązane zasoby

Cytuj jako

wyświetlono 53 razy