NDLI: Prosodic Phrasing: Machine and Human Evaluation

Content Provider	Springer Nature Link
Author	Viana, M. Céu Oliveira, Luís C. Mata, Ana I.
Copyright Year	2003
Abstract	This paper describes a set of experiments aiming at the construction and evaluation of a new phrasing module for European Portuguese text-to-speech synthesis, using classification and regression trees learned from hand-labelled texts. Using the assessment criteria of matching boundary predictions against the corresponding labelled ones, the best solution achieves an overall performance of 91.9%, with 86.3% of correctly assigned breaks and 4.3% of false insertions. Although in absolute terms such scores may be considered surprisingly good given the size of the training set, the total number of exact matches at the sentence level is much lower (22%). This suggested a more formal experiment to test the acceptability of the predicted phrasing in the judgement of human evaluators. As the model was not trained on a labelled speech corpus but on hand-labelled texts, the reference phrasing needed also to be assessed. The evaluation experiment involved 90 participants who were asked to grade both the automatic and the reference phrasings, and also to express their opinion on where the breaks should be placed. As expected, the results showed a large variability among the subjects in their acceptance of a specific sentence partition, and criteria had to be defined to summarise the data from the different evaluators. With the adopted criteria, the performance of the automatic assignment procedure at the sentence level is better rated by human evaluators than by simple matching with the reference corpus (78% vs. 22%, respectively).
Starting Page	83
Ending Page	94
Page Count	12
File Format	PDF
ISSN	13812416
Journal	International Journal of Speech Technology
Volume Number	6
Issue Number	1
e-ISSN	15728110
Language	English
Publisher	Kluwer Academic Publishers
Publisher Date	2003-01-01
Publisher Place	Boston
Access Restriction	One Nation One Subscription (ONOS)
Subject Keyword	Artificial Intelligence (incl. Robotics) Signal, Image and Speech Processing Communication
Content Type	Text
Resource Type	Article
Subject	Human-Computer Interaction Computer Vision and Pattern Recognition Software Linguistics and Language

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in

Error Detection in Spoken Human-Machine Interaction

Handling Knowledge Sources in Human-Machine Interaction

Software text-to-speech

An Automatic System for Detecting Prosodic Prominence in American English Continuous Speech

Collaborative conversational interfaces

Prosodic modeling for Mandarin Chinese speech embedded with English spelling

Audiovisual Speech Synthesis

Evaluation of Software Tutoring for a Speech Interface

Continuous speech recognition and syntactic processing in Iranian Farsi language

Prosodic Phrasing: Machine and Human Evaluation

Similar Documents

Error Detection in Spoken Human-Machine Interaction

Handling Knowledge Sources in Human-Machine Interaction

Software text-to-speech

An Automatic System for Detecting Prosodic Prominence in American English Continuous Speech

Collaborative conversational interfaces

Prosodic modeling for Mandarin Chinese speech embedded with English spelling

Audiovisual Speech Synthesis

Evaluation of Software Tutoring for a Speech Interface

Continuous speech recognition and syntactic processing in Iranian Farsi language

Prosodic Phrasing: Machine and Human Evaluation