NDLI: Stochastic Arabic hybrid diacritizer

Content Provider	IEEE Xplore Digital Library
Author	Rashwan, M. Al Badrashiny, M. Attia, M. Abdou, S. Rafea, A.
Copyright Year	2009
Description	Author affiliation: Department of Computer Science, American University in Cairo(AUC), Egypt (Rafea, A.) \|\| Department of Electronics & Electrical Communications, Faculty of Eng., Cairo Univ., Egypt (Rashwan, M.) \|\| The Engineering Company for the, Development of Computer Systems; RDI, Egypt (Al Badrashiny, M.; Abdou, S.) \|\| Faculty of Computers & Information, Cairo Univ., Egypt (Attia, M.)
Abstract	This paper introduces a two-layer stochastic system to diacritize raw Arabic text automatically. The first layer determines the most likely diacritics by choosing the sequence of full-form Arabic word diacritizations with maximum marginal probability via $A^{∗}$ lattice search algorithm and m-gram probability estimation. When full-form words are out-of-vocabulary (OOV), the system utilizes a second layer, which factorizes each Arabic word into its possible morphological constituents (prefix, root, pattern and suffix), then uses m-gram probability estimation and $A^{∗}$ lattice search algorithm to select among the possible factorizations to get the most likely diacritization sequence. While the second layer has better coverage of possible Arabic forms, the first layer yields better disambiguation results for the same size of training corpora, especially for inferring syntactical (case-end) diacritics. The presented hybrid system possesses the advantages of both layers. The paper details the workings of both layers and the architecture of the hybrid system. By comparing our proposed system with the best performing system to our knowledge of Habash et al. [9] using their training and testing corpus; it is found that the word error rates of 5.5% for the morphological diacritization and 9.4% for the syntactic diacritization by Habash et al., and only 3.1% for the morphological diacritization and 9.4% for the syntactic diacritization by our system.
Starting Page	1
Ending Page	8
File Size	333501
Page Count	8
File Format	PDF
ISBN	9781424445387
DOI	10.1109/NLPKE.2009.5313742
Language	English
Publisher	Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Publisher Date	2009-09-24
Publisher Place	China
Access Restriction	Subscribed
Rights Holder	Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Subject Keyword	Computer science Vocabulary System testing Stochastic systems Stochastic processes Lattices Morphology Training data Tagging Speech synthesis
Content Type	Text
Resource Type	Article

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in

A Stochastic Arabic Diacritizer Based on a Hybrid of Factorized and Unfactorized Textual Features

Development of a phonetic system for large vocabulary Arabic speech recognition

An unrestricted vocabulary Arabic speech synthesis system

Using morphology towards better large-vocabulary speech recognition systems

Pattern-based algorithm for Part-of-Speech tagging Arabic text

Speaker adaptation for large vocabulary speech recognition systems using speaker Markov models

Pronunciation variation generation for spontaneous speech synthesis using state-based voice transformation

Training of lexical models based on DTW-based parameter reestimation algorithm

Hybrid segmental-LVQ/HMM for large vocabulary speech recognition

Stochastic Arabic hybrid diacritizer

Similar Documents

A Stochastic Arabic Diacritizer Based on a Hybrid of Factorized and Unfactorized Textual Features

Development of a phonetic system for large vocabulary Arabic speech recognition

An unrestricted vocabulary Arabic speech synthesis system

Using morphology towards better large-vocabulary speech recognition systems

Pattern-based algorithm for Part-of-Speech tagging Arabic text

Speaker adaptation for large vocabulary speech recognition systems using speaker Markov models

Pronunciation variation generation for spontaneous speech synthesis using state-based voice transformation

Training of lexical models based on DTW-based parameter reestimation algorithm

Hybrid segmental-LVQ/HMM for large vocabulary speech recognition

Stochastic Arabic hybrid diacritizer