Academia.eduAcademia.edu

Outline

Title
Abstract
Key Takeaways
Introduction
Related Work
Arabic Pos Tagging and Morphological Analysis
Data
Data Annotation
Corpus Statistics
Methods
Interim Summary
Discussion and Conclusion
References
FAQs
All Topics
Computer Science
Artificial Intelligence

Part-of-Speech and Morphological Tagging of Algerian Judeo-Arabic

Profile image of Ofra Tirosh-BeckerOfra Tirosh-Becker

2022, Northern European Journal of Language Technology

visibility

…

description

21 pages

Abstract

Most linguistic studies of Judeo-Arabic, the ensemble of dialects spoken and written by Jews in Arab lands, are qualitative in nature and rely on laborious manual annotation work, and are therefore limited in scale. In this work, we develop automatic methods for morpho-syntactic tagging of Algerian Judeo-Arabic texts published by Algerian Jews in the 19th-20th centuries, based on a linguistically tagged corpus. First, we describe our semi-automatic approach for preprocessing these texts. Then, we experiment with both an off-the-shelf morphological tagger, several specially designed neural network taggers, and a hybrid human-in-the-loop approach. Finally, we perform a real-world evaluation of new texts that were never tagged before in comparison with human expert annotators. Our experimental results demonstrate that these methods can dramatically speed up and improve the linguistic research pipeline, enabling linguists to study these dialects on a much greater scale.

Key takeaways
sparkles

AI

  1. Developed AJATag to automate morpho-syntactic tagging of Algerian Judeo-Arabic, enhancing linguistic research scalability.
  2. AJATag achieved over 91% overall accuracy, outperforming the benchmark MarMoT by 2% in real-world evaluations.
  3. The TAJA corpus comprises 9904 sentences and 61,481 tokens, crucial for training the tagging models.
  4. Hierarchical models showed improved performance with out-of-vocabulary words, reaching 78.42% accuracy on analysis2.
  5. The study addresses the scarcity of AJA linguists, facilitating large-scale analysis of dialects through automation.

Related papers

Smoothing a lexicon-based POS tagger for Arabic and Hebrew
Khalil Sima'an

Proceedings of the 2007 Workshop on Computational Approaches to Semitic Languages Common Issues and Resources - Semitic '07, 2007

We propose an enhanced Part-of-Speech (POS) tagger of Semitic languages that treats Modern Standard Arabic (henceforth Arabic) and Modern Hebrew (henceforth Hebrew) using the same probabilistic model and architectural setting. We start out by porting an existing Hidden Markov Model POS tagger for Hebrew to Arabic by exchanging a morphological analyzer for Hebrew with Buckwalter's (2002) morphological analyzer for Arabic. This gives state-of-theart accuracy (96.12%), comparable to Habash and Rambow's (2005) analyzerbased POS tagger on the same Arabic datasets. However, further improvement of such analyzer-based tagging methods is hindered by the incomplete coverage of standard morphological analyzer (Bar Haim et al., 2005). To overcome this coverage problem we supplement the output of Buckwalter's analyzer with synthetically constructed analyses that are proposed by a model which uses character information (Diab et al., 2004) in a way that is similar to Nakagawa's (2004) system for Chinese and Japanese. A version of this extended model that (unlike Nakagawa) incorporates synthetically constructed analyses also for known words achieves 96.28% accuracy on the standard Arabic test set.

View PDFchevron_right
Exploring the Performance of Tagging for the Classical and the Modern Standard Arabic
Dia Eddin Abuzeina

Advances in Fuzzy Systems, 2019

The part of speech (PoS) tagging is a core component in many natural language processing (NLP) applications. In fact, the PoS taggers contribute as a preprocessing step in various NLP tasks, such as syntactic parsing, information extraction, machine translation, and speech synthesis. In this paper, we examine the performance of a modern standard Arabic (MSA) based tagger for the classical (i.e., traditional or historical) Arabic. In this work, we employed the Stanford Arabic model tagger to evaluate the imperative verbs in the Holy Quran. In fact, the Stanford tagger contains 29 tags; however, this work experimentally evaluates just one that is the VB ≡ imperative verb. The testing set contains 741 imperative verbs, which appear in 1,848 positions in the Holy Quran. Despite the previously reported accuracy of the Arabic model of the Stanford tagger, which is 96.26% for all tags and 80.14% for unknown words, the experimental results show that this accuracy is only 7.28% for the imper...

View PDFchevron_right
A morphological-syntactical analysis approach for Arabic textual tagging
Shihadeh Alqrainy

2008

Part-of-Speech (POS) tagging is the process of labeling or classifying each word in written text with its grammatical category or part-of-speech, ie noun, verb, preposition, adjective, etc. It is the most common disambiguation process in the field of Natural Language Processing ...

View PDFchevron_right
A review of morphosyntactic analysers and tag-sets for Arabic corpus linguistics
Eric Atwell

2015

Geoffrey Leech applied his expertise in English grammar to development of Part-of-Speech tagsets and taggers for English corpora, including LOB and BNC tagsets and tagged corpora. He also developed EAGLES standards for morphosyntactic tag-sets and taggers for European languages. We have extended this line of research to Arabic: we present a review of morphosyntactic analysers and tag-sets for Arabic corpus linguistics. The field of Arabic NLP has received a lot of contributions in the last decades. Many analysers handle its morphological-rich problem in Modern Standard Arabic text, and at least there are six freely available morphological analyzers at the time of writing this paper. However, the choice between these tools is challenging. In this extended abstract, we will discuss the outputs of these different tools. We show the challenge of comparing between them. The goal of this abstract is not to evaluate these tools but to show the differences. We aim also to ease the building ...

View PDFchevron_right
Creating Resources for Dialectal Arabic from a Single Annotation: A Case Study on Egyptian and Levantine
Ramy Nagah Eskander

International Conference on Computational Linguistics, 2016

Arabic dialects present a special problem for natural language processing because there are few Arabic dialect resources, they have no standard orthography, and they have not been studied much. However, as more and more written dialectal Arabic is found on social media, natural language processing for Arabic dialects has become an important goal. We present a methodology for creating a morphological analyzer and a morphological tagger for dialectal Arabic, and we illustrate it on Egyptian and Levantine Arabic. To our knowledge, these are the first analyzer and tagger for Levantine.

View PDFchevron_right
Statistical Part-of-Speech Tagger for Traditional Arabic Texts
yahya Elhadj

Journal of Computer Science, 2009

Problem statement: This study presented the development of an Arabic part-of-speech tagger that can be used for analyzing and annotating traditional Arabic texts, especially the Quran text. Approach: It is a part of a project related to the computerization of the Holy Quran. One of the main objectives in this project was to build a textual corpus of the Holy Quran. Results: Since an appropriate textual version of the Holy Quran was prepared and morphologically analyzed in other stages of this project, we focused in this work on its annotation by developing and using an appropriate tagger. The developed tagger employed an approach that combines morphological analysis with Hidden Markov Models (HMMs) based-on the Arabic sentence structure. The morphological analysis is used to reduce the size of the tags lexicon by segmenting Arabic words in their prefixes, stems and suffixes; this is due to the fact that Arabic is a derivational language. On another hand, HMM is used to represent the Arabic sentence structure in order to take into account the linguistic combinations. For these purposes, an appropriate tagging system has been proposed to represent the main Arabic part of speech in a hierarchical manner allowing an easy expansion whenever it is needed. Each tag in this system is used to represent a possible state of the HMM and the transitions between tags (states) are governed by the syntax of the sentence. A corpus of some traditional texts, extracted from Books of third century (Hijri), is manually morphologically analyzed and tagged using our developed tagset. Conclusion/Recommendations: It is then used for training and testing this model. Experiments conducted on this dataset gave a recognition rate of about 96% and thus are very promising compared to the data size tagged till now and used in the training. Since our Holy Quran corpus is still under revision, we did not make significant experiments on it. However, preliminary tests conducted on the seven verses of AL-Fatiha showed an encouraging accuracy rate.

View PDFchevron_right
Exploring and categorising the Arabic copula and auxiliary kāna via enhanced part-of- speech tagging
Wesam Ibrahim

2020

Arabic syntax has yet to be studied in detail from a corpus-based perspective. The Arabic copula kāna, ‘be’, functions additionally as an auxiliary, creating periphrastic tense-aspect constructions; but the literature on these functions is far from exhaustive. To analyse kāna within the million-word Leeds Corpus of Contemporary Arabic, part-of-speech tagging (using novel, targeted enhancements to a previously described program which improves the accessibility for linguistic analysis of the output of Habash et al.’s 2012 MADA disambiguator for the Buckwalter Arabic morphological analyser) is applied to disambiguate copula and auxiliary at a high rate of accuracy. Concordances of both are extracted, and 10% samples (499 instances of copula kāna, 387 of auxiliary kāna) are manually analysed to identify surface-level grammatical patterns and meanings. This raw analysis is then systematised according to the more general patterns’ main parameters of variation; special descriptions are dev...

View PDFchevron_right
Part of Speech Tagger for Tunisian Arabic: Comparing manual and ML methods for under-resourced languages
Karen McNeil

2019

UNPUBLISHED PAPER COMPARING THE HAND-ROLLED PARSER/POS TAGGER USED IN TUNISIYA.ORG WITH SOME ML METHODS This paper presents a comparison of several different part-of-speech taggers trained on a hand-annotated Tunisian Arabic sample of 6,000 words in 450 sentences. Despite the small size of the annotated corpus, the Trigram Tagger and Brill Tagger performed nearly as well as a custom rule-based tagger. This work not only contributes a new resource to Tunisian Arabic-a severely under-resourced language-but also provides insight for developing materials for other under-resourced languages.

View PDFchevron_right
Fine-Grain Morphological Analyzer and Part-of-Speech Tagger for Arabic Text
Majdi S Sawalha, Eric S Atwell

2010

Morphological analyzers and part-of-speech taggers are key technologies for most text analysis applications. Our aim is to develop a part-of-speech tagger for annotating a wide range of Arabic text formats, domains and genres including both vowelized and non-vowelized text. Enriching the text with linguistic analysis will maximize the potential for corpus re-use in a wide range of applications. We foresee the advantage of enriching the text with part-of-speech tags of very fine-grained grammatical distinctions, which reflect expert interest in syntax and morphology, but not specific needs of end-users, because end-user applications are not known in advance. In this paper we review existing Arabic Part-of-Speech Taggers and tag-sets, and illustrate four different Arabic PoS tag-sets for a sample of Arabic text from the Quran. We describe the detailed fine-grained morphological feature tag set of Arabic, and the fine-grained Arabic morphological analyzer algorithm. We faced practical challenges in applying the morphological analyzer to the 100-million-word Web Arabic Corpus: we had to port the software to the National Grid Service, adapt the analyser to cope with spelling variations and errors, and utilise a Broad-Coverage Lexical Resource combining 23 traditional Arabic lexicons. Finally we outline the construction of a Gold Standard for comparative evaluation.

View PDFchevron_right
Arabic Part-of-Speech Tagging
Rabab Ali

The study described in this paper belongs to the area of computational linguistics. Computational linguistics is a field of artificial intelligence dealing with the logical modeling of natural language from a computational perspective. It unites two areas that are quite different in appearance, computer science and natural languages. Computational linguistics might be considered as a synonym of automatic processing of natural language, since the main task of computational linguistics is just the construction of computer programs to process words and texts in natural language. There are many areas that may be considered as properly included within the discipline of computational linguistics. One of these areas is part-of-speech tagging (POS-tagging). POS-tagging is considered as a process for automatically assigning the proper grammatical tag to each word of a written text according to its appearance on the text. Thus, the task of POS-tagging is attaching appropriate grammatical or morpho-syntactical category labels to each word, token, symbol, abbreviation and even punctuation mark in a corpus. POS-tagging is usually the first step in linguistic analysis. Also, it is very important intermediate step to build many natural language processing applications. It could be used in spell checking and correcting systems, speech recognition systems, information retrieval systems and text-to-speech synthesis systems.

View PDFchevron_right

References (65)

  1. Abdelali, Ahmed, Kareem Darwish, Nadir Durrani, and Hamdy Mubarak. 2016. Farasa: A fast and furious segmenter for Arabic. In Proceedings of the 2016 Con- ference of the North American Chapter of the Associ- ation for Computational Linguistics: Demonstrations, pages 11-16, San Diego, California. Association for Computational Linguistics.
  2. Abdelali, Ahmed, Hamdy Mubarak, Younes Samih, Sabit Hassan, and Kareem Darwish. 2021. QADI: Ara- bic dialect identification in the wild. In Proceedings of the Sixth Arabic Natural Language Processing Work- shop, pages 1-10, Kyiv, Ukraine (Virtual). Association for Computational Linguistics.
  3. Ahmed, Mohamed AH. 2018. Xml annotation of hebrew elements in judeo-arabic texts. Journal of Jewish Lan- guages, 6(2):221-242.
  4. Altaher, Yousef, Ali Fadel, Mazen Alotaibi, Mazen Alyazidi, Mishari Al-Mutairi, Mutlaq Aldhbuiub, Ab- dulrahman Mosaibah, Abdelrahman Rezk, Abdulraz- zaq Alhendi, Mazen Abo Shal, et al. 2022. Masader plus: A new interface for exploring+ 500 arabic nlp datasets. arXiv preprint arXiv:2208.00932.
  5. Alyafeai, Zaid, Maraim Masoud, Mustafa Ghaleb, and Maged S. Al-shaibani. 2022. Masader: Metadata sourcing for Arabic text and speech data resources. In Northern European Journal of Language Technology Proceedings of the Thirteenth Language Resources and Evaluation Conference, pages 6340-6351, Marseille, France. European Language Resources Association.
  6. Attia, Mohammed, Younes Samih, Ali Elkahky, Hamdy Mubarak, Ahmed Abdelali, and Kareem Darwish. 2019. POS tagging for improving code-switching identification in Arabic. In Proceedings of the Fourth Arabic Natural Language Processing Workshop, pages 18-29, Florence, Italy. Association for Computational Linguistics.
  7. Ballesteros, Miguel, Chris Dyer, and Noah A. Smith. 2015. Improved transition-based parsing by model- ing characters instead of words with LSTMs. In Pro- ceedings of the 2015 Conference on Empirical Meth- ods in Natural Language Processing, pages 349-359, Lisbon, Portugal. Association for Computational Lin- guistics.
  8. Bar-Asher, Moshe. 1992. La composante hebraïque du judeo-arabe algerien: communautés de Tlemcen et Aïn- Témouchent. Magnes, Jerusalem.
  9. Belinkov, Yonatan. 2021. Large-scale electronic cor- pora and the study of middle and mixed Arabic. In Middle and Mixed Arabic over Time and across Writ- ten and Oral Genres: From Legal Documents to Tele- vision and Internet through Literature. Proceedings of the IVth AIMA International Conference (Emory Uni- versity, Atlanta, GA, USA, 12-15 October 2013), Pub- lications de l'Institut Orientaliste de Louvain, pages 43-67, Université catholique de Louvain, Louvain-la- Neuve. Peeters.
  10. Bouamor, Houda, Nizar Habash, Mohammad Salameh, Wajdi Zaghouani, Owen Rambow, Dana Abdul- rahim, Ossama Obeid, Salam Khalifa, Fadhl Eryani, Alexander Erdmann, et al. 2018. The MADAR Arabic dialect corpus and lexicon. In LREC.
  11. Caruana, Rich. 1997. Multitask learning. Machine Learning, 28.
  12. Çetinoğlu, Özlem, Sarah Schulz, and Ngoc Thang Vu. 2016. Challenges of computational processing of code-switching. In Proceedings of the Second Work- shop on Computational Approaches to Code Switching, pages 1-11, Austin, Texas. Association for Computa- tional Linguistics.
  13. Cohen, Marcel. 1912. Le parler arabe des Juifs d'Alger. Collection linguistique, pub. par la Société de linguis- tique de Paris-4. H. Champion, Paris.
  14. Darwish, Kareem, Mohammed Attia, Hamdy Mubarak, Younes Samih, Ahmed Abdelali, L. Màrquez, M. El- desouki, and Laura Kallmeyer. 2020. Effective multi- dialectal Arabic POS tagging. Natural Language En- gineering, 26:677 -690.
  15. Darwish, Kareem, Hamdy Mubarak, Ahmed Abde- lali, Mohamed Eldesouki, Younes Samih, Randah Al- harbi, Mohammed Attia, Walid Magdy, and Laura Kallmeyer. 2018. Multi-dialect Arabic POS tagging: A CRF approach. In Proceedings of the Eleventh Inter- national Conference on Language Resources and Eval- uation (LREC 2018), Miyazaki, Japan. European Lan- guage Resources Association (ELRA).
  16. Dermatas, Evangelos and George Kokkinakis. 1995. Au- tomatic stochastic tagging of natural language texts. Computational Linguistics, 21(2):137-163.
  17. Diab, Mona. 2009. Second generation AMIRA tools for Arabic processing: Fast and robust tokenization, POS tagging, and base phrase chunking. In In 2nd Interna- tional Conference on Arabic Language Resources and Tools.
  18. Dos Santos, Cicero and Bianca Zadrozny. 2014. Learning character-level representations for part-of- speech tagging. In International Conference on Ma- chine Learning, pages 1818-1826. PMLR.
  19. Duh, Kevin and Katrin Kirchhoff. 2005. POS tagging of dialectal Arabic: A minimally supervised approach. In Proceedings of the ACL Workshop on Computa- tional Approaches to Semitic Languages, pages 55-62, Ann Arbor, Michigan. Association for Computational Linguistics.
  20. El-Haj, Mahmoud. 2020. Habibi -a multi dialect multi national Arabic song lyrics corpus. In Proceedings of the 12th Language Resources and Evaluation Confer- ence, pages 1318-1326, Marseille, France. European Language Resources Association.
  21. El-Haj, Mahmoud and Rim Koulali. 2013. KALIMAT a multipurpose Arabic corpus. In Second workshop on Arabic corpus linguistics (WACL-2), pages 22-25.
  22. Ferguson, Charles A. 1959. Diglossia. WORD, 15(2):325- 340.
  23. Habash, Nizar, Mona Diab, and Owen Rambow. 2012. Conventional orthography for dialectal Arabic. In Proceedings of the Eighth International Conference on Language Resources and Evaluation (LREC'12), pages 711-718, Istanbul, Turkey. European Language Re- sources Association (ELRA).
  24. Habash, Nizar, Ryan Roth, Owen Rambow, Ramy Es- kander, and Nadi Tomeh. 2013. Morphological anal- ysis and disambiguation for dialectal Arabic. In Pro- ceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguis- tics: Human Language Technologies, pages 426-432, Atlanta, Georgia. Association for Computational Lin- guistics.
  25. Northern European Journal of Language Technology Habash, Nizar, Abdelhadi Soudi, and Timothy Buck- walter. 2007. On Arabic Transliteration, volume 38, chapter 2. Springer Netherlands, Dordrecht.
  26. Hajič, Jan, Otakar Smrž, Petr Zemánek, Petr Pajas, Jan Šnaidauf, Emanuel Beška, Jakub Kracmar, and Kamila Hassanová. 2009. Prague Arabic dependency treebank 1.0. LINDAT/CLARIAH-CZ digital library at the Institute of Formal and Applied Linguistics (ÚFAL), Faculty of Mathematics and Physics, Charles University.
  27. Hary, Benjamin. 2003. Judeo-Arabic: A diachronic reex- amination. International Journal of The Sociology of Language, 2003:61-75.
  28. Hochreiter, Sepp and Jürgen Schmidhuber. 1997. Long short-term memory. Neural computation, 9(8):1735- 1780.
  29. Inoue, Go, Salam Khalifa, and Nizar Habash. 2022. Morphosyntactic tagging with pre-trained language models for Arabic and its dialects. In Findings of the Association for Computational Linguistics: ACL 2022, pages 1708-1719, Dublin, Ireland. Association for Computational Linguistics.
  30. Inoue, Go, Hiroyuki Shindo, and Yuji Matsumoto. 2017. Joint prediction of morphosyntactic categories for fine-grained Arabic part-of-speech tagging exploit- ing tag dictionary information. In Proceedings of the 21st Conference on Computational Natural Language Learning (CoNLL 2017), pages 421-431, Vancouver, Canada. Association for Computational Linguistics.
  31. Kahn, Lily and Aaron D Rubin. 2017. Handbook of Jewish Languages: Revised and Updated Edition. Brill.
  32. Kim, Yoon, Yacine Jernite, David Sontag, and Alexan- der M. Rush. 2016. Character-aware neural language models. In Proceedings of the Thirtieth AAAI Con- ference on Artificial Intelligence, AAAI'16, page 2741- 2749. AAAI Press.
  33. Lafferty, John, Andrew Mccallum, and Fernando Pereira. 2001. Conditional random fields: Proba- bilistic models for segmenting and labeling sequence data. In Proceedings of the Eighteenth International Conference on Machine Learning, pages 282-289.
  34. Ling, Wang, Chris Dyer, Alan W Black, Isabel Trancoso, Ramón Fermandez, Silvio Amir, Luís Marujo, and Tiago Luís. 2015. Finding function in form: Compo- sitional character models for open vocabulary word representation. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Process- ing, pages 1520-1530, Lisbon, Portugal. Association for Computational Linguistics.
  35. Luong, Minh-Thang and Christopher D. Manning. 2016. Achieving open vocabulary neural machine translation with hybrid word-character models. In Proceedings of the 54th Annual Meeting of the Associ- ation for Computational Linguistics (Volume 1: Long Papers), pages 1054-1063, Berlin, Germany. Associa- tion for Computational Linguistics.
  36. Maamouri, Mohamed, Ann Bies, Tim Buckwalter, and Wigdan Mekki. 2004. The Penn Arabic treebank : Building a large-scale annotated Arabic corpus. In NEMLAR conference on Arabic language resources and tools, volume 27, pages 466-467. Cairo.
  37. Maamouri, Mohamed, Ann Bies, Seth Kulick, Michael Ciul, Nizar Habash, and Ramy Eskander. 2014. De- veloping an Egyptian Arabic treebank: Impact of di- alectal morphology on annotation and tool devel- opment. In Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC'14), pages 2348-2354, Reykjavik, Iceland. Eu- ropean Language Resources Association (ELRA).
  38. McCarthy, John J. 1981. A prosodic theory of non-concatenative morphology. Linguistic Inquiry, 12:373-418.
  39. Müller, Thomas, Helmut Schmid, and Hinrich Schütze. 2013. Efficient higher-order CRFs for morphological tagging. In Proceedings of the 2013 Conference on Em- pirical Methods in Natural Language Processing.
  40. Nivre, Joakim, Željko Agić, Maria Jesus Aranzabe, Masayuki Asahara, Aitziber Atutxa, Miguel Balles- teros, John Bauer, Kepa Bengoetxea, Riyaz Ahmad Bhat, Cristina Bosco, Sam Bowman, Giuseppe G. A. Celano, Miriam Connor, Marie-Catherine de Marn- effe, Arantza Diaz de Ilarraza, Kaja Dobrovoljc, Tim- othy Dozat, Tomaž Erjavec, Richárd Farkas, Jen- nifer Foster, Daniel Galbraith, Filip Ginter, Iakes Goenaga, Koldo Gojenola, Yoav Goldberg, Berta Gonzales, Bruno Guillaume, Jan Hajič, Dag Haug, Radu Ion, Elena Irimia, Anders Johannsen, Hiroshi Kanayama, Jenna Kanerva, Simon Krek, Veronika Laippala, Alessandro Lenci, Nikola Ljubešić, Teresa Lynn, Christopher Manning, Cătălina Mărănduc, David Mareček, Héctor Martínez Alonso, Jan Mašek, Yuji Matsumoto, Ryan McDonald, Anna Missilä, Verginica Mititelu, Yusuke Miyao, Simonetta Mon- temagni, Shunsuke Mori, Hanna Nurmi, Petya Osenova, Lilja Øvrelid, Elena Pascual, Marco Pas- sarotti, Cenel-Augusto Perez, Slav Petrov, Jussi Pi- itulainen, Barbara Plank, Martin Popel, Prokopis Prokopidis, Sampo Pyysalo, Loganathan Ramasamy, Rudolf Rosa, Shadi Saleh, Sebastian Schuster, Wolf- gang Seeker, Mojgan Seraji, Natalia Silveira, Maria Simi, Radu Simionescu, Katalin Simkó, Kiril Simov, Northern European Journal of Language Technology Aaron Smith, Jan Štěpánek, Alane Suhr, Zsolt Szántó, Takaaki Tanaka, Reut Tsarfaty, Sumire Uematsu, Larraitz Uria, Viktor Varga, Veronika Vincze, Zdeněk Žabokrtský, Daniel Zeman, and Hanzhi Zhu. 2015. Universal dependencies 1.2. LINDAT/CLARIAH-CZ digital library at the Institute of Formal and Ap- plied Linguistics (ÚFAL), Faculty of Mathematics and Physics, Charles University.
  41. Owens, J. 2013. The Oxford Handbook of Arabic Linguis- tics. Oxford Handbooks. Oxford University Press.
  42. Pasha, Arfath, Mohamed Al-Badrashiny, Mona Diab, Ahmed El Kholy, Ramy Eskander, Nizar Habash, Manoj Pooleery, Owen Rambow, and Ryan Roth. 2014. MADAMIRA: A fast, comprehensive tool for morphological analysis and disambiguation of Ara- bic. In Proceedings of the Ninth International Confer- ence on Language Resources and Evaluation (LREC'14), pages 1094-1101, Reykjavik, Iceland. European Lan- guage Resources Association (ELRA).
  43. Paszke, Adam, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang, Zachary DeVito, Martin Raison, Alykhan Te- jani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. 2019. Pytorch: An imperative style, high-performance deep learning li- brary. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché-Buc, E. Fox, and R. Garnett, editors, Ad- vances in Neural Information Processing Systems 32, pages 8024-8035. Curran Associates, Inc.
  44. Plank, Barbara, Anders Søgaard, and Yoav Goldberg. 2016. Multilingual part-of-speech tagging with bidi- rectional long short-term memory models and auxil- iary loss. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 412-418, Berlin, Germany. As- sociation for Computational Linguistics.
  45. Riabi, Arij, Benoît Sagot, and Djamé Seddah. 2021. Can character-based language models improve down- stream task performances in low-resource and noisy language scenarios? In Proceedings of the Sev- enth Workshop on Noisy User-generated Text (W-NUT 2021), pages 423-436, Online. Association for Com- putational Linguistics.
  46. Roth, Ryan, Owen Rambow, Nizar Habash, Mona Diab, and Cynthia Rudin. 2008. Arabic morphological tag- ging, diacritization, and lemmatization using lexeme models and feature ranking. In Proceedings of ACL- 08: HLT, Short Papers, pages 117-120, Columbus, Ohio. Association for Computational Linguistics.
  47. Seddah, Djamé, Farah Essaidi, Amal Fethi, Matthieu Futeral, Benjamin Muller, Pedro Javier Ortiz Suárez, Benoît Sagot, and Abhishek Srivastava. 2020. Build- ing a user-generated content North-African Arabizi treebank: Tackling hell. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 1139-1150, Online. Association for Computational Linguistics.
  48. Tedghi, Joseph. 2012. 'le livre de Jonas' traduit en judéo- arabe marocain par Samuel Malka: étude linguis- tique. In Dynamiques langagières en Arabophonies, pages 253-290, Zaragoza. Universidad de Zaragoza, Área de Estudios Árabes e Islámicos.
  49. Terner, Ori, Kfir Bar, and Nachum Dershowitz. 2020. Transliteration of Judeo-Arabic texts into Arabic script using recurrent neural networks. In Proceed- ings of the Fifth Arabic Natural Language Processing Workshop, pages 85-96, Barcelona, Spain (Online). Association for Computational Linguistics.
  50. Tirosh-Becker, Ofra. 1988. The phonology and top- ics in the morphology of a Judeo-Arabic translation of the book of Psalms from Constantine (Algeria) /
  51. אלג'יריה(‬ ‫מקונסטנטין‬ ‫בערבי��-יהודית‬ ‫תהילים‬ ‫לספר‬ ‫תרגום‬ ‫של‬ ‫במורפולוגיה‬ ‫ופרקים‬ ‫.פונולוגיה‬ Master's thesis, The He- brew University of Jerusalem.
  52. Tirosh-Becker, Ofra. 1989. On the linguistic uniformity in the "Šarḥ" of the Jews of Constantine / ‫קונסטנטין‬ ‫יהודי‬ ‫של‬ ‫בשרח‬ ‫הלשון‬ ‫אחדות‬ ‫.לשאלת‬ Proceedings of the World Congress of Jewish Studies ‫היהדות/‬ ‫למדעי‬ ‫העולמי‬ ‫הקונגרס‬ ‫,דברי‬ ‫י:791-402.‬
  53. Tirosh-Becker, Ofra. 2011a. Old and new in the transla- tion and commentary of Avot tractate / ‫ופירושה‬ ‫אבות‬ ‫משנת‬ ‫בתרגום‬ ‫וחדש‬ ‫.ישן‬ In Hikrei ma'arav u-mizrah: studies in language, literature and history presented to Joseph Chetrit (Volume 1) / (1 ‫כרך(‬ ‫שיטרית‬ ‫ליוסף‬ ‫מוגשים‬ ‫תולדה‬ ‫ופרקי‬ ‫ספרויות‬ ‫לשונות,‬ ‫ומזרח:‬ ‫מערב‬ ‫,חקרי‬ Hikrei ma'arav u-mizrah : studies in language, literature and history presented to Joseph Chetrit / ‫שיטרית‬ ‫ליוסף‬ ‫מוגשים‬ ‫תולדה‬ ‫ופרקי‬ ‫ספרויות‬ ‫לשונות,‬ ‫ומזרח:‬ ‫מערב‬ ‫.חקרי‬ Carmel.
  54. Tirosh-Becker, Ofra. 2011b. On dialectal roots in Judeo- Arabic texts from Constantine (east Algeria). Revue des Études Juives, 170:227-253.
  55. Tirosh-Becker, Ofra. 2011c. Terms for realia in an Alge- rian Judeo-Arabic translation of the Hošaʿnot. Studies in the Culture of North African Jewry, 1:171-186.
  56. Tirosh-Becker, Ofra. 2012. Mixed linguistic features in a Judeo-Arabic text from Algeria: The Šarḥ to the Hafṭarot from Constantine. In Language and Nature: Papers presented to John Huehnergard on the Occasion Northern European Journal of Language Technology of his 60th Birthday, pages 391-406, Chicago. Oxford University Press.
  57. Tirosh-Becker, Ofra. 2014. A reflection of a linguistic reality: An Algerian Judeo-Arabic book for the new year. Studies in the Culture of North African Jewry, 3:193-216.
  58. Tirosh-Becker, Ofra and Oren M. Becker. 2022. TAJA corpus: Linguistically tagged written Algerian Judeo- Arabic corpus. Journal of Jewish Languages, 10(1):24 -53.
  59. Wagner, Esther-Miriam and Magdalen Connolly. 2018. Code-switching in judaeo-arabic documents from the cairo geniza. Multilingua, 37(1):1-23.
  60. Zaidan, Omar and Chris Callison-Burch. 2014. Ara- bic dialect identification. Computational Linguistics, 40:171-202.
  61. Zalmout, Nasser and Nizar Habash. 2019. Adversarial multitask learning for joint multi-feature and multi- dialect morphological modeling. In Proceedings of the 57th Annual Meeting of the Association for Compu- tational Linguistics, pages 1775-1786, Florence, Italy. Association for Computational Linguistics.
  62. Zalmout, Nasser and Nizar Habash. 2020. Joint dia- critization, lemmatization, normalization, and fine- grained morphological tagging. In Proceedings of the 58th Annual Meeting of the Association for Computa- tional Linguistics, pages 8297-8307, Online. Associa- tion for Computational Linguistics.
  63. Zhang, Wen, Yang Feng, Fandong Meng, Di You, and Qun Liu. 2019. Bridging the gap between training and inference for neural machine translation. In Pro- ceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 4334-4343, Flo- rence, Italy. Association for Computational Linguis- tics.
  64. Zitouni, Imed. 2014. Natural Language Processing of Semitic Languages. Springer.
  65. Zribi, Inès, Mariem Ellouze, Lamia Belguith, and Philippe Blache. 2015. Spoken Tunisian Arabic cor- pus "STAC": Transcription and annotation. Research in Computing Science, 90.

FAQs

sparkles

AI

What were the results of the hierarchical chaR-cnn model for morphological tagging accuracy?add

The hierarchical chaR-cnn model achieved accuracies of 89% for analysis1 and 92.7% for analysis2, outperforming MarMoT's 82.3% and 85.6%, respectively.

How did the accuracy of the AJA tagger compare for Out of Vocabulary (OOV) words?add

AJATag's accuracy for morphologically tagging OOV words reached 74.91% for analysis1, significantly better than MarMoT's 55.82%.

What was the significant challenge addressed by this study regarding AJA linguistic analysis?add

The study tackled the difficulty of scaling linguistic analysis due to the rich morphology of AJA and decreasing speaker expertise.

How did the authors evaluate the practicality of their NLP models in a real-world setting?add

They conducted a user study with expert AJA linguists on unannotated texts from the NAJA corpus, scoring model accuracy against human annotations.

What method was used to assess inter-annotator agreement between expert linguists?add

Cohen's Kappa was calculated, yielding a score of 0.875, indicating high agreement between annotations from a senior expert and a junior expert.

About the author
Ofra Tirosh-Becker
The Hebrew University of Jerusalem, Faculty Member

https://ofratirosh-becker.huji.ac.il/

Papers
42
Followers
174
View all papers from Ofra Tirosh-Beckerarrow_forward

Related papers

TAJA Corpus: Linguistically Tagged Written Algerian Judeo-Arabic Corpus
Ofra Tirosh-Becker

Journal of Jewish Languages 10 (1): 24-53., 2022

The Tagged Algerian Judeo-Arabic (TAJA) corpus is the first linguistically annotated corpus of any Judeo-Arabic dialect regardless of geography and period. The corpus is a genre-diverse collection of written Modern Algerian Judeo-Arabic texts, encompassing translations of the Bible and of liturgical texts, commentaries and original Judeo-Arabic books and journals. The TAJA corpus was manually annotated with parts-of-speech (POS) tags and detailed morphology tags. The goal of the new corpus is twofold. First, it preserves this endangered Judeo-Arabic language, expanding on previous fieldwork and going beyond the study of individual written texts. The corpus has already enabled us to make strides towards a grammar of written Algerian Judeo-Arabic. Second, this tagged corpus serves as a foundation for the development of Judeo-Arabic-specific Natural Language Processing (NLP) tools, which allow automatic POS tagging and morphological annotation of large collections of yet untapped texts in Algerian Judeo-Arabic and other Judeo-Arabic varieties.

View PDFchevron_right
Tagging Classical Arabic Text using Available Morphological Analysers and Part of Speech Taggers
Eric S Atwell

Journal for Language Technology and Computational Linguistics, 2017

Focusing on Classical Arabic, this paper in its first part evaluates morphological analysers and POS taggers that are available freely for research purposes, are designed for Modern Standard Arabic (MSA) or Classical Arabic (CA), are able to analyse all forms of words, and have academic credibility. We list and compare supported features of each tool, and how they differ in the format of the output, segmentation, Part-of-Speech (POS) tags and morphological features. We demonstrate a sample output of each analyser against one CA fully-vowelized sentence. This evaluation serves as a guide in choosing the best tool that suits research needs. In the second part, we report the accuracy and coverage of tagging a set of classical Arabic vocabulary extracted from classical texts. The results show a drop in the accuracy and coverage and suggest an ensemble method might increase accuracy and coverage for classical Arabic. JLCL 2017-Band 32 (1)-1-26 brought to you by CORE View metadata, citation and similar papers at core.ac.uk provided by White Rose Research Online Alosaimy, Atwell Classical Arabic is the "liturgical" language that Muslims around the world use in religious practice. CA is also known as "Fussha" (the clearest), which Arabic Grammarians build their rules upon. One variant of CA is the Quranic Arabic, which is worded from CA, but differs in the sense that it is believed by Muslims to be the direct word of Allah. As time passes, different spoken variants of Classical Arabic emerged and people needed a standard form of communication: the Modern Standard Arabic (MSA). MSA is recognised as the formal and standard written Arabic. MSA is the language currently employed in media and education Bin-Muqbil (2006). Even though the morphology of MSA is inherited from CA, two studies showed that CA is not compatible with MSA taggers and vice versa. S. Rabiee (2011) tried to adapt several taggers by training them on a classical Arabic Corpus: the Quranic Arabic Corpus (QAC), and then tested them on MSA. The accuracy achieved in tagging a 66-word MSA sample was "not impressive", 73% was achieved. Alrabiah et al. (2014) compared MADA Habash et al. and AlKhalil Boudchiche et al. (2016) both designed for MSA in order to annotate the KSUCCA corpus. Using five samples from different genres of CA, an evaluation of these two systems showed a drop in their accuracy by 10-15%. This shows that current taggers need to be adapted for CA and their dictionaries need to include more classical vocabulary. We extend this evaluation to examine the coverage and accuracy of the surveyed tools. Next section reviews relevant work. The third and fourth sections list evaluated POS taggers and MAs in detail. The fifth section compares those tools by their features and demonstrates such differences on one tagged sentence. The last section reports the accuracy and coverage on a collection of classical vocabulary. 2. Related work Several previous studies surveyed the linguistic resources available for researchers in the field of Arabic NLP. Atwell et al. (2004) conducted a survey on the available MAs and came up with 10 different analysers. Authors concluded their survey pointing out that most of those analysers are not freely available or they are hard to use. Maegaard (2004) surveyed the state-of-art language resources including MAs and POS taggers. Basic Language Resource Kit (BLARK) project (2010) listed 7 MAs, three of which are commercial software. Sawalha (2011) listed 6 MAs with his proposal of a new fine-grained morphological analyser, three of which are freely available. Albared et al. (2009) surveyed the "POS tagging" techniques with a focus on Arabic: MSA and dialects. None was designed for classical Arabic. Those techniques were criticized as assuming closed-vocabulary which might not be the case with classical Arabic. Al-Sughaiyer and Al-Kharashi (2004) conducted a survey of Arabic "morphological analysis" techniques and classified the efforts in analysing Arabic morphology into four categories: tablelookup, linguistic (using finite state automaton or traditional grammar), combinatorial and pattern-based.

View PDFchevron_right
Arabic Tokenization, Part-of-Speech Tagging and Morphological Disambiguation in One Fell Swoop
Nizar Habash

2005

We present an approach to using a morphological analyzer for tokenizing and morphologically tagging (including partof-speech tagging) Arabic words in one process. We learn classifiers for individual morphological features, as well as ways of using these classifiers to choose among entries from the output of the analyzer. We obtain accuracy rates on all tasks in the high nineties.

View PDFchevron_right
A standard tag set expounding traditional morphological features for Arabic language part-of-speech tagging
Majdi S Sawalha, Eric S Atwell

2013

The SALMA Morphological Features Tag Set (SALMA, Sawalha Atwell Leeds Morphological Analysis tag set for Arabic) captures long-established traditional morphological features of grammar and Arabic, in a compact yet transparent notation. First, we introduce Part-of-Speech tagging and tag set standards for English and other European languages, and then survey Arabic Part-of-Speech taggers and corpora, and long-established Arabic traditions in analysis of morphology. A range of existing Arabic Part-of-Speech tag sets are illustrated and compared; and we review generic design criteria for corpus tag sets. For a morphologically-rich language like Arabic, the Part-of-Speech tag set should be defined in terms of morphological features characterizing word structure. We describe the SALMA Tag Set in detail, explaining and illustrating each feature and possible values. In our analysis, a tag consists of 22 characters; each position represents a feature and the letter at that location represents a value or attribute of the morphological feature; the dash '-' represents a feature not relevant to a given word. The first character shows the main Parts of Speech, from: noun, verb, particle, punctuation, and Other (residual); these last two are an extension to the traditional three classes to handle modern texts. 'Noun' in Arabic subsumes what are traditionally referred to in English as 'noun' and 'adjective'. The characters 2, 3, and 4 are used to represent subcategories; traditional Arabic grammar recognizes 34 subclasses of noun (letter 2), 3 subclasses of verb (letter 3), 21 subclasses of particle (letter 4). Others (residuals) and punctuation marks are represented in letters 5 and 6 respectively. The next letters represent traditional morphological features: gender (7), number (8), person (9), inflectional morphology (10) case or mood (11), case and mood marks (12), definiteness (13), voice (14), emphasized and non-emphasized (15), transitivity (16), rational (17), declension and conjugation (18). Finally there are four characters representing morphological information which is useful in Arabic text analysis, although not all linguists would Word Structure 6.1 (2013): 43-99 count these as traditional features: unaugmented and augmented (19), number of root letters (20), verb root (21), types of nouns according to their final letters (22). The SALMA Tag Set is not tied to a specific tagging algorithm or theory, and other tag sets could be mapped onto this standard, to simplify and promote comparisons between and reuse of Arabic taggers and tagged corpora.

View PDFchevron_right
Semi-automatic Annotation of Arabic Corpus: A Morpho-syntactic Study
Mohammad Mahyoob

2022

The most important objective of our study was to build and construct a complete and comprehensive morphological analyzing scheme, tagging, and parsing system which can be used for annotating Arabic corpora. In our dissertation, we did an analytical study, implementation, and evaluation of Arabic morphological analysis, tagging, and syntactic analysis starting from raw text. The three different systems were implemented in various methods; for morphological analysis, we use finite-state automaton as discussed in chapter four, after doing the process of tokenization and segmentation of the raw text as explained in chapter three. The tagging system was implemented under a new very rich tag set, which was designed and developed by us. It consists of 30 tags in addition to some other features and linguistic information as described in chapter five. An appropriate set of tags has a direct influence on the accuracy and the usefulness of tagging system. So, the smaller the tag set, the highe...

View PDFchevron_right

Related topics

  • Computer Science
  • Artificial Intelligence
  • Natural Language Processing
    • 580 California St., Suite 400
    San Francisco, CA, 94104
    © 2026 Academia. All rights reserved