Comparing Pre-Training Schemes for Luxembourgish BERT Models

in Proceedings of the 19th Conference on Natural Language Processing (KONVENS 2023) (2023, September)

Despite the widespread use of pre-trained models in NLP, well-performing pre-trained models for low-resource languages are scarce. To address this issue, we propose two novel BERT models for the ... [more ▼]

Despite the widespread use of pre-trained models in NLP, well-performing pre-trained models for low-resource languages are scarce. To address this issue, we propose two novel BERT models for the Luxembourgish language that improve on the state of the art. We also present an empirical study on both the performance and robustness of the investigated BERT models. We compare the models on a set of downstream NLP tasks and evaluate their robustness against different types of data perturbations. Additionally, we provide novel datasets to evaluate the performance of Luxembourgish language models. Our findings reveal that pre-training a pre-loaded model has a positive effect on both the performance and robustness of fine-tuned models and that using the German GottBERT model yields a higher performance while the multilingual mBERT results in a more robust model. This study provides valuable insights for researchers and practitioners working with low-resource languages and highlights the importance of considering pre-training strategies when building language models. [less ▲]

46. Sociolinguistics in Luxembourg

in Ball, Martin; Mesthrie, Rajend; Meluzzi, Chiara (Eds.) The Routledge Handbook of Sociolinguistics around the World (2023)

Mit Lingscape auf Pad in der Stadt. Ein Schulprojekt zu gesellschaftlicher Mehrsprachigkeit in Windhoek

in Marten, Heiko; Ziegler, Evelyn (Eds.) Linguistic Landscapes im deutschsprachigen Kontext: Forschungsperspektiven, Methoden und Anwendungsmöglichkeiten im Unterricht und Sprachmarketing (2021)

Cultural representation in Luxembourgish street naming practices

in Linguistics Vanguard (2021), 7(s5), 111

Crowdscapes. Participatory research and the collaborative (re)construction of linguistic landscapes with Lingscape

in Linguistics Vanguard (2021), 7(s1),

Findings of the VarDial Evaluation Campaign 2021

in Proceedings of the Eighth Workshop on NLP for Similar Languages, Varieties and Dialects (2021)

Exploring the Linguistic Landscape of Cities through Crowdsourced Data

in Brunn, Stanley; Kehrein, Roland (Eds.) Handbook of the Changing World Language Map (2020)

Attitudes towards multilingualism in Luxembourg. A comparative analysis of online news comments and crowdsourced questionnaire data

in Frontiers in Artificial Intelligence (2020), (3), 536086

Luxemburgisch zwischen Variation und Normierung. Ein Einwurf

Article for general public (2019)

Lörres, Möppes, and the Swiss. (Re)Discovering Regional Patterns in Anonymous Social Media Data

in Journal of Linguistic Geography (2019), 7(3), 113-134