Befunde aus dem nationalen Bildungsmonitoring ÉpStan vor dem Hintergrund der COVID-19 Pandemie (Supplement)

in LUCET; SCRIPT (Eds.) Nationaler Bildungsbericht Luxemburg 2021 (2021)

Lehrkräfte als maßgebliche Akteur*innen beim Einsatz von Informations- und Kommunikationstechnologie (IKT) in Lern- und Lehrprozessen – Zentrale Ergebnisse der ICILS 2018

in LUCET; SCRIPT (Eds.) Nationaler Bildungsbericht Luxemburg 2021 (2021)

Computer- und informationsbezogene Kompetenzen (CIL) und Kompetenzen im informatischen Denken (CT) von Achtklässler*innen – Zentrale Ergebnisse der ICILS 2018

in LUCET; SCRIPT (Eds.) Nationaler Bildungsbericht Luxemburg 2021 (2021)

Résultats longitudinaux récents issus du monitoring scolaire national ÉpStan en troisième et neuvième année scolaire : de moins bons résultats et des redoublements inefficaces

in LUCET; SCRIPT (Eds.) Rapport national sur l’éducation au Luxembourg 2021 (2021)

Befunde aus dem nationalen Bildungsmonitoring ÉpStan vor dem Hintergrund der COVID-19- Pandemie

in LUCET; SCRIPT (Eds.) Nationaler Bildungsbericht Luxemburg 2021 (2021)

Stability of Primary School Value-Added Scores over Time: A Comparison Between Math and Language Achievement as Outcome Variables

Scientific Conference (2021, November)

Value-added (VA) models are widely used for accountability purposes in education. Tracking a teacher’s or a school’s VA score over time forms oftentimes the basis for high-stakes decision-making and can ... [more ▼]

Value-added (VA) models are widely used for accountability purposes in education. Tracking a teacher’s or a school’s VA score over time forms oftentimes the basis for high-stakes decision-making and can determine whether teachers can keep their jobs or schools may receive certain funding. Despite their high-stakes application, the stability of VA scores over time has not yet been investigated for primary schools. Moreover, it is unclear whether different outcome measures (e.g., language and mathematics) may differ in their stability over time. In the present study, we aimed to clarify the stability of VA scores over time and investigate the differences across outcome variables. Furthermore, we wanted to showcase the real-life implications of (in)stable VA scores for single schools, with a focus on an informative use of VA scores rather than an evaluative way. The exploration of school VA scores in primary schools is especially relevant for heterogeneous student populations, for instance, in Luxembourg. Thus, we drew on representative longitudinal data from the standardized achievement tests of the Luxembourg School Monitoring Programme and examined the stability of school VA scores over two years in 146 schools (N = 7016 students). The overall stability, as measured by correlation coefficients, was moderate with r = .37 for VA scores in language and r = .34 for VA scores in mathematics from grade one to grade three. Real-life implications for schools will be discussed. [less ▲]

Child–Computer Interaction: From a systematic review towards an integrated understanding of interaction design methods for children

in International Journal of Child - Computer Interaction (2021), 100398

Child–Computer Interaction (CCI) is a steadily growing field that focuses on children as a prominent and emergent user group. For more than twenty years, the Interaction Design for Children (IDC ... [more ▼]

Child–Computer Interaction (CCI) is a steadily growing field that focuses on children as a prominent and emergent user group. For more than twenty years, the Interaction Design for Children (IDC) community has developed, extended, and advanced research and design methods for children’s involvement in designing and evaluating interactive technologies. However, as the CCI field evolves, the need arises for an integrated understanding of interaction design methods currently applied. To that end, we analyzed 272 full papers across a selection of journals and conference venues from 2005 to 2020. Our review contributes to the literature on this topic by (1) examining a holistic child population, including developmentally diverse children and children from 0 to 18 years old, (2) illustrating the interplay of children’s and adults’ roles across different methods, and (3) identifying patterns of triangulation in the methods applied while taking recent ethical debates about children’s involvement in design into account. While we found that most studies were conducted in natural settings, we observed a preference for evaluating interactive artifacts at a single point in time. Method triangulation was applied in two-thirds of the papers, with a preference for qualitative methods. Researchers used triangulation predominantly with respect to mainstream methods that were not specifically developed for child participants, such as user observation combined with semi-structured interviews or activity logging. However, the CCI field employs a wide variety of creative design methods which engage children more actively in the design process by having them take on roles such as informant and design partner. In turn, we see that more passive children’s roles, e.g., user or tester, are more often linked to an expert mindset by the adult. Adults take on a wider spectrum of roles in the design process when addressing specific developmental groups, such as children with autism spectrum disorder. We conclude with a critical discussion about the constraints involved in conducting CCI research and discuss implications that can inform future methodological advances in the field and underlying challenges. [less ▲]

Stability of Value-Added Models: Comparing Classical and Machine Learning Approaches

Scientific Conference (2021, September)

Background: What is the value that teachers or schools add to the evolution of students’ performance? Value-added (VA) modeling aims to answer this question by quantifying the effect of pedagogical ... [more ▼]

Background: What is the value that teachers or schools add to the evolution of students’ performance? Value-added (VA) modeling aims to answer this question by quantifying the effect of pedagogical actions on students’ achievement, independent of students’ backgrounds (e.g., Braun, 2005). A plethora of VA models exist, and several outcome measures are in use to estimate VA scores, yet without consensus on the model specification (Everson, 2017; Levy et al., 2019). Furthermore, it is unclear whether the most frequently used VA models (i.e., multi-level, linear regression, and random forest models) and outcome measures (i.e., language and mathematics achievement) indicate a similar stability of VA scores over time. Objectives: Drawing from the data of a highly diverse and multilingual school setting, where leveling out the influence of students’ backgrounds is of special interest, we aim to (a) clarify the stability of school VA scores over time; (b) shed light on the sensitivity toward different statistical models and outcome variables; and (c) evaluate the practical implications of (in)stable VA scores for individual schools. Method: Utilizing the representative, longitudinal data from the Luxembourg School Monitoring Programme (LUCET, 2021), we examined the stability of school VA scores. We drew on two longitudinal data sets of students who participated in the standardized achievement tests in Grade 1 in 2014 or 2016 and then again in Grade 3 two years later (i.e., 2016 and 2018, respectively), with a total of 5875 students in 146 schools. School VA scores were calculated using classical approaches (i.e., linear regression and multilevel models) and one of the most commonly used machine learning approaches in educational research (i.e., random forests). Results and Discussion: The overall stability over time across the VA models was moderate, with multilevel models showing greater stability than linear regression models and random forests. Stability differed across outcome measures and was higher for VA models with language achievement as an outcome variable as compared to those with mathematics achievement. Practical implications for schools and teachers will be discussed. [less ▲]

Knowledge assessment with concept maps: Opportunities and challenges

Scientific Conference (2021, July)

21st-century digital society poses tremendous challenges for education and assessment. Learners have to understand the complex relations between diverse topics and learn how to learn their entire lives ... [more ▼]

21st-century digital society poses tremendous challenges for education and assessment. Learners have to understand the complex relations between diverse topics and learn how to learn their entire lives. Concept mapping is a promising approach to address these issues. It is a method that uses concepts connected by labeled links to visualize a semantic network of knowledge. Concept mapping is predestined for a digital approach because it allows for easy interactive editing, innovative test items, and incorporation of multimodal information. Concept mapping is available for summative and formative assessment and, thus, provides the opportunity to become a vital part of modern education. The biggest advantage of concept mapping (i.e., a comprehensive and yet comprehensible visualization of complex relations) also represents the biggest challenge when it comes to assessment with - and scoring of - concept maps. The first challenge is the enormous amount of indicators used for scoring concept maps in assessment. A second challenge comes from the fact that educators using concept mapping in their assessment have to understand and interpret the indicators that are used in scoring concept maps. This presentation reports on a Ph.D. project that investigates digital concept mapping in the context of knowledge assessment from a user experience perspective. The results are based on, first, a comprehensive international systematic literature review on concept map scoring, and second, three empirical studies covering the needs and experiences of learners and educators in concept mapping. It presents key findings from the iterative user experience design of a concept mapping tool as part of the online assessment platform OASYS, an overview of indicators used in concept map scoring, and research opportunities in knowledge assessment with concept maps. Finally, it stresses the value that user experience design brings to knowledge assessment with concept maps. [less ▲]

Social Inequality in Education: Academic Achievement of First-, Second-, and Later-Generation Immigrant Students in Luxembourg

Scientific Conference (2021, April 08)

Students with immigrant backgrounds are often disadvantaged in public educational systems. In Luxembourg, about 50% of primary and secondary school students have an immigrant background, most notably from ... [more ▼]

Students with immigrant backgrounds are often disadvantaged in public educational systems. In Luxembourg, about 50% of primary and secondary school students have an immigrant background, most notably from Italy, the former Yugoslavia and Portugal. Using data from Luxembourg’s national school monitoring program, we investigate and document for the first time, existing and emerging differences in academic achievement among different immigrant generations of students. Our results indicate that student achievement in Math, German and French is differentially affected by immigrant generational status and language spoken at home. In addition, we find secondary effects of student social background. [less ▲]