![]() Emslander, Valentin ![]() E-print/Working paper (2023) Teacher-student relationships (TSRs) play a vital role in establishing a positive school climate and promoting positive student outcomes. Several meta-analyses have suggested significant associations ... [more ▼] Teacher-student relationships (TSRs) play a vital role in establishing a positive school climate and promoting positive student outcomes. Several meta-analyses have suggested significant associations between TSRs and, for example, academic achievement, a lack of disruptive behavior, school engagement, peer relationships, motivation, executive functions, and general well-being. However, these meta-analyses have differed substantially in TSR-outcome relationships, moderators, and quality, thus complicating the interpretation of these findings. In this preregistered systematic review of meta-analyses plus original second-order meta-analyses (SOMAs), we aimed to (a) synthesize the meta-analytic evidence on relationships between TSRs and student outcomes, (b) map influential moderators of these relationships, and (c) assess the methodological quality of the meta-analyses. We synthesized over 70 years of educational research in 24 meta-analyses encompassing a total of 116 effect sizes based on more than 2 million prekindergarten and K-12 students. We conducted several three-level SOMAs and found that TSRs had similar strong significant relationships with eight clusters of outcomes: academic achievement, academic emotions, appropriate student behavior, behavior problems, executive functions and self-control, motivation, school belonging and engagement, and student well-being. Age, gender, and informant (student-, peer-, or teacher-assessments) were the most frequently examined moderators in prior research, and our moderator analyses suggested student grade level and social minority status as moderators. We further found large differences in quality between the meta-analyses, and these differences were not associated with the TSR-outcome relationships. These results map the field of TSR research; present their relationships, moderators, and meta-analytic quality; and show how TSRs can contribute to improving outcomes in students via relationship building. Future research should follow meta-analytic open science procedures to improve quality and reproducibility. [less ▲] Detailed reference viewed: 51 (1 UL)![]() ![]() Emslander, Valentin ![]() ![]() Scientific Conference (2023, August 25) Theoretical background School climate is a key construct with great potential to impact student outcomes. The construct is multidimensional and includes, for instance, academic, community, safety, and ... [more ▼] Theoretical background School climate is a key construct with great potential to impact student outcomes. The construct is multidimensional and includes, for instance, academic, community, safety, and institutional environment aspects (Wang & Degol, 2016). While the dimensions may vary, researchers widely agree that teacher-student relationships play a vital role in establishing a positive school climate (Wang et al., 2020). Their role can be explained by Bronfenbrenner's (1979) bioecological theory identifying the driver of human development as the interaction with the persons in our closest (proximal) environment. Thus, in a school setting, emotional warmth and closeness or conflict and dependence in teacher-student relationships should also be associated with positive/negative student outcomes. Several meta-analyses uncovered significant associations between teacher-student relationships and school engagement, good peer relationships, executive functioning, well-being, and reductions in aggressive or disruptive behaviors (Endedijk et al., 2021; Nurmi, 2012; Roorda et al., 2011; Vandenbroucke et al., 2018). However, these meta-analyses differed in their methods and substantive findings. Moreover, the extant literature is ambiguous about which moderators (e.g., age) influence these relationships. Furthermore, the reporting and quality of meta-analyses in this field vary considerably, which can compromise the reliability and validity of their findings. Aims Given these research gaps, we systematically searched and reviewed the meta-analytic literature (Cooper & Koenka, 2012) to provide an overview of correlations between teacher-student relationships and student outcomes. In doing so, we examined three research questions: 1. To what extent are academic, behavioral, socio-emotional, motivational, and cognitive student outcomes associated with teacher-student relationships in the meta-analytic literature? 2. Which moderators influence these associations? 3. What is the methodological quality of the included meta-analyses? Methodology After preregistration, a systematic literature search was conducted. During several screening rounds, we identified 24 appropriate meta-analyses that included approximately meta-analytic 130 effect sizes for over one million students. From these meta-analyses, we extracted effect sizes on the association between teacher-student relationships and academic, behavioral, socio-emotional, motivational, and general cognitive student characteristics. We summarized the results for research questions 1 and 2 and developed a narrative overview. For research question 3, we assessed the quality of the meta-analyses using the AMSTAR-2 scale (adapted to correlational studies in psychology and education research; Shea et al., 2017). Findings and their significance Looking at the teacher-student relationship aspect of school climate, a variety of outcome variables were analyzed. The strongest associations were shown for negative teacher-student relationships with student behavior problems (r = .35 bis .57; Nurmi, 2012). Positive teacher-student relationships showed the strongest association with school involvement (r = .26 bis .34; Roorda et al., 2011), prosocial, externalizing, and internalizing behaviors (r = .25; Endedijk et al., 2021), and learning motivation combined with student involvement (r = .23; Wang et al., 2020). Age and grade level were the most frequently examined moderators, with partially contradicting findings. Gender differences, on the other hand, were found less frequently. At the same time, an informant effect was frequently examined, that is, whether and in what ways teachers, student peers, or the students themselves rated the teacher-student relationship. For research question 3, we discuss differences in reporting and the quality range of meta-analyses. With this preregistered systematic review of meta-analyses, we summarize the research landscape on correlates of the teacher-student relationship aspect of school climate. Following our findings and the bioecological theory, teachers should be made aware of the impact of teacher-student relationships and how they could contribute to a positive school climate via relationship building. Some interventions to improve these important relationships have already been meta-analytically studied with promising results (Kincade et al., 2020). Next, we need experiments to causally confirm positive teacher-student relationships as an effective strategy for improving academic, behavioral, socio-emotional, motivational, and cognitive student outcomes and school climate at large. Finally, future research should structure the broad range of conceptualizations of teacher-student relationships and review the variety of theories to explain their impact on student outcomes. References Bronfenbrenner, U. (1979). The ecology of human development: Experiments by nature and design. Harvard university press. Cooper, H., & Koenka, A. C. (2012). The overview of reviews: Unique challenges and opportunities when research syntheses are the principal elements of new integrative scholarship. American Psychologist, 67(6), 446–462. https://doi.org/10.1037/a0027119 Endedijk, H. M., Breeman, L. D., van Lissa, C. J., Hendrickx, M. M. H. G., den Boer, L., & Mainhard, T. (2021). The Teacher’s Invisible Hand: A Meta-Analysis of the Relevance of Teacher–Student Relationship Quality for Peer Relationships and the Contribution of Student Behavior. Review of Educational Research, 003465432110514. https://doi.org/10.3102/00346543211051428 Kincade, L., Cook, C., & Goerdt, A. (2020). Meta-Analysis and Common Practice Elements of Universal Approaches to Improving Student-Teacher Relationships. Review of Educational Research, 90(5), 710–748. https://doi.org/10.3102/0034654320946836 Nurmi, J.-E. (2012). Students’ characteristics and teacher–child relationships in instruction: A meta-analysis. Educational Research Review, 7(3), 177–197. https://doi.org/10.1016/j.edurev.2012.03.001 Roorda, D. L., Koomen, H. M. Y., Spilt, J. L., & Oort, F. J. (2011). The Influence of Affective Teacher–Student Relationships on Students’ School Engagement and Achievement: A Meta-Analytic Approach. Review of Educational Research, 81(4), 493–529. https://doi.org/10.3102/0034654311421793 Shea, B. J., Reeves, B. C., Wells, G., Thuku, M., Hamel, C., Moran, J., Moher, D., Tugwell, P., Welch, V., Kristjansson, E., & Henry, D. A. (2017). AMSTAR 2: A critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both. BMJ, j4008. https://doi.org/10.1136/bmj.j4008 Vandenbroucke, L., Spilt, J., Verschueren, K., Piccinin, C., & Baeyens, D. (2018). The Classroom as a Developmental Context for Cognitive Development: A Meta-Analysis on the Importance of Teacher–Student Interactions for Children’s Executive Functions. Review of Educational Research, 88(1), 125–164. https://doi.org/10.3102/0034654317743200 Wang, M.-T., & Degol, J. L. (2016). School Climate: A Review of the Construct, Measurement, and Impact on Student Outcomes. Educational Psychology Review, 28(2), 315–352. https://doi.org/10.1007/s10648-015-9319-1 Wang, M.-T., L. Degol, J., Amemiya, J., Parr, A., & Guo, J. (2020). Classroom climate and children’s academic and psychological wellbeing: A systematic review and meta-analysis. Developmental Review, 57, 100912. https://doi.org/10.1016/j.dr.2020.100912 [less ▲] Detailed reference viewed: 81 (3 UL)![]() ; Emslander, Valentin ![]() E-print/Working paper (2023) Evaluating the quality of primary studies is a key step in meta-analytic reviews to reduce the risk of bias and establish the validity of the meta-analytic inferences. However, the extant body of research ... [more ▼] Evaluating the quality of primary studies is a key step in meta-analytic reviews to reduce the risk of bias and establish the validity of the meta-analytic inferences. However, the extant body of research offers little guidance on how to represent and incorporate primary study quality (PSQ) in meta-analyses, and some common procedures, such as creating sum scores from a set of quality indicators, often lack the backing from measurement models. Addressing these issues, we present a tutorial that guides meta-analysts in their analytic decisions and approaches to represent and incorporate PSQ. Specifically, we describe, review, and illustrate approaches to (a) select or create quality indicators or scores a priori or as part of the meta-analytic model; (b) examine the possible moderator effects of PSQ; and (c) test the sensitivity of moderator effects to PSQ. We illustrate these approaches with three examples and present a step-by-step tutorial with analytic code for researchers’ guidance. Overall, we argue for representing PSQ model-based if multiple quality indicators are available, the testing of moderator effects of PSQ on the effect sizes and their heterogeneity, and performing moderator sensitivity analyses. [less ▲] Detailed reference viewed: 96 (4 UL)![]() ![]() Emslander, Valentin ![]() ![]() Scientific Conference (2023, March 01) THEORETISCHER HINTERGRUND Gute Beziehungen zur eigenen Lehrerin können sich positiv auf den Erfolg eines Schülers auswirken. Dieser Effekt kann mit Bowlby’s (1982) Bindungstheorie erklärt werden und wird ... [more ▼] THEORETISCHER HINTERGRUND Gute Beziehungen zur eigenen Lehrerin können sich positiv auf den Erfolg eines Schülers auswirken. Dieser Effekt kann mit Bowlby’s (1982) Bindungstheorie erklärt werden und wird empirisch immer wieder gestützt (z.B. Hamre & Pianta, 2001). Positive Lehrer-Schüler-Beziehungen zeichnen sich durch emotionale Wärme und Nähe aus; negative Aspekte durch Konflikt und Abhängigkeit. So stehen positive Lehrer-Schüler-Beziehungen nicht nur mit akademischen Leistungen in Verbindung, sondern auch mit einer Vielzahl anderer, wünschenswerter Schülerentwicklungen. Zahlreiche Meta-Analysen deuten auf signifikante Zusammenhänge zwischen Lehrer-Schüler-Beziehungen und schulischem Engagement, guten Beziehungen zu Gleichaltrigen, exekutiven Funktionen, allgemeinem Wohlbefinden und der Verringerung aggressiver oder störender Verhaltensweisen hin (Endedijk et al., 2021; Nurmi, 2012; Roorda et al., 2017; Vandenbroucke et al., 2018). Diese Befunde sind jedoch weit verstreut in der Literatur, sodass Forschungslücken unentdeckt bleiben. Auch unterscheiden sich bisherige Überblicksarbeiten in ihren Methoden und den gefundenen Zusammenhängen zwischen Lehrer-Schüler-Beziehungen und Ergebnisvariablen von Schüler*innen. Darüber hinaus ist die Literatur uneindeutig, welche Moderatoren (z.B. Alter oder Geschlecht) diese Beziehungen beeinflussen. Gleichzeitig variiert die Qualität der Meta-Analysen in diesem Feld merklich, was die Interpretation ihrer Ergebnisse erschweren kann. FRAGESTELLUNG Angesichts dieser Forschungslücken haben wir die meta-analytische Literatur systematisch durchsucht und zusammengefasst (Cooper & Koenka, 2012), um einen Überblick über Korrelate von Lehrer-Schüler-Beziehungen zu schaffen. Hierbei untersuchten wir drei Forschungsfragen 1. Inwieweit hängen akademische, verhaltensbezogene, sozio-emotionale, motivationale und kognitive Schülereigenschaften mit Lehrer-Schüler-Beziehungen in der meta-analytischen Literatur zusammen? 2. Welche Moderatoren beeinflussen diese Zusammenhänge? 3. Welche methodische Qualität haben die einbezogenen Meta-Analysen? METHODE Um diese Forschungsfragen zu beantworten, analysierten wir 24 Meta-Analysen, die rund 130 Effektstärken für über eine Million Schüler*innen umfassten. Nach der Präregistrierung erfolgte eine systematische Literatursuche. Während mehrerer Runden der Überprüfung mithilfe unserer Ein- und Ausschlusskriterien identifizierten wir 24 passende Meta-Analysen. Aus diesen Meta-Analysen extrahierten wir die Effektstärken zum Zusammenhang von Lehrer-Schüler-Beziehungen und akademische, verhaltensbezogene, sozio-emotionale, motivationale und allgemeine kognitive Schülereigenschaften. Für die Forschungsfragen 1 und 2 haben wir die Ergebnisse zusammengefasst und einen narrativen Überblick erarbeitet. Für Forschungsfrage 3 bewerteten wir die Qualität der Meta-Analysen mit Hilfe der AMSTAR-2 Skala (angepasst an korrelative Studien in der Psychologie und Bildungsforschung; Shea et al., 2017). ERGEBNISSE UND IHRE BEDEUTUNG Mit Blick auf die Lehrer-Schüler-Beziehungen werden unterschiedliche Ergebnisvariablen analysiert (Forschungsfrage 1). Die stärksten Zusammenhänge zeigten sich für Konflikt und Abhängigkeit in der Lehrer-Schüler-Beziehung mit Verhaltensproblemen der Schüler*innen (r = .35 bis .57; Nurmi, 2012). Positive Lehrer-Schüler-Beziehungen zeigte die stärkste Verbindung mit der Beteiligung in der Schule (r = .26 bis .34; Roorda et al., 2011), prosozialem, externalisierendem und internalisierendem Verhalten (r = .25; Endedijk et al., 2021) sowie mit Lernmotivation in Kombination mit Beteiligung der Schüler*innen (r = .23; Wang et al., 2020). Alter oder Klassenstufe waren die am häufigsten untersuchten Moderatoren mit teilweise gegenläufigen Befunden (Forschungsfrage 2). Geschlechterunterschiede wurden dagegen seltener festgestellt. Gleichzeitig wurde der Effekt der Informationsquelle häufig untersucht, d.h., ob und auf welche Weise Lehrkräfte, Gleichaltrige oder die Schüler*innen selbst die Lehrer-Schüler-Beziehung bewerteten. Für Forschunsgfrage 3 diskutieren wir die Qualitätsunterschiede der Meta-Analysen. Mit dem systematischen Review von Meta-Analysen fassen wir die Forschungslandschaft zu Korrelaten von Lehrer-Schüler-Beziehungen zusammen und zeigen, in welchem Zusammenhang diese mit Lehrer-Schüler-Beziehungen stehen. Unseren Ergebnissen folgend sollten Lehrkräfte für die Wirkung von Lehrer-Schüler-Beziehungen und deren Zusammenhängen sensibilisiert werden. Einige Interventionen zur Verbesserung von dieser wichtigen Beziehungen wurden bereits meta-analytisch mit vielversprechende Ergebnissen untersucht (Kincade et al., 2020). Ein nächster Schritt ist nun die experimentelle Überprüfung der gefundenen Korrelate, um positive Lehrer-Schüler-Beziehungen als wirksame Strategie zur Verbesserung von akademischen, verhaltensbezogenen, sozio-emotionalen, motivationalen und kognitiven Schülereigenschaften kausal zu bestätigen. LITERATUR Bowlby, J. (1982). Attachment and loss: Vol. 1. Attachment. (2nd ed., Vol. 1). Basic Books. Cooper, H., & Koenka, A. C. (2012). The overview of reviews: Unique challenges and opportunities when research syntheses are the principal elements of new integrative scholarship. American Psychologist, 67(6), 446–462. https://doi.org/10.1037/a0027119 Decristan, J., Kunter, M., & Fauth, B. (2022). Die Bedeutung individueller Merkmale und konstruktiver Unterstützung der Lehrkraft für die soziale Integration von Schülerinnen und Schülern im Mathematikunterricht der Sekundarstufe. Zeitschrift für Pädagogische Psychologie, 36(1–2), 85–100. https://doi.org/10.1024/1010-0652/a000329 Endedijk, H. M., Breeman, L. D., van Lissa, C. J., Hendrickx, M. M. H. G., den Boer, L., & Mainhard, T. (2021). The Teacher’s Invisible Hand: A Meta-Analysis of the Relevance of Teacher–Student Relationship Quality for Peer Relationships and the Contribution of Student Behavior. Review of Educational Research, 003465432110514. https://doi.org/10.3102/00346543211051428 Givens Rolland, R. (2012). Synthesizing the Evidence on Classroom Goal Structures in Middle and Secondary Schools: A Meta-Analysis and Narrative Review. Review of Educational Research, 82(4), 396–435. https://doi.org/10.3102/0034654312464909 Hamre, B. K., & Pianta, R. C. (2001). Early Teacher-Child Relationships and the Trajectory of Children’s School Outcomes through Eighth Grade. Child Development, 72(2), 625–638. https://doi.org/10.1111/1467-8624.00301 Kincade, L., Cook, C., & Goerdt, A. (2020). Meta-Analysis and Common Practice Elements of Universal Approaches to Improving Student-Teacher Relationships. Review of Educational Research, 90(5), 710–748. https://doi.org/10.3102/0034654320946836 Korpershoek, H., Harms, T., de Boer, H., van Kuijk, M., & Doolaard, S. (2016). A Meta-Analysis of the Effects of Classroom Management Strategies and Classroom Management Programs on Students’ Academic, Behavioral, Emotional, and Motivational Outcomes. Review of Educational Research, 86(3), 643–680. https://doi.org/10.3102/0034654315626799 Lei, H., Cui, Y., & Chiu, M. M. (2016). Affective Teacher—Student Relationships and Students’ Externalizing Behavior Problems: A Meta-Analysis. Frontiers in Psychology, 7. https://doi.org/10.3389/fpsyg.2016.01311 Nurmi, J.-E. (2012). Students’ characteristics and teacher–child relationships in instruction: A meta-analysis. Educational Research Review, 7(3), 177–197. https://doi.org/10.1016/j.edurev.2012.03.001 Roorda, D. L., Jak, S., Zee, M., Oort, F. J., & Koomen, H. M. Y. (2017). Affective Teacher–Student Relationships and Students’ Engagement and Achievement: A Meta-Analytic Update and Test of the Mediating Role of Engagement. School Psychology Review, 46(3), 239–261. https://doi.org/10.17105/SPR-2017-0035.V46-3 Roorda, D. L., Koomen, H. M. Y., Spilt, J. L., & Oort, F. J. (2011). The Influence of Affective Teacher–Student Relationships on Students’ School Engagement and Achievement: A Meta-Analytic Approach. Review of Educational Research, 81(4), 493–529. https://doi.org/10.3102/0034654311421793 Shea, B. J., Reeves, B. C., Wells, G., Thuku, M., Hamel, C., Moran, J., Moher, D., Tugwell, P., Welch, V., Kristjansson, E., & Henry, D. A. (2017). AMSTAR 2: A critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both. BMJ, j4008. https://doi.org/10.1136/bmj.j4008 Vandenbroucke, L., Spilt, J., Verschueren, K., Piccinin, C., & Baeyens, D. (2018). The Classroom as a Developmental Context for Cognitive Development: A Meta-Analysis on the Importance of Teacher–Student Interactions for Children’s Executive Functions. Review of Educational Research, 88(1), 125–164. https://doi.org/10.3102/0034654317743200 Wang, M.-T., L. Degol, J., Amemiya, J., Parr, A., & Guo, J. (2020). Classroom climate and children’s academic and psychological wellbeing: A systematic review and meta-analysis. Developmental Review, 57, 100912. https://doi.org/10.1016/j.dr.2020.100912 [less ▲] Detailed reference viewed: 164 (6 UL)![]() Emslander, Valentin ![]() in PLoS ONE (2022), 17(12), 0279255 Value-added (VA) models are used for accountability purposes and quantify the value a teacher or a school adds to their students’ achievement. If VA scores lack stability over time and vary across outcome ... [more ▼] Value-added (VA) models are used for accountability purposes and quantify the value a teacher or a school adds to their students’ achievement. If VA scores lack stability over time and vary across outcome domains (e.g., mathematics and language learning), their use for high-stakes decision making is in question and could have detrimental real-life implications: teachers could lose their jobs, or a school might receive less funding. However, school-level stability over time and variation across domains have rarely been studied together. In the present study, we examined the stability of VA scores over time for mathematics and lan- guage learning, drawing on representative, large-scale, and longitudinal data from two cohorts of standardized achievement tests in Luxembourg (N = 7,016 students in 151 schools). We found that only 34–38% of the schools showed stable VA scores over time with moderate rank correlations of VA scores from 2017 to 2019 of r = .34 for mathematics and r = .37 for language learning. Although they showed insufficient stability over time for high-stakes decision making, school VA scores could be employed to identify teaching or school practices that are genuinely effective—especially in heterogeneous student populations. [less ▲] Detailed reference viewed: 85 (6 UL)![]() ![]() Emslander, Valentin ![]() ![]() Scientific Conference (2022, November 09) Especially in diverse educational settings, positive relationships between students and their teachers can foster students’ learning and help alleviate systematic inequalities. Characterized by emotional ... [more ▼] Especially in diverse educational settings, positive relationships between students and their teachers can foster students’ learning and help alleviate systematic inequalities. Characterized by emotional warmth or closeness, positive teacher-student relationships (TSR) can improve several student outcomes. For instance, existing meta-analyses suggest significant links between TSR and students’ peer relations, school engagement, academic achievement, emotions, executive functions, general well-being, and reductions in aggressive or disruptive behaviors. However, the evidence on these links is scattered, and a comprehensive overview of the associations with TSR integrating academic, behavioral, socio-emotional, motivational, and general cognitive outcomes is lacking. Furthermore, researchers have been unequivocal about possible moderators, such as how these relationships vary with student age or gender. In light of these research gaps, we systematically reviewed the meta-analytic literature and examined (a) the extent to which academic, behavioral, socio-emotional, motivational, and general cognitive student outcomes are related to TSR in the meta-analytic literature; (b) which moderators influence this association; and (c) the methodological quality of the included meta-analyses. We included meta-analyses with preschool or K-12 samples in our dataset which reported some measure of the relation between TSR and student outcomes. With this dataset, we systematically mapped the evidence on (a) the TSR-outcome relationship; (b) the moderators; and (c) the methodological quality of the meta-analyses. We will present our core findings and discuss future research with this second-order, meta-analytic dataset and the impact of positive TSR in diverse and heterogeneous settings. [less ▲] Detailed reference viewed: 108 (6 UL)![]() ; Emslander, Valentin ![]() Report (2022) This project develops methods and procedures to (a) quantify the quality of primary studies in meta-analyses; and (b) account for primary-study quality in moderator analyses. As part of the project, we ... [more ▼] This project develops methods and procedures to (a) quantify the quality of primary studies in meta-analyses; and (b) account for primary-study quality in moderator analyses. As part of the project, we develop an analytic procedure to create study quality indicators and incorporate them in the meta-analysis. We will present this procedure in a step-by-step tutorial with illustrative examples. [less ▲] Detailed reference viewed: 63 (6 UL)![]() Emslander, Valentin ![]() ![]() Report (2022) The relationships between students and their teachers can impact students’ learning and development. Characterized by emotional warmth or closeness, positive teacher-student-relationships (TSR) can ... [more ▼] The relationships between students and their teachers can impact students’ learning and development. Characterized by emotional warmth or closeness, positive teacher-student-relationships (TSR) can improve a variety of student outcomes. Existing meta-analyses suggest strong links between TSR and students’ peer relations, school engagement, academic achievement, emotions, executive functions, general well-being, and reductions in aggressive or disruptive behaviors. However, this evidence base is scattered, and a comprehensive overview of the TSR-outcome associations integrating academic, behavioral, socio-emotional, and general cognitive outcomes is lacking. Further, researchers have been unequivocal about possible moderators, such as how these relationships change with student age as their relationship to family, peers, and teachers change. Considering these research gaps, we aim to systematically review the meta-analytic literature and examine the following two research questions: Research Question 1: To what extent do existing meta-analyses provide evidence supporting significant relations between TSR and children’s academic, behavioral, socioemotional, motivational, and general cognitive outcomes? (Overall relationship) Research Question 2: To what extent do these relationships vary by the characteristics of the meta-analyses, such as student samples, measurement characteristics, and the quality of the meta-analyses? To address these research questions, we conduct a systematic review of existing meta-analyses, integrating the findings of eligible studies. We will include quantitative meta-analyses with preschool or K-12 samples who have no diagnosed disorder or disability. [less ▲] Detailed reference viewed: 94 (13 UL)![]() ![]() Emslander, Valentin ![]() ![]() Scientific Conference (2022, March) Theoretical Background: Can we quantify the effectiveness of a teacher or a school with a single number? Researchers in the field of value-added (VA) models may argue just that (e.g., Chetty et al., 2014 ... [more ▼] Theoretical Background: Can we quantify the effectiveness of a teacher or a school with a single number? Researchers in the field of value-added (VA) models may argue just that (e.g., Chetty et al., 2014; Kane et al., 2013). VA models are widely used for accountability purposes in education and quantify the value a teacher or a school adds to their students’ achievement. For this purpose, these models predict achievement over time and attempt to control for factors that cannot be influenced by schools or teachers (i.e., sociodemographic & sociocultural background). Following this logic, what is left must be due to teacher or school differences (see, e.g., Braun, 2005). To utilize VA models for high-stakes decision-making (e.g., teachers’ tenure, the allocation of funding), these models would need to be highly stable over time. School-level stability over time, however, has hardly been researched at all and the resulting findings are mixed, with some studies indicating high stability of school VA scores over time (Ferrão, 2012; Thomas et al., 2007) and others reporting a lack of stability (e.g., Gorard et al., 2013; Perry, 2016). Furthermore, as there is no consensus on which variables to use as independent or dependent variables in VA models (Everson, 2017; Levy et al., 2019), the stability of VA could vary between different outcome measures (e.g., language or mathematics). If VA models lack stability over time and across outcome measures, their use as the primary information for high-stakes decision-making is in question, and the inferences drawn from them could be compromised. Questions: With these uncertainties in mind, we examine the stability of school VA model scores over time and investigate the differences between language and mathematics achievement as outcome variables. Additionally, we demonstrate the real-life implications of (in)stable VA scores for single schools and point out an alternative, more constructive use of school VA models in educational research. Method: To study the stability of VA scores on school level over time and across outcomes, we drew on a sample of 146 primary schools, using representative longitudinal data from the standardized achievement tests of the Luxembourg School Monitoring Programme (LUCET, 2021). These schools included a heterogeneous and multilingual sample of 7016 students. To determine the stability of VA scores in the subject of mathematics and in languages over time, we based our analysis on two longitudinal datasets (from 2015 to 2017 and from 2017 to 2019, respectively) and generated two VA scores per dataset, one for language and one for mathematics achievement. We further analyzed how many schools displayed stable VA scores in the respective outcomes over two years, and compared the rank correlations of VA scores between language and mathematics achievement as an outcome variable. Results and Their Significance: Only 34-38 % of the schools showed stable VA scores from grade 1 to 3 with moderate rank correlations of r = .37 with language and r = .34 with mathematics achievement. We therefore discourage using VA models as the only information for high-stakes educational decisions. Nonetheless, we argue that VA models could be employed to find genuinely effective teaching or school practices—especially in heterogeneous student populations, such as Luxembourg, in which educational disparities are an important topic already in primary school (Hoffmann et al., 2018). Consequently, we contrast the school climate and instructional quality, which might be a driver of the differences between schools with stable high vs. low VA scores. Literature Braun, H. (2005). Using student progress to evaluate teachers: A primer on value-added models. Educational Testing Service. Chetty, R., Friedman, J. N., & Rockoff, J. E. (2014). Measuring the impacts of teachers I: Evaluating bias in teacher value-added estimates. American Economic Review, 104(9), 2593–2632. https://doi.org/10.1257/aer.104.9.2593 Everson, K. C. (2017). Value-added modeling and educational accountability: Are we answering the real questions? Review of Educational Research, 87(1), 35–70. https://doi.org/10.3102/0034654316637199 Ferrão, M. E. (2012). On the stability of value added indicators. Quality & Quantity, 46(2), 627–637. https://doi.org/10.1007/s11135-010-9417-6 Gorard, S., Hordosy, R., & Siddiqui, N. (2013). How unstable are “school effects” assessed by a value-added technique? International Education Studies, 6(1), 1–9. https://doi.org/10.5539/ies.v6n1p1 Kane, T. J., McCaffrey, D. F., Miller, T., & Staiger, D. O. (2013). Have We Identified Effective Teachers? Validating Measures of Effective Teaching Using Random Assignment. Research Paper. MET Project. Bill & Melinda Gates Foundation. https://files.eric.ed.gov/fulltext/ED540959.pdf Levy, J., Brunner, M., Keller, U., & Fischbach, A. (2019). Methodological issues in value-added modeling: An international review from 26 countries. Educational Assessment, Evaluation and Accountability, 31(3), 257–287. https://doi.org/10.1007/s11092-019-09303-w LUCET. (2021). Épreuves Standardisées (ÉpStan). https://epstan.lu Perry, T. (2016). English value-added measures: Examining the limitations of school performance measurement. British Educational Research Journal, 42(6), 1056–1080. https://doi.org/10.1002/berj.3247 Thomas, S., Peng, W. J., & Gray, J. (2007). Modelling patterns of improvement over time: Value added trends in English secondary school performance across ten cohorts. Oxford Review of Education, 33(3), 261–295. https://doi.org/10.1080/03054980701366116 [less ▲] Detailed reference viewed: 111 (7 UL)![]() Talic, Irma ![]() in Learning and Instruction (2022), 81 Detailed reference viewed: 106 (3 UL)![]() Emslander, Valentin ![]() in Psychological Bulletin (2022), 148(5-6), 337-369 Executive functions (EFs) are key skills underlying other cognitive skills that are relevant to learning and everyday life. Although a plethora of evidence suggests a positive relation between the three ... [more ▼] Executive functions (EFs) are key skills underlying other cognitive skills that are relevant to learning and everyday life. Although a plethora of evidence suggests a positive relation between the three EF subdimensions inhibition, shifting, and updating, and math skills for schoolchildren and adults, the findings on the magnitude of and possible variations in this relation are inconclusive for preschool children and several narrow math skills (i.e., math intelligence). Therefore, the present meta-analysis aimed to (a) synthesize the relation between EFs and math intelligence (an aggregate of math skills) in preschool children; (b) examine which study, sample, and measurement characteristics moderate this relation; and (c) test the joint effects of EFs on math intelligence. Utilizing data extracted from 47 studies (363 effect sizes, 30,481 participants) from 2000 to 2021, we found that, overall, EFs are significantly related to math intelligence (r = .34, 95% CI [.31, .37]), as are inhibition (r = .30, 95% CI [.25, .35]), shifting (r = .32, 95% CI [.25, .38]), and updating (r = .36, 95% CI [.31, .40]). Key measurement characteristics of EFs, but neither children’s age nor gender, moderated this relation. These findings suggest a positive link between EFs and math intelligence in preschool children and emphasize the importance of measurement characteristics. We further examined the joint relations between EFs and math intelligence via meta-analytic structural equation modeling. Evaluating different models and representations of EFs, we did not find support for the expectation that the three EF subdimensions are differentially related to math intelligence. [less ▲] Detailed reference viewed: 137 (16 UL)![]() ; ; Greiff, Samuel ![]() in European Journal of Psychological Assessment (2022) Detailed reference viewed: 53 (0 UL)![]() ![]() Emslander, Valentin ![]() ![]() Scientific Conference (2021, November) Value-added (VA) models are widely used for accountability purposes in education. Tracking a teacher’s or a school’s VA score over time forms oftentimes the basis for high-stakes decision-making and can ... [more ▼] Value-added (VA) models are widely used for accountability purposes in education. Tracking a teacher’s or a school’s VA score over time forms oftentimes the basis for high-stakes decision-making and can determine whether teachers can keep their jobs or schools may receive certain funding. Despite their high-stakes application, the stability of VA scores over time has not yet been investigated for primary schools. Moreover, it is unclear whether different outcome measures (e.g., language and mathematics) may differ in their stability over time. In the present study, we aimed to clarify the stability of VA scores over time and investigate the differences across outcome variables. Furthermore, we wanted to showcase the real-life implications of (in)stable VA scores for single schools, with a focus on an informative use of VA scores rather than an evaluative way. The exploration of school VA scores in primary schools is especially relevant for heterogeneous student populations, for instance, in Luxembourg. Thus, we drew on representative longitudinal data from the standardized achievement tests of the Luxembourg School Monitoring Programme and examined the stability of school VA scores over two years in 146 schools (N = 7016 students). The overall stability, as measured by correlation coefficients, was moderate with r = .37 for VA scores in language and r = .34 for VA scores in mathematics from grade one to grade three. Real-life implications for schools will be discussed. [less ▲] Detailed reference viewed: 107 (20 UL)![]() Emslander, Valentin ![]() Scientific Conference (2021, September) BACKGROUND: Response inhibition, attention shifting, and working memory updating are the three core executive functions (EFs; Miyake et al., 2000) underlying other cognitive skills that are relevant for ... [more ▼] BACKGROUND: Response inhibition, attention shifting, and working memory updating are the three core executive functions (EFs; Miyake et al., 2000) underlying other cognitive skills that are relevant for learning and everyday life. For example, they have shown to be differentially related to the mathematical component of intelligence (i.e., math intelligence) in school students and adults. While researchers suppose these three EFs to become more differentiated from early childhood to adulthood, neither the link of these constructs nor their structure has been conclusively established in preschool children yet. Primary studies on path models connecting EFs and math intelligence diverge in the exact relation of EFs and math intelligence. It remains unclear whether inhibition, shifting, and updating exhibit distinct but correlated constructs with respect to their relation to math intelligence. OBJECTIVES: With our meta-analysis, we aimed to (a) synthesize the relation between the three EFs and math intelligence in preschool children; and (b) compare plausible models of the effects of EFs on math intelligence. METHODS/RESULTS: Synthesizing data from 47 studies (363 effect sizes, 30,481 participants) from the last two decades via novel multilevel and multivariate meta-analytic models (Pustejovsky & Tipton, 2020), we found the three core EFs to be significantly related to math intelligence: Inhibition ("r" ̅ = .30, 95 % CI [.25, .35]), shifting ("r" ̅ = .32, 95 % CI [.25, .38]), and updating ("r" ̅ = .36, 95 % CI [.31, .40]). Looking at the three core EFs as one construct, the correlation was "r" ̅ = .34, 95 % CI [.31, .37]. Utilizing correlation-based, meta-analytic structural equation modeling (Jak & Cheung, 2020), our results exhibited significant relations of all EFs to math intelligence. These relations did not differ between the three core EFs. DISCUSSION: Our findings corroborate the positive link between EFs and math intelligence in preschool children and are similar to other age groups. From the model testing, we learned that representing EFs by a latent variable, thus capturing the covariance among the three core EFs, explained substantially more variation in math intelligence than representing them as distinct constructs. [less ▲] Detailed reference viewed: 159 (4 UL)![]() Emslander, Valentin ![]() ![]() Scientific Conference (2021, September) Background: What is the value that teachers or schools add to the evolution of students’ performance? Value-added (VA) modeling aims to answer this question by quantifying the effect of pedagogical ... [more ▼] Background: What is the value that teachers or schools add to the evolution of students’ performance? Value-added (VA) modeling aims to answer this question by quantifying the effect of pedagogical actions on students’ achievement, independent of students’ backgrounds (e.g., Braun, 2005). A plethora of VA models exist, and several outcome measures are in use to estimate VA scores, yet without consensus on the model specification (Everson, 2017; Levy et al., 2019). Furthermore, it is unclear whether the most frequently used VA models (i.e., multi-level, linear regression, and random forest models) and outcome measures (i.e., language and mathematics achievement) indicate a similar stability of VA scores over time. Objectives: Drawing from the data of a highly diverse and multilingual school setting, where leveling out the influence of students’ backgrounds is of special interest, we aim to (a) clarify the stability of school VA scores over time; (b) shed light on the sensitivity toward different statistical models and outcome variables; and (c) evaluate the practical implications of (in)stable VA scores for individual schools. Method: Utilizing the representative, longitudinal data from the Luxembourg School Monitoring Programme (LUCET, 2021), we examined the stability of school VA scores. We drew on two longitudinal data sets of students who participated in the standardized achievement tests in Grade 1 in 2014 or 2016 and then again in Grade 3 two years later (i.e., 2016 and 2018, respectively), with a total of 5875 students in 146 schools. School VA scores were calculated using classical approaches (i.e., linear regression and multilevel models) and one of the most commonly used machine learning approaches in educational research (i.e., random forests). Results and Discussion: The overall stability over time across the VA models was moderate, with multilevel models showing greater stability than linear regression models and random forests. Stability differed across outcome measures and was higher for VA models with language achievement as an outcome variable as compared to those with mathematics achievement. Practical implications for schools and teachers will be discussed. [less ▲] Detailed reference viewed: 260 (10 UL)![]() Emslander, Valentin ![]() Scientific Conference (2021, July) Introduction: Executive functions (inhibition, attention shifting, updating) are linked to math intelligence in school students and adults. This link is particularly important because performance in ... [more ▼] Introduction: Executive functions (inhibition, attention shifting, updating) are linked to math intelligence in school students and adults. This link is particularly important because performance in school mathematics is predictive of various competencies later in life. While some researchers argue that tests of executive functions and math intelligence measure the same underlying construct, others argue that they measure distinct but correlated constructs. Also, evidence on the differentiation of cognitive skills over time exists. Clarifying the relation between executive functions and math intelligence is, however, challenging, especially because preschoolers cannot fill in commonly used questionnaires that require them to read. As a consequence, researchers have to resort to behavioral, verbal, apparatus-, or computer-based assessments of executive functions. Objectives/Methodology: With this meta-analysis of 29 studies containing 268 effect sizes, we examined the link between executive functions and math intelligence for a total sample of 25,510 preschool children. Specifically, we synthesized the corresponding correlations and sought to clarify which executive function assessments were used for preschool children and how the assessment characteristics may moderate the correlation between executive functions and mathematical skills. Results: Utilizing three-level random-effects meta-analysis, we found a moderate correlation between executive functions and mathematical skills in preschool children, r = 0.35. The type of assessment (behavioral, verbal, apparatus-, or computer-based assessments) did not moderate this relation. Differentiating between the three executive functions revealed average correlations of r = 0.30 between math and inhibition, r = 0.38 between math and attention shifting, and r = 0.36 between math and updating. These analyses will be supplemented by further moderator analyses. Conclusions: Our findings support the significant link between executive functions and mathematical skills in preschoolers—yet, the average correlations do not suggest that both measures are identical. Results will be discussed against the background of deployed assessments and testing environments. [less ▲] Detailed reference viewed: 140 (6 UL)![]() Emslander, Valentin ![]() Scientific Conference (2021, May 20) Background: Executive functions (i.e., response inhibition, attention shifting, working memory updating) have shown to be related to the mathematical component of intelligence, which, in turn, is ... [more ▼] Background: Executive functions (i.e., response inhibition, attention shifting, working memory updating) have shown to be related to the mathematical component of intelligence, which, in turn, is predictive of various competences later in life. While this relation has already been thoroughly researched in school students and adults, a comprehensive research synthesis on preschool children—a group for which the assessment of these constructs is more challenging—is still missing. Evidence on the differentiation of cognitive skills over time suggests a differential relation of the three executive functions with math intelligence in older but not in younger children. It remains unclear, however, whether and which one of the three executive functions is more closely related to math intelligence in preschool children. Further research gaps concern the measurement of both executive functions and math intelligence in preschool children, as they cannot complete reading- and writing-based questionnaires. Addressing this measurement challenge, a plethora of inventive measurements has been used to assess both cognitive skills. These measurement differences might also have an influence on the relation between executive functions and math intelligence. Objectives: With our meta-analysis, we aimed to clarify the relation between executive functions and math intelligence in preschool children. Further, we wanted to investigate the influence of different measurement methods on this relation and look into the specific links of inhibition, shifting, and updating with math intelligence more closely. Research questions: 1. To what extent are inhibition, shifting, and updating (as a composite and separately) related to math intelligence in preschool children? (Overall correlations) 2. Which sample, study, and measurement characteristics moderate this relation? (Heterogeneity and moderators) 3. How much variation in math intelligence do inhibition, shifting, and updating explain jointly? (Model testing) Methods: We examined the relation between executive functions and math intelligence for 268 effect sizes from 29 studies for a total sample of 25,510 preschool children. Specifically, we synthesized the corresponding correlations by means of three-level random-effects meta-analyses (RQ 1) and examined the study, sample, and measurement characteristics as possible moderators of this relation between EFs and math intelligence via mixed-effects modeling (RQ 2). Further, we performed meta-analytic structural equation modeling to investigate the joint and differential effects inhibition, shifting, and updating on math intelligence (RQ 3). Results: We found executive functions and math intelligence to correlate moderately in preschool children (r = .35). Investigating inhibition, shifting, and updating separately also revealed moderate average correlations to math intelligence (r = .30, r = .38 , and r = .36, respectively). While we did not find age to explain significant amounts of heterogeneity, four measurement characteristics moderated the relation between executive function and math intelligence. When considered jointly through meta-analytic structural equation modeling, the relations of inhibition, shifting, and updating to math intelligence were similar. Conclusions and Implications: By presenting evidence for a significant relation between executive functions and math intelligence also in preschool children, our findings contribute to the discussion on the differentiation of cognitive skills. They highlight the importance of considering measurement characteristics when researching executive functions and math intelligence. Further, we could not confirm that inhibition, shifting, and updating are differentially related to math intelligence. Further research is needed to clarify the impact of age on the relation between executive functions and math intelligence. [less ▲] Detailed reference viewed: 136 (6 UL)![]() ![]() Talic, Irma ![]() Scientific Conference (2021) Detailed reference viewed: 193 (0 UL)![]() ; ; Greiff, Samuel ![]() in Motivation Science (2021), 7(3), 306318 Detailed reference viewed: 53 (0 UL)![]() ![]() Emslander, Valentin ![]() Scientific Conference (2020, July) Introduction: Executive functions (inhibition, attention shifting, working memory) are linked to mathematical skills in school students and adults. This link is particularly important because performance ... [more ▼] Introduction: Executive functions (inhibition, attention shifting, working memory) are linked to mathematical skills in school students and adults. This link is particularly important because performance in school mathematics is predictive of various competencies later in life. While some researchers argue that tests of executive functions and mathematical skills measure the same underlying construct, others argue that they measure distinct but correlated constructs. Also, evidence on the differentiation of cognitive skills over time exists. Clarifying the relation between executive functions and mathematical skills is, however, challenging, especially because preschoolers cannot fill in commonly used questionnaires that require them to read. As a consequence, researchers have to resort to behavioral, verbal, apparatus-, or computer-based assessments of executive functions. Objectives/Methodology: With this meta-analysis of 26 studies containing 238 effect sizes, we examined the link between executive functions and early mathematical skills for a total sample of 24,256 preschool children. Specifically, we synthesized the corresponding correlations and sought to clarify which executive function assessments were used for preschool children and how the assessment characteristics may moderate the correlation between executive functions and mathematical skills. Results: Utilizing three-level random-effects meta-analysis, we found a moderate correlation between executive functions and mathematical skills in preschool children, r = 0.35. The type of assessment (behavioral, verbal, apparatus-, or computer-based assessments) did not moderate this relation. Differentiating between the three executive functions revealed average correlations of r = 0.31 between math and inhibition, r = 0.38 between math and attention shifting, and r = 0.36 between math and updating. These analyses will be supplemented by further moderator analyses. Conclusions: Our findings support the significant link between executive functions and mathematical skills in preschoolers—yet, the average correlations do not suggest that both measures are identical. Results will be discussed against the background of deployed assessments and testing environments. [less ▲] Detailed reference viewed: 155 (5 UL) |
||