![]() Value-added models: To what extent do estimates of school effectiveness depend on the selection of covariates?Levy, Jessica ; ; Keller, Ulrich et alScientific Conference (2019, September) Detailed reference viewed: 171 (6 UL)![]() Value-added modeling in primary school: What covariates to include?Levy, Jessica ; ; Keller, Ulrich et alScientific Conference (2019, August) Detailed reference viewed: 182 (9 UL) Methodological Issues in Value-Added Modeling: An International Review from 26 CountriesLevy, Jessica ; ; Keller, Ulrich et alin Educational Assessment, Evaluation and Accountability (2019), 31(3), 257-287 Value-added (VA) modeling can be used to quantify teacher and school effectiveness by estimating the effect of pedagogical actions on students’ achievement. It is gaining increasing importance in ... [more ▼] Value-added (VA) modeling can be used to quantify teacher and school effectiveness by estimating the effect of pedagogical actions on students’ achievement. It is gaining increasing importance in educational evaluation, teacher accountability, and high-stakes decisions. We analyzed 370 empirical studies on VA modeling, focusing on modeling and methodological issues to identify key factors for improvement. The studies stemmed from 26 countries (68% from the USA). Most studies applied linear regression or multilevel models. Most studies (i.e., 85%) included prior achievement as a covariate, but only 2% included noncognitive predictors of achievement (e.g., personality or affective student variables). Fifty-five percent of the studies did not apply statistical adjustments (e.g., shrinkage) to increase precision in effectiveness estimates, and 88% included no model diagnostics. We conclude that research on VA modeling can be significantly enhanced regarding the inclusion of covariates, model adjustment and diagnostics, and the clarity and transparency of reporting. [less ▲] Detailed reference viewed: 320 (35 UL)![]() The use of value-added models for the identification of schools that perform “against the odds”Levy, Jessica ; ; Keller, Ulrich et alPoster (2019, July) Value-added (VA) modeling aims to quantify the effect of pedagogical actions on students’ achievement, independent of students’ backgrounds. VA modeling is primarily used for accountability and high ... [more ▼] Value-added (VA) modeling aims to quantify the effect of pedagogical actions on students’ achievement, independent of students’ backgrounds. VA modeling is primarily used for accountability and high-stakes decisions. To date, there seems to be no consensus concerning the calculation of VA models. Our study aims to systematically analyze and compare different school VA models by using longitudinal large-scale data emerging from the Luxembourg School Monitoring Programme. Regarding the model covariates, first findings indicate the importance of language (i.e., language(s) spoken at home and prior language achievement) in VA models with either language or math achievement as a dependent variable, with the highest amount of explained variance in VA models for language. Concerning the congruence of different VA approaches, we found high correlations between school VA scores from the different models, but also high ranges between VA scores for single schools. We conclude that VA models should be used with caution and with awareness of the differences that may arise from methodological choices. Finally, we discuss the idea that VA models could be used for the identification of schools that perform “against the odds”, especially for those schools that have positive VA scores over several years. [less ▲] Detailed reference viewed: 202 (8 UL)![]() Social and linguistic background influences on performance in a multilingual educational systemUgen, Sonja ; Levy, Jessica ; Martini, Sophie Frédérique et alScientific Conference (2019, June 27) Detailed reference viewed: 155 (9 UL)![]() Exploration of Different School Value-Added Models in a Highly Heterogeneous Educational ContextLevy, Jessica ; ; Keller, Ulrich et alScientific Conference (2019, April) Detailed reference viewed: 171 (16 UL)![]() Wat droen d’Schoulen zur Leeschtung vu Schülerinnen a Schüler bäi? E systemateschen Iwwerbléck iwwer déi aktuell Fuerschungslag an Uwennung u lëtzebuergesche Längsschnëtt-DonnéeënLevy, Jessica ; ; Keller, Ulrich et alScientific Conference (2018, November 08) Detailed reference viewed: 144 (8 UL)![]() Value-added (VA) modelling: An Integrative Review of 674 Publications and VA Application in the Luxembourg School Monitoring ProgrammeLevy, Jessica ; Keller, Ulrich ; et alScientific Conference (2018, September) Detailed reference viewed: 167 (14 UL)![]() Évaluation de l'efficacité du système éducatif par le calcul de valeur ajoutée (VA) : Revue de littérature et exploration de modèles de VA dans un contexte scolaire hautement hétéroclite.Levy, Jessica ; Gamo, Sylvie ; Keller, Ulrich et alScientific Conference (2018, July) Detailed reference viewed: 160 (11 UL)![]() Modéliser la « valeur ajoutée » en éducation primaire et secondaire : 674 publications en revueLevy, Jessica ; Gamo, Sylvie ; Keller, Ulrich et alScientific Conference (2018, January) L’approche statistique du type de « valeur ajoutée » (« value added ») a comme but de quantifier l’effet des acteurs pédagogiques sur la performance des élèves, indépendamment de leur origine (p. ex ... [more ▼] L’approche statistique du type de « valeur ajoutée » (« value added ») a comme but de quantifier l’effet des acteurs pédagogiques sur la performance des élèves, indépendamment de leur origine (p. ex. Braun, 2005), c’est-à-dire de déterminer la valeur dans la performance de l’élève du fait qu’il étudie avec tel professeur ou /et qu’il soit dans telle école. Ces indices de valeur ajoutée une fois déterminés sont souvent utilisés pour prendre des décisions de reddition de compte (« accountability » ; p.ex. Sanders, 2000) L’idée est de faire une évaluation standardisée de la qualité des enseignants ou des écoles à travers l’évolution des résultats des élèves. Même si les valeurs ajoutées sont devenues plus populaires durant ces dernières années, il n’y a pas de consensus concernant la méthode pour les calculer, ni sur l’intégration de variables explicatives (p. ex. Newton et al., 2010). Le but de notre étude est de faire une revue de littérature concernant les valeurs ajoutées en éducation primaire et secondaire. Pour ce faire, nous avons utilisé les bases de données ERIC, Scopus, PsycINFO et Psyndex et nous avons analysé et classifié rigoureusement 674 études de 32 pays différents. La moitié des études recensées concerne les valeurs ajoutées au niveau des enseignants et les autres concernent celles au niveau des écoles ou directeurs. 370 études ont utilisé des données empiriques pour calculer des indices de valeur ajoutée. Dans un certain nombre d’études, les variables utilisées sont précisées, mais dans approximativement 15% des publications, le modèle statistique utilisé n’est pas spécifié. La plupart des études ont utilisé la performance des années précédentes des élèves comme prédicteur ; en revanche, des variables cognitives ou motivationnelles des élèves n’ont presque jamais été prises en considération. Cette revue de littérature permet de souligner, en vue des enjeux politiques importants des valeurs ajoutées, qu’il est nécessaire d’avoir plus de transparence, rigueur et consensus, surtout sur le plan méthodologique. [less ▲] Detailed reference viewed: 307 (27 UL)![]() Value-Added Modelling in Primary and Secondary School: An Integrative Review of 674 PublicationsLevy, Jessica ; Keller, Ulrich ; et alScientific Conference (2017, December) Value-added (VA) modelling aims to quantify the effect of pedagogical actions on students’ achievement, independent of students’ backgrounds (e.g., [1]); in other words, VA strives to model the added ... [more ▼] Value-added (VA) modelling aims to quantify the effect of pedagogical actions on students’ achievement, independent of students’ backgrounds (e.g., [1]); in other words, VA strives to model the added value of teaching. VA is typically used for teacher and/or school accountability (e.g., [2]). Although, VA models have gained popularity in recent years—a substantial increase of publications is to be observed over the last decade—, there is no consensus on how to calculate VA, nor is there a consensus whether and which covariates should be included in the statistical models (e.g., [3]). The aim of the present study is to conduct a to date non-existent integrative review on VA modelling in primary and secondary education. Starting with an exhaustive literature research in the ERIC, Scopus, PsycINFO, and Psyndex databases, we reviewed and thoroughly classified 674 VA publications from 32 different countries. Half of the studies investigated VA models at teacher level; the remaining looked at school or principal level. 370 studies used empirical data to calculate VA models. Most of these studies explained their covariates, but approximately 15% did not specify the model. Most studies used prior achievement as a covariate, but cognitive and/or motivational student data were almost never taken into consideration. Moreover, most of the studies did not adjust for methodological issues such as missing data or measurement error. To conclude, given the high relevance of VA—it is primarily used for high-stakes decisions— more transparency, rigor and consensus are needed, especially concerning methodological details. References [1] Braun, H. I. (2005). Using student progress to evaluate teachers: A primer on value-added models. Princeton, NJ: Educational Testing Service. [2] Sanders, W. L. (2000). Value-added assessment from student achievement data: Opportunities and hurdles. Journal of Personnel Evaluation in Education, 14(4), 329–339. [3] Newton, X., Darling-Hammond, L., Haertel, E., & Thomas, E. (2010). Value-added modeling of teacher effectiveness: An exploration of stability across models and contexts. Education Policy Analysis Archives, 18(23). [less ▲] Detailed reference viewed: 234 (26 UL) |
||