Fact Sheet on the FL Value-Added Model (VAM) Required by SB 736

What you need to know about the context

·         40% of all Florida instructional personnel teach courses associated with FCAT (FCAT teachers)

·         60% of Florida instructional personnel do not teach courses associated with FCAT (non-FCAT teachers)

o   Some are classroom teachers who do not teach FCAT subjects, e.g. art, music, PE, technology, PreK, Grades 1-2 and 11-12

o   Some are non-classroom teachers, e.g. guidance counselors, instructional coaches, speech-language pathologists, psychologists

 

SB 736 requires that districts:

·         Use the state-adopted student growth formula (VAM) for all FCAT teachers

·         Use  VAM or, if verifiable data on district assessments are available, an equally appropriate formula of student growth as a minimum of 40% for a classroom teacher’s evaluation – this includes FCAT and non-FCAT classroom teachers

o   If the district uses one year of data, the percentage of the evaluation may be not less than 40%

o   If a district uses three years of data the percentage of the evaluation must be at least 50%

·         Use VAM or a combination of VAM and appropriate measurable student outcomes as a minimum of 20% of for a non-classroom teacher’s evaluation

o   If the district uses one year of data, the percentage of the evaluation may be not less than 20%

o   If a district uses three years of data the percentage of the evaluation must be at least 30%

·         Use teacher observation score and Student Growth (VAM) to rate teaches in one of four categories:

o   Highly Effective (HE)

o   Effective (E)

o   Developing (1-3 year teachers)/Needs Improvement (3+ year teachers) (D/NI)

o   Unsatisfactory (U)

 

The Florida VAM

·         Is a covariant model that recognizes aspects out of a teachers control and makes statistical adjustments so these aspects do not negatively impact a teacher’s VAM score; the FL VAM adjusts for:

o   Students with Disabilities

o   English Language Learners

o   Gifted Students

o   Mobility

o   Student attendance

o   Homogeneity of class composition

o   Difference from modal (typical) age

·         Uses FCAT data

·         Uses only the student data within the school in the calculations; this means that teachers are compared  to other teachers at their school and their VAM scores are relative to others in their school, not others in their district or the state

·         Uses past performance and the other variables to quantify the projected student learning outcomes, then, determines quantitatively to what degree the students associated with the teacher exceeded, met or  did not meet projections; these calculations yield the teacher effect score

·         Calculates the school effect using all student data in a similar statistical process

·         Combines the teacher effect as 1.0 and the school effect as 0.5 (2/3—1/3) to arrive at the final VAM score for each teacher

 

Problems with the FL VAM

·         Data for FCAT teachers – most VAM experts assert that, statistically-speaking,  at least three years of data should be used to reduce errors so that  teachers are not misidentified 

o   Florida school data gathered before 2010-11 (historic data) was gathered for other purposes and is not verifiable

o   Districts verified that the teacher –student linkage is accurate in the 2010-11 data; i.e. the list of students in each class/course is associated with the correct teacher

 

·         Data for non-FCAT teachers – There is minimal or NO subject area data available for approximately 60% of Florida’s teachers because they do not teach FCAT subjects in grades 4 thru 10. 

o   New teachers, even those who teach FCAT subjects, have no existing data associated with them.

o   PreK-2 teachers, 11-12th grade teachers, PE, music, art, and technology teachers have no FCAT scores associated with their teaching position.  Most districts have no tests and data are not yet available to follow student learning growth in any of these courses.  For these teachers, school districts must either:

§  Develop data streams for each teacher using student learning growth or performance data on their own district developed tests;

§  OR, (if these district level tests do not exist or there is insufficient student data) create data streams from existing FCAT data tangentially related to a teacher’s position

For both FCAT and non-FCAT teachers, districts really only have one year of verifiable data.  We can expect that, due to limited data, there will be random errors and teachers will be misclassified.

 

·         Student attendance

o   Is one of the variables outside a teacher’s control and is included in the formula to mitigate the impact on the teacher effect score

o   Data is reported to the DOE using daily attendance not course attendance

o   Students do miss particular courses consistently, and their attendance should be a mitigating factor in calculating their teachers VAM score

The DOE does not have the student attendance data by course; consequently, teachers with students who consistently miss their course(s) will be disadvantaged in the teacher effect VAM calculation.

 

·         The VAM formula calculation at school level

o   Teachers’ VAM scores in high performing schools are likely to regress to the mean; it will be very difficult to identify high performing and/or low performing teachers at high performing schools

o   Teachers in low performing schools are more likely to demonstrate the extremes in the VAM range and can be classified as Highly Effective and Unsatisfactory even if their actual teaching performance is equal to that of a mean-score teacher at a high performing school

o   Districts are required to set ranges across the district even though the calculation is computed within the school, forcing an apples-to-oranges comparison 

The VAM formula comparisons from school-to-school create an uneven field and an increased likelihood that classifications will be inconsistent with a teacher’s performance.  The teacher-to-teacher comparison, especially for high stakes employment decisions, pits teacher against teacher and drives down any interest in collaboration.

 

·         The four required classifications – Districts must determine cut scores between each category

o   Without a middle category, teachers with similar teaching performance and scores that vary by one or two points can be assigned to completely different categories; e.g. Effective and Needs Improvement

Random error must be considered and gauged for all teachers with proximal scores near cut offs so that teachers are not misclassified or negatively affected due to statistical inaccuracies

 

 

 

 The policy (SB 736) is too far ahead of the data and infrastructure.