Fact Sheet on the FL Value-Added
Model (VAM) Required by SB 736

**What you need to
know about the context**

· 40% of all Florida instructional personnel teach courses associated with FCAT (FCAT teachers)

· 60% of Florida instructional personnel do not teach courses associated with FCAT (non-FCAT teachers)

o Some are classroom teachers who do not teach FCAT subjects, e.g. art, music, PE, technology, PreK, Grades 1-2 and 11-12

o Some are non-classroom teachers, e.g. guidance counselors, instructional coaches, speech-language pathologists, psychologists

**SB 736 requires
that districts:**

· Use the state-adopted student growth formula (VAM) for all FCAT teachers

· Use VAM or, if verifiable data on district assessments are available, an equally appropriate formula of student growth as a minimum of 40% for a classroom teacher’s evaluation – this includes FCAT and non-FCAT classroom teachers

o If the district uses one year of data, the percentage of the evaluation may be not less than 40%

o If a district uses three years of data the percentage of the evaluation must be at least 50%

· Use VAM or a combination of VAM and appropriate measurable student outcomes as a minimum of 20% of for a non-classroom teacher’s evaluation

o If the district uses one year of data, the percentage of the evaluation may be not less than 20%

o If a district uses three years of data the percentage of the evaluation must be at least 30%

· Use teacher observation score and Student Growth (VAM) to rate teaches in one of four categories:

o Highly Effective (HE)

o Effective (E)

o Developing (1-3 year teachers)/Needs Improvement (3+ year teachers) (D/NI)

o Unsatisfactory (U)

**The Florida VAM **

· Is a covariant model that recognizes aspects out of a teachers control and makes statistical adjustments so these aspects do not negatively impact a teacher’s VAM score; the FL VAM adjusts for:

o Students with Disabilities

o English Language Learners

o Gifted Students

o Mobility

o Student attendance

o Homogeneity of class composition

o Difference from modal (typical) age

· Uses FCAT data

· Uses only the student data within the school in the calculations; this means that teachers are compared to other teachers at their school and their VAM scores are relative to others in their school, not others in their district or the state

· Uses past performance and the other variables to quantify the projected student learning outcomes, then, determines quantitatively to what degree the students associated with the teacher exceeded, met or did not meet projections; these calculations yield the teacher effect score

· Calculates the school effect using all student data in a similar statistical process

· Combines the teacher effect as 1.0 and the school effect as 0.5 (2/3—1/3) to arrive at the final VAM score for each teacher

**Problems with the
FL VAM**

·
__Data for FCAT teachers__ – most VAM experts
assert that, statistically-speaking, at
least three years of data should be used to reduce errors so that teachers are not misidentified

o Florida school data gathered before 2010-11 (historic data) was gathered for other purposes and is not verifiable

o Districts verified that the teacher –student linkage is accurate in the 2010-11 data; i.e. the list of students in each class/course is associated with the correct teacher

·
__Data for non-FCAT teachers__ – There is
minimal or NO subject area data available for approximately 60% of Florida’s
teachers because they do not teach FCAT subjects in grades 4 thru 10.

o New teachers, even those who teach FCAT subjects, have no existing data associated with them.

o PreK-2
teachers, 11-12^{th} grade teachers, PE, music, art, and technology
teachers have no FCAT scores associated with their teaching position. Most districts have no tests and data are not
yet available to follow student learning growth in any of these courses. For these teachers, school districts must
either:

§ Develop data streams for each teacher using student learning growth or performance data on their own district developed tests;

§
OR, (if these district level tests do not exist
or there is insufficient student data) __create__** **data streams from existing FCAT data __tangentially__ related
to a teacher’s position

**For both FCAT and
non-FCAT teachers, districts really only have one year of verifiable data. We can expect that, due to limited data,
there will be random errors and teachers will be misclassified.**

·
__Student attendance __

o Is one of the variables outside a teacher’s control and is included in the formula to mitigate the impact on the teacher effect score

o Data is reported to the DOE using daily attendance not course attendance

o Students do miss particular courses consistently, and their attendance should be a mitigating factor in calculating their teachers VAM score

**The DOE does not
have the student attendance data by course; consequently, teachers with
students who consistently miss their course(s) will be disadvantaged in the teacher
effect VAM calculation. **

·
__The VAM formula calculation at school level__

o Teachers’ VAM scores in high performing schools are likely to regress to the mean; it will be very difficult to identify high performing and/or low performing teachers at high performing schools

o Teachers in low performing schools are more likely to demonstrate the extremes in the VAM range and can be classified as Highly Effective and Unsatisfactory even if their actual teaching performance is equal to that of a mean-score teacher at a high performing school

o Districts are required to set ranges across the district even though the calculation is computed within the school, forcing an apples-to-oranges comparison

**The VAM formula
comparisons from school-to-school create an uneven field and an increased
likelihood that classifications will be inconsistent with a teacher’s
performance. The teacher-to-teacher
comparison, especially for high stakes employment decisions, pits teacher
against teacher and drives down any interest in collaboration.**

·
__The four required classifications__ –
Districts must determine cut scores between each category

o Without
a middle category, teachers with similar teaching performance and scores that
vary by one or two points can be assigned to completely different categories;
e.g. Effective and Needs Improvement

**Random error must
be considered and gauged for all teachers with proximal scores near cut offs so
that teachers are not misclassified or negatively affected due to statistical
inaccuracies**

**The policy (SB 736) is too far ahead of the data and
infrastructure.**