Agenda Item No

 

The Vale of Glamorgan Council

 

Scrutiny Committee (Lifelong Learning) 14 December 2009

 

Report of the Director of Learning and Development

 

School Performance - The Annual Performance Review

 

Purpose of the Report

1.       To bring to the attention of the Scrutiny Committee for Lifelong Learning the position in respect of the Performance of Schools.

Recommendations

1.       Scrutiny Members welcome the progress being achieved across Vale of Glamorgan schools.

2.       Scrutiny Members note the need for further research about Value Added performance.

3.       Scrutiny Members request a further report on School Performance and Value Added November 2010.

Reasons for the Recommendations

1.       Members are apprised of School Performance

2.       Members develop a shared understanding about the concept of Value Added

3.       To keep School Performance under review

Background

2.       The education system in Wales is arranged in Key Stages which outline the phases a pupil progresses through, during their time in school.  At the end of each Key Stage pupils are assessed in the core subjects of English, Mathematics, Science and Welsh (1st Language) via statutory teacher assessment.  Statutory teacher assessment is reported to parents and the local Authority at the end of each Key Stage.  Currently pupils undergo teacher assessment in the following Key Stages:-

.        Baseline assessment for each pupil in a Reception Class

.        End of Key Stage 1 teacher assessment for each pupil at the end of Year 2

.        End of Key Stage 2 teacher assessment for each pupil at the end of Year 6

.        End of Key Stage 3 teacher assessment for each pupil at the end of Year 9

.        Assessment in KS 4 and KS 5 is via external marking and moderation:

         ·        Key Stage 4: GCSE plus a range of learning pathways

         ·        Key Stage 5: A Levels

 

3.      Pupils in Reception classes undergo Baseline Assessment 6 weeks after joining the class. Currently in the Vale, we do not administer Baseline Assessment to Nursery aged pupils.

4.      Additionally, there is no common Baseline Assessment for pupils across Wales. Consequently, within the Central South Consortium and across South Wales, there is no common Baseline Assessment in place. This makes overall comparison across Wales and between Authorities difficult and can lead to inconsistencies in judgements.

5.      Value added judgements can also be difficult and possibly insecure, due to the absence of a common Baseline.  However, DCELLS are piloting the All Wales Baseline Assessment for release to all schools in 2011. Schools in the Vale are participating in the pilot programme. 

6.      The proportion of pupils entitled to free school meals (% FSM) in the Vale of Glamorgan is significantly lower than the Wales means for all sectors. The 2008-09 overall figure of 12.7%, is the sixth lowest of the 22 unitary authorities within Wales. Using FSM as a proxy indicator of socio-economic deprivation, this indicates that the Vale is relatively advantaged.  In terms of % FSM, Monmouth and Powys hold joint 1st (9.8%), Flint (11.4%), Ceredigion (12.2%), Gwynedd (12.6), the Vale (12.7%) with Wales at 17.8%.  This means that our overall performance across any number of PIs should not fall greater than 6th position when compared with the all Wales figures.

7.      The scores of 2008 Welsh Index of Multiple Deprivation (WIMD) reinforce the relatively advantaged position of the Vale. The summary charts (Appendix 1) suggest that the Vale’s nearest overall match is Flint LA (FSM – 11.4%).

8.      With the Authority’s close proximity to the Capital, rural setting, good housing stock and business links, schools in the Vale attract families and pupils from out of County. Additionally, the St. Athan Military Base has traditionally attracted families from further afield and with the planned future development at the site is set to continue to do so.

9.      Job opportunities for migrant workers are also comparatively good and Vale schools effectively support a large number of pupils from Ethnic Minority backgrounds and who do not speak English fluently. Data analysis reveals that this is a growing trend.  Broadly 2% of school’s population in both the Primary and Secondary Phase present as not possessing fluency in English. This may be observed as an increasing trend overall between 2006/2009.  The Vale has the fourth highest population of black minority ethnic and pupils with English as an additional language (BME/EAL) pupils in Wales.

10.    With the establishment of Unitary Authorities in 1996 the Vale became host to three Special Schools. The schools provide settings appropriate to pupils’ needs and many places are secured by pupils who are domiciled out of County.

11.    Compared with Wales there are significantly higher numbers of pupils in the Nursery Phase (2.5 that of Wales) with pupil numbers in Special Schools also exceeding that of the all Wales figures (271/187).  Mobility rates are also relatively high being approximately 30% at any one time.

12.    When comparing the number of pupils with statements of Special Education Needs (SEN) with numbers in Wales, the Vale has a significantly lower number of pupils with statements of SEN in Primary and Secondary phases and significantly higher numbers in our Special Schools.  Trends in pupil numbers with statements of SEN in the Vale have broadly remained the same (07/08), while in Wales numbers have decreased.

13.    There are fewer primary and secondary schools in the Vale compared with the LA means. As a consequence, the numbers of pupils in Vale schools are generally higher than average. No primary school in the Vale has less than 100 pupils on role (2008).  Pupil teacher ratios and class sizes are similar than the all Wales means except for Years 12 and 13. This reflects the generally high retention rates and relatively large numbers of students within school sixth forms.

14.    Schools in the Vale of Glamorgan effectively support children who are Looked After (LAC): taken overall, an increasing trend is observed for both Primary and secondary phase with 43 pupils presenting as LAC in the Primary phase and 46 in the Secondary. However, within-year variation can be significant as actual numbers fluctuate on a week by week basis.

15.    Permanent Exclusions: data for the last 4 years shows that for Primary schools there has been only 1 permanent exclusion in 05/06 with no permanent exclusion since 06/07.  In Secondary schools, numbers have remained broadly the same with 4 permanent exclusions in 05/06, none in 06/07 and 3 in 07/08. No figures are available yet for 08/09 for the Secondary phase.  Fixed Term exclusions: in Primary schools, taken overall, numbers have remained broadly the same (acknowledging a significant decrease in 07/08). For Secondary schools, an increasing trend is observed (again acknowledging a decrease in 06/07) between 05/06 and 07/08.  In secondary schools (in 07/08), 332 pupils were excluded on a fixed term basis.  In Primary schools, (in 08/09), 79 pupils were excluded on a fixed term basis.

Relevant Issues and Option

16.    Overall, these findings indicate that the aggregated performance of Vale schools should be significantly higher than for Wales as a whole and generally rank in the 6 highest performing authorities.  Our Authority wide expectation for performance in all phases and for all PIs thus reflects our relatively advantaged position and recognises that achievement and attainment targets must be both challenging and achievable, whilst remaining aligned with national and local expectations.  Our aim therefore, for any PI, is to consistently exceed National and local expectations, which are set appropriately, within the broader context aforementioned.  Actual performance data reflects the following:-

Key Stage 1 Performance (also see Appendix 1b)

.        Pupil outcomes are significantly above the all Wales means for all performance indicators for the last few years.

.        Gender differences are generally similar to or below those for Wales for the last few years except for science.

.        Aggregated national benchmarking performance is better than expected. The proportion of school performance indicators (PIs) above the first quartile has exceeded 25% for the last 3 years. Similarly, the proportion of school PIs above median has exceeded 50% for the last 3 years.

.        Comparisons with other LAs indicate that Vale performance (rounded to nearest whole number) ranks no lower than 3rd for all PIs for the last 2 years.

.        Graphs constructed to indicate LA performance vs. %FSM indicate that Vale performance is significantly above expectation for all PIs for 2009.

.        NC Level 3+ performance is also significantly above the all Wales means for the last few years. This indicates that more able pupils are achieving well within Vale primary schools at KS1

Key Stage 2 Performance (also see Appendix 1c)

.        Pupil outcomes are above the all Wales means for all performance indicators for the last few years.

.        Gender differences are generally similar to or below those for Wales for the last few years except for science.

.        Aggregated national benchmarking performance is weaker than expected. While the proportion of school PIs above median has generally exceeded 50% for the last 3 years, the proportion above the first quartile was less than 25% for 2009 (except for science). There have been less than 25% of aggregated PIs above the first quartile for each of the last 3 years. However, the proportion of PIs below the third quartile fell for 2009.

.        Comparisons with other LAs indicate that Vale performance (rounded to nearest whole number) ranks no lower than 5th for all PIs for the last 2 years except for Welsh.

.        Graphs constructed to indicate LA performance vs. %FSM indicate that Vale performance is at or above expectation for all PIs for 2009.

.        NC Level 5+ performance is slightly above but similar to the all Wales means for the last few years. This indicates that more able pupils are achieving less well than expected within Vale schools at KS2.

Key Stage 3 Performance (also see Appendix 1d)

.        Pupil outcomes in core subjects are generally above the all Wales means for all performance indicators for the last few years. English performance improved notably for 2009.

.        Gender differences are generally similar to or below those for Wales for the last few years except for science.

.        Aggregated national benchmarking performance is better than expected. The proportion of school performance indicators (PIs) above the first quartile has been at or exceeded 25% for the last 3 years. Similarly, the proportion of school PIs above median has exceeded 50% for 2 out of the last 3 years. The proportion of PIs below the 3rd quartile has fallen for the last 2 years except for English in 2008.

.        Comparisons with other LAs indicate that Vale performance (rounded to nearest whole number) ranks no lower than 5th for all PIs for the last 2 years.

.        Graphs constructed to indicate LA performance vs. %FSM indicate that Vale performance generally matches or is slightly above expectation for all PIs.

.        NC Level 6+ performance is significantly above the all Wales means for most core subjects for the last few years. This indicates that more able pupils are achieving well within Vale secondary schools at KS3.

.        Pupil outcomes in non-core subjects are generally above the all Wales means for all performance indicators for the last few years except for PE.

.        Gender differences are generally similar to or below those for Wales for the last few years except for MFL, PE and Welsh 2nd language.

.        Aggregated national benchmarking performance is better than expected. The proportion of school performance indicators (PIs) above the first quartile has been at or exceeded 25% for 2009. Similarly, the proportion of school PIs above median is at or has exceeded 50% for 2009. The proportion of PIs below the 3rd quartile does not exceed 25% for any subject for 2009. The highest performing subject for 2009 is geography and the lowest PE (which still matches expectations with 25% performance in each quarter.

.        Comparisons with other LAs indicate that Vale performance for 2009 (rounded to nearest whole number) ranks no lower than 6th for any non core subject (geography and IT) with D&T, history and history ranking 1st.

.        Graphs constructed to indicate LA performance vs. %FSM indicate that Vale performance generally matches or is above expectation for all non core subjects for 2009.

.        NC Level 6+ performance for 2009 is generally at or above that for Wales. The lowest performing subjects are art, IT, music and PE while the highest performing are D&T, geography, history, MFL and Welsh 2nd language.

Key Stage 4 Performance (also see Appendix 1e)

.        Pupil outcomes are generally above the all Wales means for all performance indicators for the last few years. The (relatively) highest performing indicators are L2 Threshold, wider points score and science. The (relatively) lowest performing indicators are English, mathematics and % achieving 5+A*-C grades.

.        Gender differences are generally below those for Wales for the last few years.

.        Aggregated national benchmarking performance is at or better than expected except for L2+ Threshold, CSI and mathematics indicators while the highest performing indicators are again L2 Threshold and wider points score. This indicates that the weaker performance of English and mathematics impacts negatively on other KS4 indicators which they contribute to. Conversely, good performance in non-core subject performance impacts positively on other indicators. CSI benchmarking performance has declined over the last few years although no schools were below the 3rd quartile for this indicator for 2009.

.        Comparisons with other LAs (for 2008) indicate that Vale performance (rounded to nearest whole number) ranks no lower than 3rd for the main KS4 PIs (excluding core subjects).

.        Graphs constructed to indicate LA performance vs. %FSM (for 2008) indicate that Vale performance is generally above expectation for all PIs. L2 Threshold & average points score performance is significantly higher that expected.

 

17.     Consideration of School Inspection grades reveal the following:-

.         Primary schools ~ only 1% of all grades awarded was a grade 4

.         Secondary Schools ~ no grade 4s were awarded

.         Inspection Data for Special Schools and PRU: nothing below a grade 2 awarded for each Key Question.

 

18.      Value added is a relative measure, of the extent to which an individual pupil (or an identified cohort of pupils), make progress, which is better than expected, over a specified timeframe and is based on secure and valid judgements of prior attainment and previous best performance.  When this occurs, we say that an intervention programme or the education system as a whole, has added significant value to the pupil’s learning and achievement.  Specified timeframes can be termly, annually or over a key stage.  Value added can be demonstrated within and/or between key stages.  Implicit in the notion of value added, and underpinning any judgement, is the shared and collective understanding held amongst professionals, about expectations and the rate of progress over time, both within and between key stages.  Additionally, validity is enhanced when value added judgements are made based on the use of standardised data.  Where standardised test score are used, judgements about value added tend to be more secure and robust i.e. this is observed in the decreasing FFT residuals.

19.      The inherent inconsistencies in value added judgements based on teacher assessment, makes such evaluations both fragile and less robust.  Generally speaking, the greater the timeframe span e.g. KS 1 to KS 4 used to arrive at value added judgements, the greater the magnitude of any inconsistency in the process i.e. inconsistencies become accentuated over a larger timeframe and as a result, value added judgements become unreliable.

20.      Value added performance looks at the how well pupils move on to the next phase of their learning.  It looks at the extent to which pupils make progress against prior achievement and the extent to which pupils make progress in line with expectations.  The measure is essentially about tracking an individual pupil (or same cohort of pupils) and arriving at a judgment about how well the pupil has achieved over time and whether or not targets have been met or exceeded.  Implicit in this, is the nature and consistency in teacher assessment, the type of assessment used and the level of challenge underpinning individual pupil targets.  Therefore security and validity of value added judgments is enhanced and essentially more accurate when the same cohorts of learners are tracked as they move through the system.  As value added judgements depend on an analysis of previous best, the measure is a relative one, and as such, dependent upon a range of factors at various stages in the system, as above.

21.      Other factors which may affect value added include:-

.          The percentage of pupils presenting as BME/EAL with limited fluency in English at any one time, and trends in admission of these pupils at  various key stages.

.          The mobility rate, as pupils move into the education system and accordingly move out.

.          The number of SEN pupils and the extent to which this data is included in schools overall performance data.

.          The consistency in teacher assessment and the type of assessments used e.g. Baseline.

 

22.       For the purpose of this report, and across Wales as a whole, value added is expressed as a rank order position calculated when the Vale is compared with each of the 22 Local Unitary Authorities in Wales, for any particular PI.  The position of the Vale of Glamorgan is detailed in Appendix 1.

23.       The value added comparative data within the Vale of Glamorgan is subject to the concerns about accuracy explained in paragraphs 20 to 23 above.  Whilst this means that the data is likely to be less reliable than the actual data presented to members, it is a data set which will permit closer examination by the Education Service and schools.

24.       Additionally, it is recognised that the following information has underpinned Service Objectives included in the School Improvement Service Plan for 2009/2010:

.           The gap in performance at level 5 in KS 2 is closing i.e. there are fewer numbers of pupils achieving a level 5, particularly in English.

.           Gender differences exceed that observed across Wales for 2008 in Welsh and Science at KS2.

.           Benchmarking for 2009 demonstrates that only 14% of schools were placed in the UQ which is lower than that observed for performance in KS1.

.           A downward trend in benchmarking for the period 2007 /09 is observed for schools placed in the UQ in KS2.

.           Relative performance of IT & PE at KS3.

.           Gender difference in outcomes in science, MFL, PE & Welsh 2nd language at KS3.

.           Level 6+ performance for art, IT, music & PE at KS3.

 

25.        Questions for further discussion include:-

.            What is the underpinning rationale for the Key Stage timeframe currently being used to determine value added? i.e. KS 2 to KS 4.  Is there a reason for not adopting baseline to KS4?

.            Can we be sure that we are measuring like with like? i.e. some schools continued to use SATs in 2005 while others used teacher assessment only. All schools currently use teacher assessment only.

.            As there is no standardised teacher assessment methodology currently in use in Wales, is it wise to judge measures of value added on the basis of systems which do not meet basic reliability tests.

.            How is pupil turnover managed within the data.  Given the fact that some schools have high levels of pupil turnover, data needs to be attributed to individual pupils, not to year group populations in order to arrive at reliable information about value added.

.            Is there an expectation that authorities with comparatively high levels of EAL pupils will have depressed value added outcomes?

.            Does the data include pupils in Special Schools?  The comparatively high special school population which the Vale will be a factor, if this is the case.

.            How would the VAP change if it was analysed against categories of % FSM for each Key Stage?

Conclusions

26.        There is real merit in developing our understanding of value added trends as this has the potential to help identify areas where targeted intervention may impact upon outcomes.  As a result, specific research will be commissioned to ensure that the Directorate has a clearer picture of value added that can be shared with schools. The research will be conducted by our in-house Value Added Statistical Team.

27.        We envisage the research will centre on some of the following key areas:-

.            An in-depth analysis of current data for those pupils who have remained within Vale schools.

.            An in-depth analysis of  NFER data which is likely to be more standardised and comparing our performance with other local Authorities where NFER is also used.

.            Undertaking an historical comparison with the SATS based value added data previously available, to ascertain whether this data reflects the same trends, albeit with different cohorts.

.            Additionally, the School Improvement Service is committed to working with all stakeholders in the Vale i.e. Headteachers, teachers, pupils, Governors and support staff, in order to continue to secure improved outcomes for all our children and young people, across a range of PIs, including that of value added.

Resource Implications (Financial and Employment)

28.        There are no revenue implications arising directly from this report.

Legal Implications (to Include Human Rights Implications)

29.        None.

Crime and Disorder Implications

30.        The achievement of successful learning outcomes for pupils will help reduce the risks of increased crime and disorder associated with unskilled and unqualified young people.

Equal Opportunities Implications (to include Welsh Language issues)

31.        The Council's commitments to Welsh Medium Education in respect of the performance data.

Corporate/Service Objectives

32.        Lifelong Learning.

Policy Framework and Budget

33.        This matter is for Executive decision by the Cabinet.

Consultation (including Ward Member Consultation)

34.        No consultation necessary at this stage.

Relevant Scrutiny Committee

35.        Lifelong Learning.

Background Papers

Welsh Assembly Government examination data.

Contact Officer

Lynette Jones, Head of School Improvement

Officers Consulted

None

 

Responsible Officer:

Bryan Jeffreys

Director of Learning & Development