Data informed teaching

There's so much data#

Summary/points from Selwyn (2016) seeking to explore empirically the everyday use of data from a social perspective. Over a year at two schools - part of broader research. Research questions

What conditions and logics of governance are at work within schools?
What forms of data generation and data work underpin this governance?
What are the consequences of different forms of data-based governance?

Concerns about data supporting effective governance

issues of reductionism and the privileging of an ‘instrumental rationality’ that presumes the disaggregation of complex social and cultural situations into neatly modelled and calculable problems that can be addressed through computational means (Mattern, 2013)
exacerbation of unequal social relations between powerful and non-powerful groups through data-based calculations and judgments (Selwyn, 2015)

Found three distinct forms of data-based governance and a limitation

for 'system-wide accountability
- Indicators of institutional performance: NAPLAN, VCE exams, standardised online surveys from school goernment (MySchool)
- state wide - enterprise system as the "source of truth for all the core student data"
- school based system to share within school stuff "source of truth" within the school
- lots of manual/admin work feeding these systems, some of it very manual (per student), no uploads
- lots of centralised analysis and report generation
- primarily for the purpose of administrative compliance
- central systems provide little ability to school folk to query/analyse, even limited in school's ES
for 'within-school' accountability
- school identified areas of L&T to focus on improvement
- one school has teachers inputting indicative grades every 4 weeks to enable school leader review
- also scraping individual student's Google site/portfolio for goals
- other school using eventually compulsory online surveys
- data processing/analysis done by small school leadership team
- all fairly simple with spreadsheets and "traffic lights" and other simple representations
for 'within-classroom' accountability
- ad hoc, personally driven by keen teachers
- often focused on quantified indicators of performance, improvement and effective teaching/learning
- in-class quizzes, self-report exercises as progress indicators
the perceived limitations of these forms of data generation and data work
- data often used by school leaders as part of presentations
- but indications that little was done with the data (e.g. reflection data focused more on ensuring all students had entered the data)
- backward focus of much of the data seen as unhelpful for future planning
- issues with the abstracted nature of the data
- also partial in coverage - personal issues, reliance on f-t-f meetings.

Instead, the data regimes in both schools were distinguished primarily by their mundane and curtailed nature. The data practices, flows and circulations in both schools were most likely to take the form of simple arithmetic rather than complex algorithms; manual rather than automated calculations; and a reliance on unsophisticated technical and classificatory procedures.

QCoT research digest: data in schools#

Using data to improve learning poses many challenges. A significant challenge is that of transforming data into information which leads teachers to improve the learning of their students. Presenting data in ways that encourage teachers to take on a questioning, problem-solving role (scientist practitioner) with respect to their students’ learning causes changes in their teaching practice and results in improvements in student learning. (Axworthy, 2005)

Sources of data about student achievement

standardised, norm-referenced, criterion-referenced tests
questioning in class
performance and standards-based assessments
teacher-made tests, projects, quizzes
teachers’ observations
student work.

Earlier mentions how "educational data were slow to turn around" but digital technology has helped. But then most of the sources above are not readily available in digital form, and typically there is some complexity in weaving contextually together the disparate data sources.

References work in WA on "Data Club" - stepping toward PIRAC

There is evidence that the best support for interpreting data is a combination of information about data analysis and representation strategies together with application of this information to participants’ own data. (Wildy, 2003)

But at the same stage identifies the driver "accountability and data are at the heart of contemporary reform efforts worldwide" (Earl 2005), but pointing out the complementary/more agentic framing as "Becoming inquiry-minded and data literate are major changes in practice that are consistent with the notion of professional learning communities and that warrant concerted attention to new shared learning" (Earl 2005)

Uses case study that reinforces benefits from using data as part of a PLC

And of course the promises "Significant challenges have been overcome through the devleopment of state-of-the-art information and communication system (ICT) that bring complex data to the finger tips of staff in schools and regions in highly usable forms" (Smith 2005)

Examples given are focused on overall results. Coarse data that fails to capture the complexity of the lived experience

Purposes for using data#

Argues that for teachers, this is improving students' learning. Using literature to position it as differently as from

the sociologist (seeking to undrestand patterns of participation)
the policy analyst (understand the impact of policy)

i.e. echoing Dron's take on ed tech, it's all about the purpose for which the technology is being shaped.

Cites Hattie as arguing for data/evidence needs to be located in the classroom. The argument being that the teacher is the "major agent that influences student and learning". But which also points to an argument for a "teacher-shared language about the achievement progression". See table below from Hattie (2003)

The reason for locating the power of data to enhance student outcomes at the teacher level comes from the many recent studies on the epicentre of causal effects on learning: the teachers. (Hattie, 2005)

Source	% impact
Student	50
Teacher	30
Home, peers, schools and principals	20

A characteristic of good data is its potential to help teachers make good decisions about children’s learning. Data tell a story. ... The two questions uppermost in teachers’ minds should be: What does it all mean and; how can we use it to improve children’s achievements?

Does performance feedback lead to improvement?#

Argument seems to be it's complex, but it helps if

feedback directs attention to an achieveable gap
formative assessment/assessment-for-learning is key, but "current practices are week" -- from 2003 observation

On a large scale#

Interesting quote from Kirkup et al (2005) - which establishes link with PIRAC

‘Good practice’ emerged from the use to which the data was put rather than specific systems or tools. A recurrent theme was that data only becomes effective if it stimulates questions about the actual learning that is taking place and how it can be developed further. (Kirkup, et al., 2005)

From Kirkup et al (2005) effective use of data

informs accurate curricular targets for individual pupils
highlights weaknesses in specific topics for classes or groups
highlights specific weaknesses for individual pupils
provides evidence to support decisions as to where to focus resources and teaching
informs setting and grouping of pupils.

Some of this may not apply to all teaching approaches. e.g. Thinking Classroom's focus on random groups.

References#

QCoT. (2008). Data informed teaching (QCoT Research Digest). QCoT. https://cdn.qct.edu.au/pdf/Research%20Periodicals/QCTResearchDigest2008-3.pdf

Selwyn, N. (2016). 'There's so much data': Exploring the realities of data-based school governance. European Educational Research Journal, 15(1), 54--68. https://doi.org/10.1177/1474904115602909