Data 5x4 - Research-based Design

Designed-based research project

Data 5x4

Data 5x4 Framework Design is a design-based research project rooted in teacher's needs of data literacy training. Today’s teachers are expected to develop abilities in questioning, analyzing, and interpreting assessment data as well as using data to make instructional decision-making. However, DDDM is a highly complex process and requires systematic thinking process (Mandinach & Gummer, 2016). Researchers and practitioners have attempted to identify the framework of DDDM (Gill, Borden, & Hallgren, 2014; Mandinach & Gummer, 2016; Marsh, 2012) and the definition of data literacy (Marsh, 2012; Mandinach & Gummer, 2012).

After close examination and analysis data literacy literature and interventions such as workshops, data coaching, comprehensive district-wide reforms, and policy-mandated interventions (Coburn & Turner, 2011), we identified three required skills in data use for decision-making and utilizing 4C/ID model (van Merrenboer, Clark, & De Croock, 2002) as our framework in designing D5x4 Framework workshop intervention. The designing concept and the purpose of this intervention are to ensure the novice participant’s cognitive schema is constructed and transferred to produce reasonably complex learning.

Design Process

Stage I

Needs Assessment

1. Data use for Decision Making
Whilst we studied existing data literacy interventions such as workshops, data coaching, comprehensive district-wide reforms, and policy-mandated interventions (Coburn & Turner, 2011), we identify three levels in data use for decision-making:

the types of data (e.g., qualitative and quantitative, boxplot, charts),
the differentiation of data aggregation (e.g., individual, classroom, subgroup),
the use of data to ask and answer different questions in data interpretation (e.g., data identification, instruction, strength and weakness).

2. Challenges

Analyzing from the literature, we concluded whether the participants were in- and pre-service teachers, they all shared common challenges:

have anxiety and low self-efficacy when working with data (Jimerson, Choate, & Dietz, 2015),
Struggles in interpretation various qualitative and quantitative data through all aggregation (Pierce, Chick, Watson, Les, & Dalton, 2014),
Struggling with minimal opportunities to learn how to interpret standardized test data for decision-making (Mandinach, Friedman, & Gummer, 2015).

3. Define the Needs

After identifying the design framework, we merged 4C/ID instructional design concept in designing and supporting novice learners’ recurrent constituent skills and non-constituent skills in data types, interpretation and use. The fundamental concept of D5x4 Framework is to build teachers' data literacy skills in a holistically way:

vertically to build their skills in interpretation data by asking and answering four types of questions (i.e., location and data identification, strength and weakness, status and growth, and instruction) and
horizontally to build their skills in examining students' data in five student aggregation categories (i.e., individual, subgroup, classroom, grade, and school)

Stage II

Design Concept

1. Identify Learning Theory

Schema theory is known that human has a tendency to learns from previous experience and learn better when schemata are accumulated over time. knowledge then is unpacked through consistent organize, merge, retain, and retrieve from the given information thus enable learners to interpret events and solve different problems (Rumelhart, 1980). To ensure meaningful learning occurs, we adopt schema theory into our instructional systematic design by building structure learning blocks to assist participants to form the generic knowledge of four types of questions learning schema in data interpretation (Figure 1).

Figure 1: The illustration of the intervention design aspect and theoretical framework

2. Design of Framework

In our approach, we created four types of question (location and data identification, strength and weakness, status and growth, and instruction) learning schema at five different levels. Each type of question contains different data forms to build learners’ schemata of the standardized data format and questions. Thus, a repeated learning process on facets of assessment and data use was formed within a systematic learning procedure: identifying, analyzing, interpreting, summarizing, comparing, and making decisions. The learning sequence starts at the individual level, subgroup level, class level, grade level, and school level. Participants with or without prior knowledge will build or gain the new schema of data use from this learning process.

The table below demonstrates the alignment of the design concept of D5X4 with the four components in the 4C/ID Model. Please click the table to enlarge the content.

Stage III

Design & Development

Jui-Ling (Raye) Chiang