Conceptualising education design practice - where do we fit?#

Chat with Emma#

My aim is to better understand the work they're doing and figure out how I integrate it into my practice.

Appears to be focused on exploring the if, how, and utility of using Goodyear's conceptualistion of the educational design problem space pedagogical framework to "Evidence-informed IT planning, procurement, and evaluation" - largely drawn from the slide/diagram.

Questions

What have I missed from the purpose?
Where is it up to?

Start - draft#

The first line of the abstract for Goodyear (2005) is

There is a growing demand for advice about effective, time efficient ways of using ICT to support student learning in higher education.

Almost 20 years later, higher education is still trying solve this problem.

"Despite the increased use of digital resources and technology along with the desire for increased flexibility, most participants did not agree that faculty had the skills and know-how to effectively deliver courses in multiple modes." #highered https://t.co/OEVe1PXiqn
— Neil Mosley (@neilmosley5) April 20, 2023

Echoing the impression I took away from the THETA'2023 conference a couple of weeks ago (apart from all the AI/LLM discussions). Jason Lodge and colleagues presented on work they are doing to build on the work of Goodyear (2005) to inform IT planning, procurement, and evaluation. Steven Booten, Henry Cook, and I shared our experience from an LMS migration.

In that presentation, we derived principles which have appear to have broader application in addressing the core problem both Goodyear and Lodge et al were focused on - providing effective and efficient use of digital technologies to support learning and teaching. something about productivity commission The following seeks to explore the relationship between Goodyear (2005), subsequent work from Goodyear and others, and what we're doing.

TLDR?

The gather, weave, augment analogy we use is evident in the work of Goodyear and Lodge.
From a linear education design and development perspective (see Table 1), Goodyear's work focuses on educational design. Lodge et al appear to be coming from design into the early stages of development (procurement of digital tools). Our work is focused explicitly on contextual educational development - working with the available tools in a context.
Of course, educational design and development is entangled and not linear. We argue that we need to get better at recognising and responding to this reality.
While we demonstrate the need for our work, it is also evident in Goodyear's other writing, including:
- McAndrew, Goodyear & Dalziel (2006) identify a weakness of Goodyear's (2005) work in the complexity of implementing design patterns in the LMS (educational development). A task which appears to have only gotten more difficult.
- Goodyear (2009) suggests that gathering and weaving design knowledge into learning and teaching tools as a method for informing design practice.

Leaving the big questions of if/how we do this better and what happens as a result?

The question of where does the following fit? - Tim Klapdoor's ASCILITE work - qutoe from ASCILITE by the UTS person - ABC learning design stuff

The educational design problem space, networked learning, and design patterns#

Goodyear's (2005) focus is on networked learning. Defined as learning in which digital technologies are used to promote learner-learner, learner-teacher, learning community-resources connections. It is not just putting materials online or distance learning. The central element is the connectedness, not the digital technologies. To be successful, networked learning required effective design. Educational design is defined and distinguished from educational development as follows.

Table 1. Definitions of educational design and development (Goodyear, 2015, p. 82)

Term	Definition
educational design	"set of practices involved in constructing representations of how to support learning in particular cases"
educational development	"the practices of turning these representations into support of learning (materials, task specifications, tools, etc)"

While recognising that resources are provided to help teachers with educational design work, Goodyear (2005) that such resources

are (a) hard to relate to one another, and (b) hard to locate in relation to a particular pedagogical framework. Moreover, © it is unusual to find examples and templates constructed in such a way that they capture, and distil the practical implications of, research based knowledge or (d) sit comfortably with the iterative nature of design practice (p. 92).

This critique and his suggested solution draws on the following model (Figure 1 of the problem space of educational design and a description of the typical educational design process typically used by an individual teacher.

Goodyear (2005) suggests that and explains how design patterns and pattern languages for network learning can help address the identified limitations. In particular, Goodyear (2005, p. 92) suggests that design patterns can help educational design by:

Providing the teacher-designer with a comprehensive set of design ideas

Providing these design ideas in a structured way – so that relations between design components (design patterns) are easy to understand

Combining a clear articulation of a design problem and a design solution, and offering a rationale which bridges between pedagogical philosophy, research based evidence and experiential knowledge of design

Encoding this knowledge in such a way that it supports an iterative, fluid, process of design, extending over hours or days.

Evidence-informed IT planning, procurement, and evaluation in higher education#

Jason Lodge and colleagues gave some early insight into on-going work that is leveraging Goodyear's (2005) conceptualisation of the educational design space to inform IT procurement. Figure 2 is a visual representation of their extension. Beyond adding more detail to Goodyear's conceptualisation of the pedagogical framework, it seeks to connect the work of educational design with educational development. Apparently with the intent of ensuring that the result of technology procurement will be fit for purpose.

Random thoughts - done a lot more expanding out the pedagogical framework - But perhaps the arrows are too coarse grained - And the boxes of the various components of LMS are too blackbox - don't represent the entangled nature

What went "wrong" with design patterns?#

There was a lot work in the mid-noughties on learning design patterns work in the mid-noughties (e.g. the e-LEN project and the Pedagogical Patterns group). Work inspired by both Alexander's originating work on design patterns in architecture and the adoption of that work by the object-oriented design & programming (OOP) community. As an occasional practitioner in both the OOP and the learning design communities, my observation is that design patterns have been more successfully integrated into the practice of the OOP community.

Anecdotally, I've not seen anything like this in action over recent years. It doesn't appear to have entered the mainstream.

McAndrew, Goodyear, and Dalziel (2006) examine three different ways to support educational design with a particular focus on how effectively they may descriptions of learning designs that are useful for creation, re-use, and adaptation. Design patterns are compared with the IMS Learning Design specification and learning activities as used in the Learning Activity Management System (LAMS). The IMS work offers a formal specification for how to describe quite complex learning activities and the roles involved. A specification that can be then used to digitally use, share, and adapt learning activities. LAMS is (was?) a specific information systems that could be used for learning and teaching. Central to LAMS design was the assumption that learning is "people doing activities with resources". It supported design through the sequencing of learning different activities for use by students.

Table 2. comparing issues in LAMS, IMS Learning Design, and Patterns (adapted from McAndrew et al, 2006, pp. 217--218)

Issue/Question	LAMS	IMS Learning Design	Patterns
Features
Representation	Visual sequence flow & embedded text	XML & UML	Stylised sequence of expository text
How do you modify?	Rearrange visual flow and rework task text.	Rework XML/UML	Rework expository text
How do you aggregate?	Collect sequences within folders	Build bigger designs with sub units.	Create pattern language
What is missing?	Pedagogic wizard	Abstract tool definitions and operational links. Pedagogic wizard.	Learning Management System (LMS) and the expertise to get the pattern into the LMS
Users
Who can easily understand?	Academic Practitioner with a little technical knowledge	Technically aware expert	Academic Practitioner
What is the minimal prior knowledge for use?	Some pedagogical knowledge, Some technical knowledge	Some pedagogic knowledge, high technical ability	Only pedagogically adept teachers
What does ideal use require?	More pedagogic knowledge and technical understanding	More pedagogic knowledge and technical understanding	Pedagogically adept teachers linked to moderate technical knowledge
Characteristics
Is a creative jump necessary for implementation	No	No	Yes
Minimal complexity in design	Small	Small	Great
Ease of adaptation mid-stream with students	Hard	Hard	Easy (though depends on supporting technology)
Potential for student participation in creation of design	Limited – only possible prior to running the design.	Moderate	Extensive

Arising out of web development and user experience (UX) design systems are a way to describe visual and interactive design components in a way that is useful for creation, re-use, and adaptation (Churchill, 2019). Use of design systems intends to help design teams document, share, and communicate to make the design process more efficient and effective (MacDonald, 2019). Werle (2021) argues that a key turning point for design systems was an evolution from being libraries of guidelines followed during design practice into libraries of live artifacts that embody those guidelines but could also be used in development.

Given the increasing use of web-based systems for networked learning and increased use of design systems to design web-based system it was perhaps only a matter of time before learning design systems were developed. For example, Joyce Seitzinger gave a webinar titled How learning design systems can help scale and accelerate learning design. Tim Klapdoor has shared descriptions of the work his team have been doing on A Learning Design System and later work on Design at scale. Work that links design systems with a content management system that actively helps in designing lessons (learning experiences?) and in turn draws on a library of learning patterns. But as Tim notes, work that has been focused on providing his team of learning designers tools to help them do their job effectively.

The library learning patterns from Tim's group don't follow the Alexandrian format outlined in Goodyear (2015). Hence, don't explicitly provide advice covering the pedagogical framework section in Figure 1. One assumes because they have been initially designed to help learning designers - folk generally expected to have some expertise in this area - rather than teachers. However, that absence may still limit the sharing/communicative aspects of design patterns amongst the designers.

In my limited experience, I've also not seen any design system that implements fully Werle's (2021) key turning point for design systems. That is, include libraries of live artifacts that can be used in development. At least one example design system provides components for this purpose but they demonstrate the origins of design systems by focusing on user interface design, rather than learning design. This may say more about the limits of my experience. It may be related to the initial work on design systems being focused on helping learning designers engage in educational design. Development is still coming?

The missing entanglement of educational design and development?#

It's fairly typical for educational design and development (as defined in Table 1) to be treated separately both in practice and research The work that Steven, Henry and I have been doing is dealing with the entanglement of educational design and development.

In fact, the definition of design patterns is that they are abstract descriptions of a design that can be implemented in many different, contextually appropriate ways...lead to weakness

refenrece paper on learning objects

Structural complexity#

Goodyear and Carvalho (2016)

So the challenge comes from numbers and a structural complexity that arises – in large part – from the freedom that people have to distribute their activity across a wide number of platforms and spaces

Following on from that gather/weave comment, came across the following quote from ⁠Goodyear & Carvahlo, which is useful in our context. In particular and from a somewhat technical perspective, this echoes the challenge of moving from the single LMS to the Griffith VLE (i.e. an ecosystem of tools). A shift from the ⁠integrated approach to a best of breed approach which has literature echoing back to 2001 which identified one of the challenges of this shift - integration of applications is time consuming. i.e. the structural complexity has increased.

End structural complexity#

The question is, does this approach work?

Goodyear (2005) mentions the use of patterns by the object-oriented programming (OOPs) crowd as an example of another discipline adopting Alexander's architectural design patterns work. While not universal, design patterns remain fairly widely used by the OOPs community. Much more so than in higher education learning and teaching. Why?

	OOP	Higher Ed L&T
Design, develop, maintain	Software applications	L&T environments & activities
Practitioner	Primarily OO programmers	Primarily teaching staff
Practitioner purpose	OO programming	Research etc, maybe L&T
Practitioner expertise	OO programming	Disciplinary, bit of L&T etc
Support tools	Integrated development environments	Ad hoc collection of institutional tools with only a few integrated
Focus on improving	Developer experience	Student experience

Perhaps it is because the translation of the design problem into practice/artefact is more straight forward and consistent in OO programming than higher ed L&T. In OO programming, the design pattern the software developer translates it into code (that can be tested etc) using increasingly fairly common integrated development practices and tools. Knowledge of those practices and tools are seen as fairly central to the expertise of the software developer. Software development itself has quite an explicit focus on improving the developer experience. On the other hand, in higher ed L&T translating a design pattern into a learning environment relies on the complex gathering and weaving of of whatever random collection of physical/digital tools and processes within their individual institution. This complex gathering and weaving is not typically seen as central to the expertise of the practitioner. Often quite the opposite. Also, there is typically no on-going focus on "teacher experience" that echoes the focus on the "developer experience".

THETA presentation - Henry, Steven and I
Jason Lodge and colleagues were wondering how to weave "what we know about learning" into we do and was harking back to Goodyear (2005)
If/how does our work fit/relate

Gather, weave, augment#


Adapting, the adaptation to include gather, weave & augment

Link this to - and also the idea of the typology of scale idea - Ahn et al (2019) - design in context - Ellis & Goodyear (2019) - "research practice partnership" stuff

But still need to spread it to the teachers and the students

Other examples of gather, weave, augment#

Student to student advice

Describing how one teacher evolved how she gathered and wove student-to-student advice for her course over time. Largely achieved using tools outside the institutional LMS, but also visible through that LMS. - Emble

RMIT developed tool to help educators gather and weave visual design knowledge to augment pages in Canvas course sites.
various other Canvas community stuff

References#

Ahn, J., Campos, F., Hays, M., & Digiacomo, D. (2019). Designing in Context: Reaching Beyond Usability in Learning Analytics Dashboard Design. Journal of Learning Analytics, 6(2), 70-85-70--85. https://doi.org/10.18608/jla.2019.62.5

Churchill, E. F. (2019). Scaling UX with design systems. Interactions, 26(5), 22--23. https://doi.org/10.1145/3352681

Goodyear, P., & Carvalho, L. (2016). Activity centred analysis and design in the evolution of learning networks. 10^th International Conference on Networked Learning. http://www.networkedlearningconference.org.uk/abstracts/pdf/P16.pdf

Inventado, P. S., Scupelli, P., Heffernan, C., & Heffernan, N. (2017). Feedback Design Patterns for Math Online Learning Systems. Proceedings of the 22^nd European Conference on Pattern Languages of Programs, 1--15. https://doi.org/10.1145/3147704.3147738

MacDonald, D. (2019). Patterns in design systems. In D. MacDonald (Ed.), Practical UI Patterns for Design Systems: Fast-Track Interaction Design for a Seamless User Experience (pp. 143--192). Apress. https://doi.org/10.1007/978-1-4842-4938-3_4

McAndrew, P., Goodyear, P., & Dalziel, J. (2006). Patterns, designs and activities: Unifying descriptions of learning structures. International Journal of Learning Technology, 2(2--3), 216--242. https://doi.org/10.1504/IJLT.2006.010632

Naimi-Akbar, I., Weurlander, M., & Barman, L. (2023). Teaching-learning in virtual learning environments: A matter of forced compromises away from student-centredness? Teaching in Higher Education, 0(0), 1--17. https://doi.org/10.1080/13562517.2023.2201674

Werle, D. R. (2021). Design Systems: Definitions and Main Elelments [Masters, Tallinn University]. https://www.idmaster.eu/wp-content/uploads/2022/01/Daniel-Werle-thesis-Vdrw-2.pdf