Pass the SoLT

This week the New Zealand Ministry of Education released the latest report (well actually an annotated bibliography) identifying major themes and messages for the tertiary sector from an analysis of the e-learning literature. The key findings of the report entitled "Learners' Participation, Retention and Success in e-learning: An Annotated Bibliography" include few surprises and generally underscores the importance of institutional policy, teacher pedagogy and the way in which e-learning is part (or not) of the learning culture. Derek Wenmoth provides a useful summary of the major themes.

Instead of restating or repeating Derek's blog posting, and the report's Executive Summary, in the spirit of constructive critique there are a number of statements and conclusions that do not sit comfortably with me. For example, arguably one of the most serious flaws in the report is the claim associated with learning styles. The report concludes that “Learning styles are predictors and determinants of learner outcomes” (p.11). As far as I'm concerned the learning styles literature has been well and truly discredited by a number of leading scholars and there is very little evidence supporting this movement. Here is a link to just one report of many in recent years that challenges the validity of learning styles. Indeed some proponents go so far as to totally reject the claim that learning styles exist and in many respects the literature in this area is nothing more than pop psychology. I'm very surprised to see such high level endorsement of learning styles in a Ministry of Eduction report and it raises questions about whether the report was subject to sufficient peer review.

Another statement in the report that raised eyebrows to say the least is the claim that “females prefer a more collaborative, interactive approach whereas males tend to take a more functional approach” (p.14). Such sweeping generalisations are not helpful and this type of conclusion is not as well empirically grounded as common folk knowledge would suggest. Again I'm most surprised to read such a statement in a Ministry report which tends to discredit the validity of other more solidly grounded conclusions.

The other issue I have with this report is a common flaw throughout the e-learning literature in that the use of the term e-learning encompasses a range of practices. Arguably relatively little can be gleamed from studies that do not unpack the different assumptions, pedagogical orientations and tools of e-learning which range from reinforcing the practice of multi-choice tests to supporting media rich and highly complex online scenarios exploring wicked problems. Put another way e-learning is not a stable or fixed entity that can be studied as an independent variable.

At Massey University we intend to devote a forthcoming “Hot Topic” lunchbox session to discussing the major themes and messages in this report. I've already invited the author of the Ministry report to attend this session in order to engender some lively discussion.

Image Source:

http://www.flickr.com/photos/ghholmes2/5426242071/sizes/m/in/photostream

A recent paper entitled 'Talking Back to Theory' by Sue Bennett and Martin Oliver in the latest issue of Research in Learning Technology is one of the more thought-provoking reads of 2011.

The paper argues that research into learning technology is driven by far too much rhetoric and more explicit attention needs to be given to theory. It makes a case for greater engagement with theory through examples from three case studies and goes some way to illustrating the value of creating a more dynamic relationship between theory and practice. Although I'm not sure I totally agree with the sweeping claim that 'theory has had a relatively small role to play in learning technology research to date', there is no question that the field is dominated by 'hype and excitement rather than evidence' (Bennett & Oliver, 2011, p.179). Moreover, a pragmatic interest in exploring the affordances of the latest technological developments is often at the expense of serious theorizing.

Having said that, it's important to recognise that social, economic and educational theories (at least the established ones) can be blinding and rather than 'make sense' of the affordances and possibilities of new learning technologies through our existing theoretical lens, we need to engage in theory building in order to truly imagine alternative learning futures. In particular, I'm talking about futures which are more equitable, socially just and sustainable... for all. Nevertheless, I can't disagree with the following conclusion:

'Research in learning technology has focused on practical, instrumental concerns, to the detriment of its ability to engage with theory. Previous reviews have show that theory has been relatively neglected, with most of those examples that do use theory best characterised as applying it rather than engaging with it in a critical or scholarly way. This situation risks turning the field into a narrow and derivative area of work: at best, only able to draw from other areas; and at worst, only of relevance to those with a vested interest in the specific practical situation currently under study' (p.187).

Let's hope we see more creative and imaginative theories in 2012 as real alternatives solutions to the deep-seated problems we face, which have haunted us over the last 12 months, as our futures really do depend on them. That's probably a good note to sign off for the year as I plan to take a few weeks of much needed rest and relaxation.

Footnote: I need to state for the record that many of our so-called theories are based at best on pop psychology and unsubstantiated sweeping generalizations, so I'm not advocating for more creative theories without supporting evidence. Put bluntly the field suffers from far too much rubbish written in the name of theory, as illustrated by concepts such as the digital native.

The annual US-based ECAR study of undergraduate students' use of technology provides useful baseline and comparative data. The survey has been conducted since 2004 and includes a nationally representative sample of 3,000 students in 1,179 colleges and universities. In brief, the study reports:

• Students are drawn to hot technologies, but they rely on more traditional devices
• Students report technology delivers major academic benefits
• Students report uneven perceptions of institutions' and instructors' use of technology

There is nothing new hear except for the role traditional as opposed to 'hot' technologies still play in students' study, lives and work. The lumpy nature of how students experience technology within institutions remains an ongoing challenge.

This is just a quick comment for readers who may have missed this paper in the British Journal of Educational Technology which explores trends of e-learning research from the year 2000 (Hung, 2010). The study is a bit dated now and I'm being lazy by copying the Abstract below without any real analysis.

However, in plain language, the paper identifies two major growth stages in e-learning research and concludes the literature is still at the early majority stage in the diffusion of innovation model. Having said that, importantly, comparing the effectiveness of e-learning with that of traditional instruction is no longer a popular topic, which is pleasing to read given the methodological issues associated with this type of comparative research. It is also reassuring to hear that the research based on this bibliometric analysis is now focusing more on the educational aspects of use and implementation rather than technical dimensions of elearning. The paper would be a useful discussion piece to use with a cohort of postgraduate students reflecting on what the research tells us about the use of elearning in education.

Abstract
This study investigated the longitudinal trends of e-learning research using text mining techniques. Six hundred and eighty-nine (689) refereed journal articles and proceedings were retrieved from the Science Citation Index/Social Science Citation Index database in the period from 2000 to 2008. All e-learning publications were grouped into two domains with four groups/15 clusters based on abstract analysis. Three additional variables: subject areas, prolific countries and prolific journals were applied to data analysis and data interpretation. Conclusions include that e-learning research is at the early majority stage and foci have shifted from issues of the effectiveness of e-learning to teaching and learning practices. Educational studies and projects and e-learning application in medical education and training are growing fields with the highest potential for future research. Approaches to e-learning differ between leading countries and early adopter countries, and government policies play an important role in shaping the results.

Full paper available through the Massey University Library for staff and students.

enjoy!

The latest issue of the Review of Educational Research contains an article that attempts to bring together more than 40 years of investigations on the general question, "Does computer technology use affect student achievement in formal face-to-face classrooms as compared to classrooms that do not use technology?”

In this commentary I just want to highlight a few key observations rather than provide a detailed account of the statistical findings, as there are unresolved methodological issues in completing a second order meta-analysis of the research literature. If you have a head for numbers then you may wish to drill down through the statistics for yourself to see how the authors calculated and validated the effect sizes. If you just want to read the highlights of the statistical analysis then the Abstract may give you sufficient information.

For me the narrative or interpretation of the findings makes more interesting reading. As I commented last year in reporting Bernard et. al., (2009) meta-analysis of distance education research, this is another publication which hopefully puts to bed the value of crude comparison studies. The authors write:

"... similar to classroom comparative studies of distance education and online learning (e.g., Bernard et al., 2004, 2009), we feel that we are at a place where a shift from technology versus no technology studies to more nuanced studies comparing different conditions, both involving CBI treatments, would help the field progress" (Tamim, et. al., 2011, p. 16).

They go on to say...

"And because such a rich corpus of meta-analyses exists, spanning virtually the entire history of technology integration in education, we feel that it may be unnecessary to mount yet another massive systematic review, limited to technology versus no-technology studies" Tamim, et. al., 2011, p. 16).

Although there is a danger of adopting an overly socially deterministic understanding of the impact of technology on learning, the paper supports Clark’s (1983, 1994) view that technology serves at the pleasure of instructional design, pedagogical approaches, and teacher practices. Tamim, et. al. (2011) state that they generally agree with the view of Ross, Morrison, and Lowther (2010) that “educational technology is not a homogeneous ‘intervention’ but a broad variety of modalities, tools, and strategies for learning" (p.19). What this means is that the effectiveness of educational technology depends on how well it helps teachers and students achieve their desired instructional goals” (p. 19).

Following this line argument the key point is that '...it is aspects of the goals of instruction, pedagogy, teacher effectiveness, subject matter, age level, fidelity of technology implementation, and possibly other factors that may represent more powerful influences on effect sizes than the nature of the technology intervention' (Tamim, et. al., 2011, p. 17).

The authors go on to conclude that it is incumbent on future researchers to unpack these nuances, so that technology is used as effectively as possible to support the aims of teaching and learning. Put another way, we need more ecological research on the conditions under which educational technology contributes to learning--for better and worse. This is not a new argument and there is a sense in which history is repeating itself almost two decades on from the early work of Gavriel Salomon and colleagues.

References

Tamim, R., Bernard, R., Borokhovski, E., Abrami, P., & Schmid, R. (2011). What forty years of research says about the impact of technology on learning: A second-order meta-analysis and validation study. Review of Educational Research, 81(1), 4-28

For Massey University staff and students a full version of this article is available through the Library.

The Times Higher Education reports the idea that online learning is a "poor substitute" for campus provision is a myth. According to the head of the UK's Online Learning Task Force, Dame Lynne Brindley, chief executive of the British Library, the best examples of online learning are "not cheap alternatives" and require "deep consideration".

The Task Force includes experts from Microsoft, Apple and Pearson, and the heads of The Open University, King's College London and the University of Wolverhampton.

Brindley goes on the say the task force is "trying to dispel some of the myths that online learning is a second-rate alternative". Moreover, given the financial crisis, it seems likely that online study with UK institutions will become an increasingly attractive option for overseas students. There is, she believes, no time to waste. It is clear that competition is growing internationally, while private providers' ambitions in the field are expanding.

A separate study conducted by the National Union of Students is reported which examined demand for online learning among UK students. The report claims that students' technological expectations were broadly being met, but that there were concerns about "a gap between the ability of students to use these technologies and the ability and willingness of (academics) to use them". Of course, this is not a new concern and it should not be dismissed as merely a generational gap as the barriers to the pedagogically sound and appropriate use of online learning are far more complex.