You are browsing the archive for assessment.

UAL Digital Learning Services Support Blog

March 3, 2017 in Uncategorized

Everything you wanted to know about Moodle, Workflow, OAT, myblog.arts, process.arts and more, now at http://technologyenhancedlearning.myblog.arts.ac.uk/

UAL Course Teams – want to discuss Digital Learning?

September 22, 2016 in Assessment, Digital Identity, Digital Literacy, eLearning, eTeam

If you are a Course Team at UAL and would like to discuss or explore ways of using Digital Learning within your courses – please contact UAL’s Digital Learning Team.

If you would like to understand a little more about how we work with course teams have a look here – or listen to colleagues discussing using digital learning spaces with their students.

Solstice Conference 2016, Edge Hill University

July 11, 2016 in Assessment, Digital Identity, eLearning

I had the great pleasure this year of again attending and presenting at the welcoming and fascinating Solstice / CLT Conference at Edge Hill University – full details on the conference website.

As part of this I presented on two subjects:

  • Exploring Online Teaching Spaces in an Art and Design Environment
  • Digital Feedback in an Art and Design University
This was the first time that I had chosen the ‘Pecha Kucha’ format – where you use 20 slides which are each visible for 20 seconds – and change automatically. It is a format which forces you to really hone your content and only leave in what is necessary for your message – which I found strangely liberating.
The links to my the slides I used for the Pecha Kuchas are:
The links to the slides I used are:
For the full ‘Pecha Kucha’ experience, click on the ‘PK’ link in the 2nd slide in each to access the timed presentation.

Learning outcomes and self assessment

December 14, 2015 in PGcert

Assessment is quite a new topic for me, as I generally work with courses on presentation and employability skills, and I’m not closely involved in the learning outcomes and assessment. The sessions I teach are very practical and slot into, and are adapted for, other people’s programmes.

First thoughts

The question I’m considering is: How useful do you think intended learning outcomes are in the teaching of your discipline?

I can see the benefit of planning ahead and shared expectations between staff and students, or “the implicit contract with students… is that only the preset criteria will be used in deciding the grade.” (Sadler, 2009, p167)

But learning outcomes seem potentially prescriptive, and I wonder if the language used could be challenging for arts and design students. For example, ‘aims, objectives and outcomes’ feel quite corporate. (Though this might be influenced by some of my reading, see illustration in the image below!)

Queen Marys photo

Danvers, J. says that “learning objectives determined by, and for, others” can alienate individuals from the learning process.  (Danvers, J. (2007) Qualitative rather than Quantitative: the assessment of arts education.) This teacher as authority feeling is echoed in guidelines from Oxford Brookes: “Once you, the teacher, have decided what knowledge and skills students will demonstrate…” (Carrol, J. (2001) Writing Learning Outcomes: some suggestions).

I like Danvers comments on the derivation of the word ‘assessment’, from the Latin ‘assidere’ which means ‘to sit beside’. This feels more equal, supportive and collaborative than the meaning commonly associated with ‘assessment’: “the ​act of ​judging or ​deciding the ​amount, ​value, ​quality, or ​importance of something” (Cambridge Dictionaries online).

Drafting learning outcomes 

I’ve spent a bit of time getting my head around how to write effective learning outcomes. I’ve written a summary of the reading I’ve done to help inform my understanding (Creating learning outcomes).

Creatin

So this is my first stab at writing down the activities and the learning outcomes:

What are our aims? Get students to practice making presentations

What are they going to do? Present to their peers as part of a workshop

How are we going to assess it? This is a tough one. It could be ‘Judge how well the students present’ but that doesn’t fit at all with my ethos (I explore this more at the end of this blog post).

Activities:

  • Prepare presentation
  • Talk in pairs to discuss presentation
  • Present in bigger groups
  • Give and receive feedback

Learning outcomes:

  • Participate in the workshop and speak during each exercise
  • Prepare a presentation to demonstrate your research work on your chosen film
  • Practice delivering your presentation to improve it
  • Listen to presentations from your peers and give feedback that will help them make improvements
  • Evaluate and apply feedback from your peers to develop and improve your presentation

Any feedback on how I could improve the learning outcomes would be much appreciated!

Activity

Now that I understand a bit more about the nuts and bolts of putting together learning outcomes, I am having a look through different approaches to assessment. I am most interested in emergent outcomes and activities. I tend to focus on activities in my teaching, possibly because my research has shown that you improve presentation skills by practicing.

We were asked the question ‘What’s one thing you’re good at, and how did you get good at it?’ in the online seminar. My answer was: ‘By doing a lot of it.’ Several people said ‘repetition’, and the biggest word on Lindsay’s word cloud was ‘experience’.

Activity seems to be central to ‘constructive alignment’, something that John Biggs’ talks about: “Learning is constructed by what activities the students carry out; learning is about what they do, not about what we teachers do.” John Biggs www.johnbiggs.com

I was particuarly interested in Elliott Eisner’s focus on activity theory, starting from the point of the activity. I found an essay called ‘Educational Objectives: Help or Hindrance?’ Eisner, E. (1967).  He says that the rational approach to curriculum development is to state objectives before the formulation of activities:

“At first view, this seems to be reasonable way to proceed with curriculum construction: one should know where he is headed before embarking on a trip. Yet, while the procedure of first identifying objectives before proceeding to identify activities is logically defensible, it is not necessarily the most psychologically efficient way to proceed. One can, and teachers often do, identify activities that seem useful, appropriate, or rich in educational opportunities, and from a consideration of what can be done in class, identify the objectives or possible consequences of using these activities.”

It’s the chicken and egg question: does the outcome come from the activity, or the activity from the outcome. In some ways, this circular process is appropriate. Esser-Hall says: “interpretation has no final result and each ending holds a new beginning”. Esser-Hall, G. (2000) ‘Perpetual Beginnings: The Role of Phenomenological Hermeneutics in Art Education’, The International Journal of Art & Design Education 19:3 (p289)

Danvers says: “Outcomes may well be unpredictable, unknown at the outset of an activity or only become apparent long after the supposed period of learning.” (Danvers, J. (2007) Qualitative rather than Quantitative: the assessment of arts education.)

He says that summative assessment “tends to develop convergent thinking at the expense of divergent thinking.” He thinks that the “increasingly deterministic emphasis on goal-orientated behaviour in which linear systematic processes lead to predictable outcomes”. In contrast, divergent thinking puts an emphasis on inventiveness and innovation, and creates a “willingness to revise, re-think and re-formulate” (Danvers, J. 2003 ‘Towards a Radical Pedagogy: Provisional Notes on Learning and Teaching in Art & Design’, The International Journal of Art & Design Education 22:1 (p.47–57)

I don’t like the idea of locking things down and not allowing creativity to thrive, or ignoring something that could be profound or transformative just because it doesn’t fit the criteria.

An example of an unintended learning outcome could be the invention of the microwave oven in 1945, after radar researcher Percy Spencer noticed his equipment had melted a bar of chocolate.

chocolate bar

 

Ipsative assessment and measuring improvement

I found this blog post by John Kleeman blog post quite helpful in understanding the idea of ipsative assessment. He says that it compares an individual’s results against his or her previous result, and how “progress and improvement is a useful thing to measure as well as achievement and competency”.

The study by Orr, S. & Bloxham, S. (2012, Making judgements about students making work: Lecturers’ assessment practices in art and design) on examining conversations of examiners when marking student work is interesting.

“…she starts weaker and you can really see now where she’s come on.”

For my sessions, it feels like the most important aspect is how much the individual has improved, and whether they feel more comfortable. As part of my evaluation, I ask students how they feel before the workshop and after it, so we can compare the two sets of data and track any difference, as well as any narrative feedback. This feels like it links well with ipsative assessment.

Although there are improvements that I notice during workshops with the students, I think my assessment of their progress is far less important than their own. The measure is ‘Do they feel more comfortable speaking’? It’s more about self-reflection, comparing their before and after, and formative assessment.

After thinking about this more, I’d now add another learning outcome:

Reflect on how you feel about making presentations before and after the workshop

Are intended learning outcomes useful in my work?

It feels useful to have some guidelines and shared expectations between staff and students, and for the students to know what they will be assessed on.

But it feels like there needs to be a degree of flexibility, and certainly take into account emergent outcomes, which may allow for greater creativity and also takes into account any unexpected learnings or demonstration of skill.

However, it also seems important to be consultative, to make sure that we don’t create learning outcomes that alienate students.

For my workshops, I do wonder if the assessment should be aimed at the teacher rather than the student. If the student doesn’t feel more comfortable speaking following the workshop, then I feel it’s because I’ve not fully understood their needs or feelings and I need to do more research in order to improve the teaching. For example, I want to do more research into international students to ensure that I am tailoring my teaching to their needs (see Topic 4) in order to make the workshop more effective.

Alignment and Assessment (Fifth Topic Task)

December 13, 2015 in topic task

1947 Anthropology Class Reed

I read chapters six and seven of Biggs and Tang (2011) which cover ‘Constructively aligned teaching and assessment’ and ‘Designing intended learning outcomes’. Chapter seven in particular was hard going, partly because B&T (as I will refer to them) throw in many other concepts such as the constructivist theory of learning and SOLO (Structure of Observed Learning Outcomes) which I felt I needed a basic understanding of to get what they were trying to say. I suspect it is not a book you can just dip into.

I thought it would be useful to undertake the task set at the end of chapter six which consists of answering three questions about a course you are teaching. The first question is:

A. What are the three things that you expect your students to be able to do at the end of the course?

This is not as easy as it may sound, at least it was not for me. The unit I teach does of course already have Intended Learning Outcomes (ILO’s) but I will leave these aside for the moment. I have been teaching this unit for ten years and have been thinking for a while that they might need updating so it seemed better to start from scratch.

B&T state that you have to be aware of the kind of knowledge you want students to acquire. Following Leinhardt et al. (1995) they differentiate between ‘university’ knowledge which is ‘abstract, declarative’ and ‘professional’ knowledge which is ‘functioning’ (2011: 97). Your ILO’s should clarify what kind is intended.

Getting back to the question, so what do I want the students to be able to do? Having tried in vain to write this down in three succinct sentences I am going to ‘talk’ around it first in the hope this will help coming up with short phrases later.

One: The outcome that is easiest to describe but – in my view – is least important involves ‘declarative’ knowledge: the students should come out of this unit knowing more about fashion and society in England between the wars. Maybe this does not even have to be one of the three ‘things’? B&T mention that outcomes that are not necessarily ‘intended’ or anticipated should always be allowed for. Upon pondering, I decided knowledge of the period was actually an intended outcome but will have to be reformulated so that it is something students are able TO DO. This is what I find difficult. B&T are very keen on the verbs being used in ILO’s and in chapter seven (2011: 123-4) they show three different sets of verbs based on two taxonomies (don’t ask), relating to different levels of understanding, see the example relating to declarative knowledge below.  You will see how I got on with the verbs later.

Knowledge verbs

Two: My main objective is that each student goes to a physical archive in London, finds a source relevant to fashion between the wars and analyses it (is that an objective?). Some of the archives the students might have already visited in a group with a tutor, but I want them to experience all the stages of going yourself from finding out the opening times, making an appointment (if required), finding the archive (if they have not been before), ordering a source and evaluating how useful it is.  We also look at various online archives in more or less detail. By the end of the unit students should ‘be more aware’ of what is out there that can help with their research but ‘being aware’ is not ‘allowed’ to appear in an ILO according to B&T (2011: 119).

Three: We also discuss what you can actually find in these archives, e.g. amateur film, images, dress objects, and how to ‘read’ these different kinds of things. As you know, a big part of the unit is a ‘close reading’ of Vile Bodies, using a novel as another source of information about the period with the advantages and disadvantages a work of fiction presents.

What follows is another attempt to say the above in three sentences. Students will be able to:

  1. identify online and physical archives providing source material relevant for the history of dress between the wars and beyond (I had use first but that does not appear in the lists of verbs – could also be evaluateappraise or select.)
  2. analyse source material including images, text and material culture relevant to the history of dress
  3. describe basic features of fashion in England between the wars and relate it to social context

Here are the current ILO’s:

ILO's as they are now

I think the second of the present ILO’s needs to be reformulated, the third might have to be split up and generally they should be in plainer English. I cannot quite decide which bits of which set I should use (apart from the fact that a change might not be possible, all of this is hypothetical).

Feel free to stop reading now, I just felt compelled to continue.

B&T’s second question is easier: B. How do you teach your students these things?

  1. Students visit one archive each and present their findings to the rest of the group. Their presentation should include practical information if appropriate, e.g. are the archivists helpful, can you take photos etc., as well as information about the particular source they examined.
  2. Close-reading of a novel; analysis (together or in small groups) of different kinds of materials including song lyrics, articles from fashion magazines, amateur film, images from a variety of contexts, dress objects.
  3. There are no lectures on interwar dress but as all the material discussed focuses on this period, students should learn about it by using the sources.

This brings me to C. How do you assess your students on doing these three things?

  1. Presentation: the advantage of this is that students hear about archives they have not visited themselves. Having said that, experience has shown that those students not presenting on a particular day often do not turn up (the presentations are towards the end of term when they have to finish a lot of essays). This year I also included a short session on how to do presentations so getting some training in presenting could be considered an ILO that is not expressively mentioned but also not entirely ‘un-intended’.  I have been wondering about another way of documenting the archive visit, e.g. by asking students to produce a set of instructions. Let me know if you have any ideas. I would also prefer this part not to be marked but am wondering whether students might then just not do it. Maybe I should try peer-assessment but that is such hard work.
  2. The students write an essay picking up on a theme in Vile Bodies, e.g. women wearing trousers, fancy dress, make-up and explore this by using one primary source (could be a magazine from the period, film, a particular type of image). I mentioned in a previous post that the dual nature of the essay – and that they have to choose their own topic – sometimes confuses students. This term I gave over more time to explaining the essay and for questions which hopefully helped (I have not yet seen the essays).

What do you think of the alignment between A, B and C above?

The three parts are not totally unaligned but there is room for improvement, some of which I already mentioned above. The ILO’s need more work and I am not sure I picked the right three things (maybe there should be four). I would like to come up with a task that makes students use some of the online image archives I just briefly introduce so that they are actively explored. One idea I had was to give pairs of students ten minutes or so to find images of something particular online, e.g. what aviators wore, and then discuss what they found. It always defeats me that this  would ideally be done in a room where every computer can be linked to a screen without fiddling with cables. Maybe you have suggestions?

Lastly, B&T also talk about aligning you course ILO’s with the overall programme’s ILO’s and with those of the university as a whole. I shall leave that for another time …

Illustration
Anthropology Class at Reed College, Portland Oregon, 1947

References
‘Constructively aligned teaching and assessment’ and ‘Designing intended learning outcomes’, in John Biggs and Catherine Tang, Teaching for Quality Learning at University, McGraw Hill 2011 (fourth edition, first published 1999): 95-132

Gaea Leinhardt, Kathleen McCarthy Young and Jennifer Merriman, ‘Integrating professional knowledge: the theory of practice and the practice of theory’, Learning and Instruction 01/1995, 5(4): 401-8
Please note I have not read this, but B&T refer to it.

Skip to toolbar