This is the second of three posts on “innovative skill-based complex formative assessments.” It is about individual savoir-vivre ePortfolios. 

FREN 101 is designed to have several different kinds of assignment and of assessment. Like any course, it’s a work in progress; like any language course, it’s informed by the history of the field, a field whose written history alone already goes back millennia. Like other French courses in our department, it’s aligned with the Common European Framework of Reference in Languages; on which, and on which change, see “Shiny new French courses” (September 2018). So it also has some kind of balance of reading comprehension, listening comprehension, spoken production (including interaction), and written production. I’ve also tried to maintain some balance and variety with formative continuing work structuring the course and a student’s learning (and thinking about learning) as a whole—the group project in the previous post, the savoir-vivre individual journal in this present post—as well as regular continuing practice:

Like other UBC courses at the 100- and 200-level, this course has to have a formal final examination. We also have summative tests: currently one at the end of each unit, except for the fourth and final one because a test on it would be too close to the final exam. We’re not supposed to have exams at the very end of term, in the winter that’s in the last two weeks, in the summer the last week; this is to prevent a course from having constant testing, and especially right before an exam; and to prevent exams being set (in class or booked separately outside class) at the end of term, so as not to have an exam in the exam period. This is against university regulations, yet every year colleagues still do it, and set their poor students umpteen “midterms” including booking them at times that clash with students’ other classes. Sure, it would be nice to get away for the whole exam period; but that’s only going to happen for a faculty member who sets all their exams up this way. It doesn’t help students because they’re all doing several courses, with assorted other faculty, and they’ll still have exams in that exam period. But I digress. (I get grumpy about rule-breaking like this that’s selfish and hurts students.)  That fourth unit’s topic, competences, vocabulary, grammar, etc. all feature in the final exam; as does, potentially, everything else in the course; being a cumulative exam as is usual in languages, and not just regurgitating acquired material but putting it into applied practice in new (but not completely alien) situations.

There are many possible kinds of exam, and other kinds not in current use in our institution or in our times, and other kinds are imaginable, and others have yet to be imagined. In the next and last post in this series, some of the questions I’m thinking about (to which I don’t have an answer) are: what is an exam? what is it supposed to do? how? why?

 

Here’s what assessment looks like overall, in the current incarnation of the course:

As see in the previous post, that final examination includes a final question which students can to some extent prepare beforehand, and on which they are the experts; thus ending the exam and their work for the whole course on, I hope, a happier note.

In previous FREN 101 and 102, that question was both on the projects and the savoir-vivre journal-cum-scrapbook collection; for summer courses, it will just be on the latter.

CONTINUOUS ASSESSMENT AS SELF-ASSESSMENT

I’ve used some kind of regular writing that includes asking questions and questioning oneself in several courses over the last ten years. Previous incarnations: a course blog, weekly writing, asking a simple question and a complex question and making a comment; more comments, replying to others, more interaction, etc. = more points; in marking, I assess both quantity (inc. regularity) and quality (inc. participation). In some cases, I asked students to put together their own portfolio of what they considered to be their best—most worthwhile and valuable—contributions to that online body of intellectual work that they were co-creating ove the course of the course. This sort of thing is an old idea, and in frequent use in university literature and philosophy courses. Examples over the last ten years, in one form or another:

• A guide to the practice and limits of online commentary
• RMST 221: Romance Studies: Literatures and Cultures of the Romance World I: Medieval to Early Modern
  (taught in English with original languages included for comparison; texts read and worked on in original or English depending on student programme and degree requirements)
  “ADVENTURES”
  “MISCHIEF”
  “INTRIGUE”
• MDVL 301A: Medieval European Literature of the 5th to the 14th centuries
  (taught in English with original languages included for comparison; texts read and worked on in original or English depending on student programme and degree requirements)
  “THE LIBERAL ARTS” @ UBC Blogs
  
• MDVL 302: Medieval European Literature of the 14th to the 16th centuries
  (taught in English with original languages included for comparison; texts read and worked on in original or English depending on student programme and degree requirements)
  “CRITICISM” @ UBC Blogs
  
• MDVL 310D: Topics in Medieval Studies
  (taught in English with original languages included for comparison; texts read and worked on in original or English depending on student programme and degree requirements)
  “MARVELS” @ UBC Blogs
• FREN 336: The Francophone World in Images
  (in French)
  “QU’EST-CE QUE LA BANDE DESSINÉE ?” @ UBC Blogs
• FREN 333: French civilization I: A historically-based approach to French civilization and culture from their origins (the dawn of human time and the domestication of fire) to the Third Republic (1875)
  (in French)
  FREN 333 @ UBC Blogs
  

(The first time I did this sort of thing was in 2004, using non-electronic technologies, as a graduate student TA preceptor in an undergraduate course on French literature, culture, society, and history. The first time I did this at UBC and online was FREN 420L (“Éducations sentimentales: Perceptions masculines du féminin dans la littérature française des 18e et 19e siècles”) in 2009-10, it was a disaster. That’s another story for another time. I’ve lived and learned.)

A variation on the two-tiers-of-question-and-a-comment assignment is useful for students to prepare for discussion sessions. These can be awkward (I remember them that way both as a a student and as a novice TA) and terrifying. They can be a fine test of a student’s extroversion, arrogance, and ability to talk off the top of their head. They can be great test of shyness, fear, panic, paralysis, mortal dread, anxiety, stress, PTSD. They can be a disastrous way for students to show what they know, share it with others, learn, and make knowledge. But that can change if students can prepare ahead of time. They could also do so responding to questions set by their instructor beforehand; and one can of course do both these things. I rather like having students set their own questions as soon as possible, though, and have a stage where they all read all of their questions together and then work on them, including working out what sorts of questions are being asked, and which questions they find most interesting and why. In my experience teaching, students tend to ask good questions, and often better ones than faculty do especially when faculty have expertise (and excess experience) in an area; best case in point, teaching the Roman de la Rose. I’ve also had students prepare potential exam questions drawn from the whole collection of the class’s questions. Rereading questions at the end of a course is reassuring: some difficult questions have now become easy, some that looked duller and less stimulating have also changed, and “is it interesting?” becomes as enticing as “can I answer it?”. Students then each vote on ten questions they would like to see on the exam, ten that they would not, and I cobble together a list that means that every student will have at least one question that they chose.

A student-generated and “student-centred” weekly discussion session is also “learning-centred” or “knowledge-centred”: where the centre of our combined attention is something bigger than us, knowledge itself, to which we are contributing together. Metaphorically, this is a network or ecosystem or constellation. A political and ethical move away from the neoliberal egotism of centring students as customers, and away from the ancien régime absolute monarchy (worse, sometimes not an aristocracy but autarchy) of centring teachers as experts. For we are not all enlightened benevolent despots or philosopher-kings, and The Republic isn’t a simple and simplistic non-fiction Project Management Handbook. Learning-centred learning is the radical revolutionary respectful cooperation of peers. Ideally, it supports and sustains equity and diversity, working together in shared wonder, in solidarity and mutual aid.

SAVOIR-VIVRE

In FREN 101 and 102, I’m using individual student journals for a similar purpose: building knowledge over a whole term, questioning and questioning oneself, thinking about learning. I wrote about this assignment back when I was first designing it: “Savoir-vivre plurilingual intercultural learning portfolios” (October 2018).

Here’s what it looked like in that first term this year, with a few tweaks over the term, as I updated instructions (informing students along the way) in response to student questions, problems, and ideas. I’ve never been a fan of totally rigid course structure and design (and suffered for it in evaluation from rigid students), flexibility isn’t necessarily a bad and disorganised thing, it can be a marker of adaptability (of a course and its instructor) to circumstances, life, and individual humans (students and faculty), and a different conception of structure and organisation: living, motile, organic, interactive, cooperative, feminist, humanist, anarchist.

This slideshow requires JavaScript.

As you can see, it was rather complicated. An additional complication, UBC has two different things that are both called “ePortfolio”: one is within Canvas, the other is a version of WordPress for students in the Faculty of Arts (open to all UBC students, but housed with us). Given a free choice, the majority of students chose the former. The latter is to my mind more interesting as a way for students to have permanent space online, which they learn to use over a whole degree, and use for any number of courses including their portfolios and masterworks; and for work, extracurricular activities, and anything else. Sure, one could build a personal site on WordPress that does much the same thing, and many students have been doing that for years. We’re a long way from early individual websites like this one’s ancestor, hand-crafted in HTML on a university Unix server back in 2003. One of the university’s para-academic wings has also been selling “learning portfolios” to graduate students for several years, as part of “professionalisation”; I continue to have serious reservations about “professionalism,” to which I should add, regarding online portfolios, a discomfort with spin, marketing, image curation, sales, and perceiving oneself as a product.

I mean, let’s be serious: students should be making things like ePortfolios anyway as they should be learning to read and write online, as part of becoming responsible adult citizens. The ability to design and build a site, and the ability to compose and communicate ideas in a number of forms and genres and shapes, including online: these are part of higher-level literacy skills. Advanced reading and writing. It’s not unreasonable for a student to practise writing over the course of an undergraduate university degree, to expand their repertory and horizons and range, whatever they might be thinking of doing later and wherever life takes them along the way. There’s a lot more to literary life and the life of the mind than a certain kind of 1500-2500-word written essay and a 3000-6000-word research or analysis paper. (Even if some of us still end up back there: “Make essays Montaignian again” and “Make the essay Montaignian again (2),” August 2017.)

Writing isn’t just a small set of Approved Academic Writing Types (and goddesses help us when you see the main Official Authoritative Holy Book thereof that so many students suffer), nor on paper, and it’s not just written. So, for example, in FREN 101 and 102 the assignments include videos. I’ve also had students in RMST and MDVL courses submit videos and other multimedia creations as final work in lieu of written papers, first for accessibility reasons and later as variations on the anti-essay and un-essay.

Videos also assess spoken production. We do not have the resources to do that in the traditional way— individual oral interviews / examinations, 5-10 minutes long, with a jury of two faculty each time—given the logistical constraints of 400-500 students (and that’s just 101 in one term), scheduling in the final examination period, and faculty contracts as governed by the Collective Agreement between our employers and the Faculty Association. When I’ve done this elsewhere, we all got a week’s paid overtime. Other colleagues did this in private language schools, in working conditions that would be unacceptable in this university. We’re thinking on it. Watch this space. In the meantime, we have other ways of assessing a student’s ability to speak French and to understand and respond to spoken questions: which is the point of the exercise.

Videos are handy as, compared to audio, you can see who’s speaking (and that it is that student and not someone else) and it’s potentially more transparent: if a student were to complain about your marking, a third party can watch and video and mark it afresh. Video also helps students who would be less able or comfortable with another way of assessing spoken production: presentations in front of the whole class. Though one variation on these, group skits, can be fun (given the right people and circumstances).

This slideshow requires JavaScript.

I made a few changes, after a first term’s FREN 101, a second term’s FREN 102, and using a slightly different kind of journal (as designed by those courses’ coordinator) for FREN 111 and 112, and listening to student comments and confusions. Sometimes a thing is complicated, and that reads as confusing; especially if you’re in your first year and your school un-complicated everything including things whose complication should have been kept, as a matter of respect and honesty. Sometimes it’s not, and there are better ways to organise and present it. Sometimes I pull out complications as joys and jewels in which to delight; too much close reading and deep creative critical reading? not always a good thing. And sometimes, of course, I add complications where there were few or none before. It’s a talent, like over-explaining jokes that weren’t very good the first time around. When a student says “it’s confusing” or “I’m confused,” I listen: they might be able to help me, and help other students, as the chances are that other students might be asking the same questions. (There’s a whole blog-post to be written about the sentence “I’m confused” and why I love hearing it; quick plot-spoiler, it translates to “my brain is doing things and I’m learning and also thinking about what’s going on in there at the same time.”) Learning from students is one of the most important parts of teaching and of the course-design part of teaching.

Here’s the current version, set up as a “module” on Canvas so that one can go from page to page, one of which is the assignment itself a.k.a. where a student submits their ePortfolio. The known practical problem with this is that it works beautifully on a computer or a tablet, but is an unmitigated disaster on the tablet or smartphone app version of Canvas:

This slideshow requires JavaScript.

The video had previously been placed next to the final exam; I was thinking of it as the oral production part of the final exam, in a final exam that was 25% listening comprehension, 25% reading comprehension, 25% written production, 25% spoken production; for 40% of the final grade, so 10% per part; so 10% for the final oral video, 30% for the traditional exam. That didn’t work out anyway as we were unable to include listening comprehension in the final exam, because the room in which we ended up (such things are decided by non-academic central administration) was inadequately equipped. That’ll be OK in the summer, as we have fewer sections so will be in a smaller room, and there are fewer classes in the summer so if we need to change room that’s easier (even though, as last term and the term before, when filling out a form months before I asked for a room that was adequately equipped). We’ll see what happens next year.

For this summer version of the course, I moved the video into the savoir-faire, savoir-être, & savoir-vivre module. The assignment itself hasn’t changed, just thinking of it as part of continuing formative self-evaluative meta-learning. Like all assignments that include personal information, students did not have to tell the truth, but had the choice of going partial or full autofiction and fiction, and of assuming the persona of a character from a work of fiction. (This is in part in accordance with student privacy of information law; it’s also a way to incorporate imaginative and creative work, even in an absolute beginners’ course.) Some additional changes thanks to FREN 102 student conversations: adding the option doing the video in pairs or small groups, and in costume.

ASSESSMENT

As I had not done this exercise before, and as this was in experimental mode part of which was figuring out what marking criteria were suitable from what students actually did and showed that they were capable of doing at this level: I marked it purely qualitatively. This is not unreasonable or absurd, as this is a course in the literate qualitative humanities after all, not in the social or applied sciences or their offshoots. Before setting up a rubric, I wanted to see what students would do, left to their own devices. Another approach would have been to give more guidance and direction: but that would be inappropriate for a journal, a journey of self-discovery, a personal adventure and “wayfinding.”

Réflection: I’m not changing this assignment and its assessment, simply done / not done. That makes the assignment a low-stakes one, and encourages students to do more if they wish to, out of interest, for their own benefit. I’m hoping this will actually actively be low-stakes and low-stress for students, and will be keeping an eye on it, for example in designing survey questions to ask students: we know that primary and secondary school assignments are often designed to be low-stakes, with a large number of them, every one of which counts even if it’s for 0.1% of a final grade; but this backfires by increasing student stress and performance anxiety, as everything counts.

Carnet de bord / journal: I’ve changed the assessment of this slightly, having seen what students did over the year. The current rubric to some extent doesn’t change much, but it helps some students to see the same information in a table and with a maximum number attached.

Revisions, designed in late fall and April, in current practice right now:

This slideshow requires JavaScript.

I’m a recent (and qualified) convert to rubrics / des grilles d’évaluation. They offer clarity in the point and aim of an exercise, and transparency in marking. They are also a move away from box-ticking and point-scoring, from modes of assessment labouring under the illusion that “quantitative” means “objective,” and that both of these are the highest (or indeed only) ways to have standards and to show value. Sometimes also miscalled “scientific,” this is at best pseudoscientific, in the Popperian closed-system sense, reinforced by the usage of jargon that excludes outsiders. At worst, it is scientism: a belief based in half-knowledge and ignorance.

In some kinds of academic work, where answers are either right or wrong, it is possible to have an A+. At a university level—“higher” and “advanced” education—this should rarely be the case. Beginners’ French in a university differs from beginners’ French in primary or lower-secondary school, in further and vocational training, and in professional credentialising qualifications. Most university-level work should go beyond the level of testing the acquisition of right/wrong kinds of knowledge and assessments should be designed to do so; to practise and develop learners’ skills, knowledge (and more and bigger kinds of knowledge, not just the kind that is simple / simplified / simplistically right/wrong), and ability to apply that knowledge: critically and creatively, imaginatively and intellectually. This is the crucial distinction between savoir and savoir-faire.

In designing marking rubrics, I’ve tried to include space for actual qualitative excellence, going above and beyond merely what is expected; to rethink the expected and satisfactory as being merely the starting-point for quality work. A base-line and expectation for good work that satisfies the required criteria is a B. Beyond that is an A (and an A+ is exceptional and brilliant, in a different league and dimension). Between them, a B+ is for very good work that already starts to move beyond the expected in some areas.

Objectives:

—for wellbeing and mental health:

—for students, more rapid feedback (ask me again this time next year if this worked out in the fall, when teaching three courses not just one), lower-stakes assignments, cumulative term-long work.

—for those teaching, faster marking but without resorting to multiple-choice and the mechanically quantitative, to an illusion (or delusion) of objectivity that is an abnegation of responsibility. I struggle with marking. I have existential crises with marking. It makes me anxious and depressed. Worrying about getting it right gets me into tangles and paralysed out of doing it. And then I’m late, so it has to be even better so as to compensate, and that makes me more worried and tangled up and frozen, and then more panicked. All of that then hits sleep and appetite, and mood and relationships, and makes me physically ill. Marking time is a dark time. An accumulation of marking is a slough of despond.

So far, more than half-way through this summer-session course, I am up to date with my marking and I’ve found it more manageable, in terms of time and in terms of my own mental and physical health, than last year. On the other hand, I’m only teaching one class and it’s smaller; as opposed to three big ones (100+ students total) in winter-session terms, and without the extra coordination and student advising work at those busier times of year. I’m also formally scheduling in time for marking next year.

I’m hoping that this is going to be a story with a happy ending for all of us, students and faculty and French, and a world of knowledge and lifelong learning. I;m hoping that this will be meaningful “engaged transformative collaborative learning-centred learning” for all concerned, in which each of these words says what it means and means what it says.

Albert Camus has some wise words on engagement, for example in two of his most famous public speeches in 1957:

“Quelles que soient nos infirmités personnelles, la noblesse de notre métier s’enracinera toujours dans deux engagements difficiles à maintenir : le refus de mentir sur ce que l’on sait et la résistance à l’oppression.”
—Nobel prize banquet speech, 10 December 1957 via https://www.nobelprize.org/prizes/literature/1957/camus/25232-albert-camus-banquet-speech-1957/ 

 “À partir du moment où l’abstention elle-même est considérée comme un choix, puni ou loué comme tel, l’artiste, qu’il le veuille ou non, est embarqué. Embarqué me paraît ici plus juste qu’engagé. Il ne s’agit pas en effet pour l’artiste d’un engagement volontaire, mais plutôt d’un service militaire obligatoire. Tout artiste aujourd’hui est embarqué dans la galère de son temps. Il doit s’y résigner, même s’il juge que cette galère sent le hareng, que les gardes-chiourme y sont vraiment trop nombreux et que, de surcroît, le cap est mal pris. Nous sommes en pleine mer. L’artiste, comme les autres, doit ramer à son tour, sans mourir, s’il le peut, c’est-à-dire en continuant de vivre et de créer.”
—“L’artiste et son temps,” discours du 14 décembre 1957, Uppsala via https://www.lyc-daudet-nimes.ac-montpellier.fr/sites/lyc-daudet-nimes/files/cpge/inscription_cpge/kh_bl_camus_discours_de_stockholm.pdf; see also p. 9 et seq on liberté, responsabilité, authenticité, intelligence, espoir, vie vs servitude

—for all concerned, inspired by Camus again: I hope that we can move thinking about assignments and assessment, and the point and purpose of education—a language, a university degree, and education as a whole—from short-term pseudo-learning (to score points and move on to the next level, like in games and airmiles plans) to deeper and longer-term learning, that builds over time. More appropriate to learning a language, which is progressive and cumulative, formative and transformative. More appropriate to university undergraduate learning, as a foundation for lifelong learning and/as intelligent meaningful beautiful engagé life.

NEXT POST: the next steps

FOR MORE:

I’m looking forward to learning more from other people: from colleagues in my department and in two allied departments. You can RSVP at this link to join in too, this coming Wednesday!