Aligning Performance Objectives with Purpose: Developing Assessments, Pt 1

One of the common errors I catch SMEs who are asked to develop training and new IDs make is a lack of congruency between the end goal for the learning experience and the assessments they develop.

For example, they may pose quiz questions that ask learners to recall definitions of things in the content that are just "good to know" instead of "need to know" because those are the low hanging fruit, and they just have to meet a question bank quota 😬

So this blog post is going to be part 1 of a two-part series on assessment development. Here's a break down of what will be covered over the next two weeks:

Part 1: Aligning Performance Objectives with Purpose
Part 2: Providing Meaningful Feedback Opportunities

Performance-Based Objectives

Too many times, learning objectives focus on vague notions of "learning." For example, understand is probably one of the most overused verbs when it comes to learning objectives. These objectives don't actually define the need that the learning experience is trying to fill or help to provide parameters for assessment, feedback, or evaluation.

So what to do instead?

Do is the key word here. What do you want learners to do? What demonstrable action can they take to show they have achieved what they need to achieve in order to be successful? This is why I'm all about losing the term "learning objective" and shifting over to "performance objective," particularly in workplace learning contexts.

What you can do is so much more important than what you have stored somewhere in your long-term memory. And performance is tied to more than just memory. It considers things like confidence, timing, and audience response-- but that's a talk for another day.

For now, let's take a look at a scenario.

Ellen is a teacher transitioning into her first corporate role in instructional design. Her boss wants to support her in upskilling. It seems like a good idea to have Ellen learn some instructional design theory, so she asks Ellen to start by reviewing Richard Mayer's Multimedia Principles of Learning.

When Ellen asks if there are any goals for her to consider as she works through the material, her boss tells her, the objective is really just to "understand the 12 multimedia principles of learning."

Ellen goes about memorizing them. She's able to restate them in her own words and to recognize examples of each when her boss asks about what she's learned.

Awesome!

Two weeks later, Ellen's boss decides to put her learning to work, "Now that you've got these principles down, you're ready to create some storyboards for video." She assigns Ellen a simple explainer.

Ellen is surprised when her boss isn't happy with the end product. She used as many of the Principles as she could. Her boss said the video was boring and sometimes overly simple.

Where did they go wrong?

Ellen's boss didn't focus her performance objectives on what she wanted Ellen to actually do, nor did she focus the assessment on the objective she laid out. Ellen's boss asked Ellen to focus on theory instead of application, when the latter is what she really wanted.

And even if "understand" were an ok objective (which I'd say it's not), the objective was too narrow for the need. It might have been a learning objective, but it wasn't a terminal performance objective. It was a piece of the puzzle, or an enabling performance objective.

Essentially, Ellen was set up for failure.

What might have been a better objective for Ellen's professional development experience?

To figure this out, let's look at what still doesn't quite work. If you wanted to stick to the Principles, you might say something like:

Create a storyboard for an explainer video, blending visual storytelling with at least 6 / 12 multimedia learning principles.

This is better because it is demonstrable. Ellen could easily show where and how she was making use of 6 of Mayer's principles. She could also focus more on the ones that support storytelling, like the personalization principle and the voice principle.

Still, this objective is not a complete reflection of what Ellen's boss is hoping she takes away from the professional development experience. Something like this could have been more effective:

Create a storyboard for a 3 minute explainer video that 80% of learners choose to watch to completion.

Here, we see a focus on engagement in a simple, but measurable way. This objective would be a big red flag that multimedia learning principles were, perhaps, not a priority or that the learning experience was poorly framed, as it didn't really encourage any sort of focus on engagement. The Principles are focused on retention of learning and streamlining encoding to reduce cognitive load, not making "sticky" content.

Let's look at two more examples to see how drastically the experience can change with different objectives:

Create a storyboard for an explainer video that increases learner's self-reported confidence in their comprehension of complex concepts by 50% between pre- and post-video.

In this objective, we see that the purpose is really to help the company's end users feel confident through video lessons. Focusing on this terminal performance objective (instead of the vague enabling learning objective of "understand") would have better shaped Ellen's professional development experience. Ellen would have known what was critical versus what was "good to know." For instance, she might have used the Pre-Training Principle, Redundancy Principle, and Spatial Contiguity Principles to make concepts feel "easier" to learn.

Additionally, having this more defined, easy to measure objective would have better shaped the assessment Ellen's boss assigned, allowing Ellen to demonstrate skills her boss actually wanted her to learn.

Here's one last terminal performance objective that could have been used, but again, with a slightly different purpose:

Create a storyboard for an explainer video that increases learning retention after 24 hours by 20% from previous training.

In this case, the objective would have focused on the end user's ability to recall information rather than their feelings about their own learning. Again, this might have shaped what principles Ellen applied to her storyboard and how. Principles like segmenting and signaling might have been prioritized.

From here, additional layers of enabling performance objectives could be considered, ones that support the achievement of the terminal performance objective.

The Recap

If you've made it this far, you're awesome...

Let me reiterate the big takeaways for you. In writing objectives, consider these three things:

What's the purpose of the learning experience? What do you want learners to do with the learning-- usually this is some kind of skill competency or behavior change?

What can they do to demonstrate their level of success? What can they do to show you they have achieved the objective/behavior change? Hint: This is the key to assessment development.

How can you measure the degree to which they have been successful? Remember, an assessment isn't always a "final" demonstration. It can be a check in that offers an opportunity for feedback. In order to check, though, you have to know how you're measuring success. Additionally, in order to evaluate the success of the learning experience overall, you'll need metrics (e.g., the 50% boost in confidence in the 2nd performance objective example).