Category Archives: Instructional Design

Active Play

Leave a reply

Earlier I blogged about the 6 activity modes. Over the next few posts I am going to present some examples of games that highlight each of the activity modes. Today the focus is on Active Play. For those of you who don’t want to exercise the power of links or the scroll bar, here’s a review:

The gameplay mode most often thought of in connection with computer and video games is the active mode. In this mode, the player must respond quickly, using rapid-fire techniques, “twitch” speed, and combinations of keys or buttons to achieve the goals of the game. The game clock and/or threat of character “death” often provide structure and consequences. In games that shape play around action, the story lines frequently emphasize dichotomous conflict where the player embodies good against an evil opponent. Many of the most popular commercial games, including “shooters”, arcade-style games, and puzzles incorporate active modes into their game play.

The following games are quite different from one another in subject matter, game controls and genre, but they all incorporate active play.

Upgrade Complete I typically hate shooters, because I have no twitch speed, but this one has an interesting purchasing aspect. As you play you earn points that allow you to upgrade from a bare-bones Atari style game to a jazzy modern shooter with pretty graphics.

Word Bubbles This fast-paced word game requires you to form words with the same 3 beginning letters in a limited amount of time. I tend to find it addictive.

Typing of the Dead Awesome! A typing program that has makes your “ASDF” blast and explode zombies.

Tetris The classic combo of active and problem-solving play. You know you want to practice your spatial reasoning.

Introduction to the 6 Activity Modes

Leave a reply

A few years ago I was working on some research that required me to analyze the game playing preferences of different children. I started off trying to use the existing game taxonomies to categorize their preferences, but found that they weren’t descriptive enough. Two games might both be puzzles, but have vastly different feel of play. After meditating on it for a while, analyzing a bunch of games, and looking back at the preferences of the students. I came up with the following 6 activity modes to describe the play that takes place within games. Multiple activity modes can appear simultaneously or sequentially in a single game.

  • Active Play

The gameplay mode most often thought of in connection with computer and video games is the active mode. In this mode, the player must respond quickly, using rapid-fire techniques, “twitch” speed, and combinations of keys or buttons to achieve the goals of the game. The game clock and/or threat of character “death” often provide structure and consequences. In games that shape play around action, the story lines frequently emphasize dichotomous conflict where the player embodies good against an evil opponent. Many of the most popular commercial games, including “shooters”, arcade-style games, and puzzles incorporate active modes into their game play.

  • Creative Play

Creative play offers the opportunity to create elements during play.  Some games provide opportunities to develop characters’ skills or appearance or to build or modify aspects of the environment. SimCity and the Tycoon games allow players to determine the components, layout and palette of cities, golf courses, roller coasters, and theme parks, while other games allow selection of character powers and appearance. Other creative elements in games can include free-form drawing, or the use of stamps to design printable or email-able documents that the player can use during or outside of game play.

  • Explorative Play

Another activity mode widely experienced in games is explorative play, where physical space and travel is simulated through the layout of the game arena. By hiding certain areas from view, the player is allowed to discover new areas and challenges in turn. Explorative play can be easily modified by the addition of other activity modes. Many three-dimensional “shooters” combine active and explorative play, where players find their way through virtual buildings or cities while dodging bullets and shooting enemies. Slower-paced educational games often pair exploration of an area with problem-solving activities.

  • Problem-Solving Play

Problem-Solving play is commonly encouraged via inclusion of puzzles, both in educational games and in commercial titles such as Myst. Here, there are specific rules for the activity sequence and the solution to the challenges. Even if there are a number of challenges within a given game, they are generally well defined, and undertaken independently. The problems may be hierarchical, requiring one problem to be solved before moving on to another, or the problems may be parallel and unrelated. Problem-solving may be fast-paced and reliant upon hand-eye coordination, or it can take a slower form where logic prevails.

  • Social Play

Social play can take a number of different forms. It can be the interactions between players and game characters or between the players themselves. This mode often provides the opportunity to manipulate the behavior of game characters, providing a god-like level of power. Interacting in a multi-player mode allows interaction between players in whatever manner the game allows, be it fighting, cooperating or romancing. Another form of player interaction takes place when two individuals use the same computer and station to control characters or action, when they must cooperate to some degree. This activity provides players with support, camaraderie, and/or help.

  • Strategic Play

Strategic play emphasizes the manipulation of resources–military, financial, or “human”–over a longer term. In games that emphasize problem-solving, achieving pre-set goals determines progress, while games that encourage a strategic mode of play often enable players to select their own or the computer’s benchmarks. Games that can include strategic activities include The Sims, titles in the Tycoon series, Civilization, and the Age of Empires. Few educational titles have incorporated this play mode into their games.

Creating Losing Designs

Leave a reply
We spend a great deal of time planning for learner success. Do we ever create the ability for learners to lose? Should we?Our society puts a premium on “winning” — nowadays it seems like every team gets a trophy — not just the winner. Americans are culturally bound to the ideal of winning. However, in the real world people lose all of the time: athletes lose games and tournaments, politicians lose elections. In high stakes situations like these, losing has a real consequence. How does this affect learning?Low stakes losing occurs every day. When we lose, we tend to either give up, or continue to practice to overcome the failure. Any casual gamer is familiar with losing and attempting to overcome the loss by quickly replaying the game. Losing, in this context, can be a big motivator, driving the will to practice, which can lead to increased skill. Learning to lose effectively is actually a skill in itself. Resiliency in the face of losing and using strategies to improve is an important life lesson and professional skill. If losing always equates to complete failure, then learners stop striving, stop attempting creative solutions, and see themselves as incapable.In learning design, a primary goal should be to create an atmosphere where losing is acceptable and intrinsically motivates the learner to try again. The potential learning value from this type of experience is measured not by how often the player loses, but by how much they improve through repeated practice.

In your training design include opportunities for low stakes losing before learners are thrust into their high stakes, real-world situations — be it a qualifying exam or on the job performance.

Recommendations for a “losing” design:

  • Make content available on demand so that learners can review material as needed
    • A common mistake is being overly controlling with content. If the goal is for learners to gain knowledge and skills, shouldn’t they have free access to the tools that can make that happen?
  • Communicate how the learner is performing relative to passing scores or via peer-to-peer review as appropriate
    • By communicating how comparatively well they are doing, learners can better sense where they can improve. This also gets their competitive juices flowing.
  • Allow plenty of practice opportunities before the final assessment occurs
    • If the only time that learners can demonstrate knowledge is a final graded test, their opportunities to “lose” are reduced to one high stakes moment. Build in low stakes assessment opportunities that prepare learners for the one that counts.

This post originally appeared in July 2011 on The Total Learner Experience.

Microsoft’s Digital Literacy Curriculum Sucks. Surprised?

Leave a reply

I originally posted this in May of 2009. 

I work as a program director for a non-profit devoted to mentoring and tech literacy in low income 7-12 graders. As such, I am often looking for useful curriculum related to a wide variety of programs, tasks and skills. I am often in debate with myself and others if it is better to provide the students with practical skills related to secretarial-type work or to explore the outer boundaries with projects in graphics, animation and web design. Our approach is to focus on allowing students to explore and produce with the more creative software while providing auxiliary work with students who see the value of learning the more practical Office suite.

Not wishing to reinvent the wheel, I regularly look for clear, self-directed lessons suited for my students. Oh, and free or cheap, because we’re a non-profit. I’ve looked at Microsoft’s Digital Literacy Curriculum several times to see if we can use it to demonstrate proficiency. Now I can hear some of my techie friends slapping themselves in the head, about to eviscerate me because I even mention the Evil Empire. Let me introduce you to the real world outside of your basement and Silicon Valley. Most people out here in the light, don’t care or know about Open Source. They are just grateful that they can reliably turn on their computer and spit out some information. Their focus is on their work, not arguing about the benefits of Ubuntu versus Morphix.

Back to the Digital Literacy Curriculum, it sucks. No surprise there, it is a Microsoft product. Now I know I am late to the bandwagon, but I’ve still got to pile on. Everything about Microsoft sucks. I am a native proud Mac user, but practicality has required me to have a fairly good working knowledge of the Windows world. I work with Microsoft as a recipient of donated software and OSs as part of our non-profit work. Each month when I need to report our usage I curse in my head. It’s almost as though they take pride in having an interface that is clunky and unusable. For a company that’s flush with capital they sure are stingy with any ease of use or follow through to their systems.  The Digital Literacy Curriculum reflects this attitude. It’s almost as though Microsoft is full of idea-men with no one to come back to ensure quality control. hmmmm….

I’ve looked at it before at work, but I wanted to give it a more detailed fish eye if I was going to record my impressions. Here on my mac computer I cannot even log into their curriculum because I don’t have Internet Explorer. Really, I mean, really? Can Microsoft not see the writing on the wall. Nevermind that I’m on a Mac. What about all those people out there who are using Firefox or Chrome? (Too bad I can’t round out that list with Netscape… I like 3s.) Why is it that they are so dead set on protecting their product that they cannot break their walls of paranoia and protectionism to become more welcoming and user friendly?

24 hours later…I had to reinstall IE, or maybe I didn’t. It had been set to “work offline” which took a while to find. They have two tools pulldown menus that do not have the same functionality. It would be cooler to pretend like I don’t have “user error” issues.  Everyone has user error issues; those of us with healthy self-image can own it. Anyway, accessed the curriculum at long last.

I was disappointed. After all my sturm und drang getting it set up, it wasn’t horrible, it was just typically boring. A metallic voice read some prose to me; the sound screeched through my veins like fingernails on chalkboard. There was a lot of tasteful blue and gray on white. It was just dull. And basic. Now, I do have to consider the fact that this is designed for neophytes, people who still need basic computer skills. Just because people are ignorant shouldn’t mean that they should get bad training; that’s perhaps one reason that they are ignorant in the first place.

This material is clearly not current, and likely I shouldn’t even be wasting breath on it. The last update was October 2007 which is so last century in Internet terms. Except it’s not. I get emails regularly from Microsoft touting this curriculum. They just sent me another disk encouraging me to use it with my clients. More of the same didactic lecture and demonstration that is already failing learners all over.

–Dolly

Suck Quotient– 2

usability 1 the actual content was fine, getting to it was an exercise in perseverance and patience.
aesthetics 2 non offensive
innovation 1 no
accuracy 4 the content was correct, just dull
fun 0 none

**Note
I was going to look at version 2 to be all fair and balanced, but while it worked when I first accessed it, once I installed the add ons that Microsoft recommended, I ended up with a black screen, no content. You win, Microsoft!

Serving your Learner

Leave a reply

My natural inclination is to design for the imagination. I’ve long believed that if you give students the best tools to create with, that they then can transfer that knowledge to other applications. By the end of my time at the k-8 school, where I taught the same students for 3 consecutive years, I had 3rd grade students creating Flash animations and writing action-scripting. To me this was a major accomplishment, indicative of incredible potential for these students to become producers of multimedia, interactive projects. Every year, however, the complaint was that the students weren’t learning how to format papers in Word. The parents saw computer class as a support to the other academic classes, not as a curriculum in its own right.

Now that I am working with students who have been traditionally under-served, I wonder about my own obligations to provide opportunities for academic success. Should this curriculum emphasize more immediate academic skills, formatting papers, learning powerpoint? Or should I emphasize their personal expression and storytelling through exposure to professional graphics, audio and video software applications.

I already know where I stand on this. I am still in touch with the majority of my inspirational art teachers. I learned more about how to present myself professionally from them than any of my drudgerous academic classes. I don’t know if I can inspire disaffected students with any software app, but I feel that it’s more likely in Alice, than it is with Excel.

Moving to Online Universities

Leave a reply

Originally posted on The Total Learner Experience. 

A friend and colleague approached me because my alma mater is interested in pushing into the world of online courses. I began thinking about what I think is critical about the university and college experience. Reflecting on all the numerous reasons why people go to college, I developed this list:

  • It’s what’s done
  • to make more more money
  • to avoid adulthood, or the real world, or whatever you want to call that delaying tactic
  • to pursue their interests
  • to network
  • to be qualified for the position they desire
  • to remain competitive
  • for the prestige and social status
  • to learn
  • personal satisfaction

I think that these reasons can also be applied to graduate degrees as well. Of course all these factors and more influence which institution a student might choose as well. These variables include:

  • Location
  • Price
  • Flexibility of courses
  • Relative academic rigorousness, relative emphasis on socializing, demographics of student body– including race, age, social status, economic status
  • Trajectory of graduates

I think where you go to school matters, but not necessarily for the reasons people always give. More than anything the college becomes a kind of identifier of the sort of person they might be, just like another accessory. Regionally, I know what a choice to attend a particular college or campus might say. Nationally, we may tend to share those judgments. Reflect on what you might think about a individual you meet who graduated from Berkley or MIT or West Point. Even without knowing anything further about the individual you might make assumptions about their politics, interests, “style”, or major.

For all their supposed rigor, The US News and World Report and other magazines’ annual college rankings feel fairly subjective and ultimately ring a bit false. I’m not super excited to have a veterinarian who claims to have matriculated at Yale perform surgery on my pup. Ultimately, I don’t really care what college a person has gone to as long as they perform their role well.

Others may not share that view. Online colleges need to stake out their expertise and proclaim what their requirements for enrollment are. I’m not impressed by online colleges advertising on every random social networking. Online colleges need to identify not only how they serve their audience, but how their audience, their graduates, serve us.

But also, if we accept that there is some name-brand recognition influencing why people choose to go to an institution, then we must consider how opening up an entire new product line will affect an established brand.

How much of higher education is breathing the rarefied air in the libraries and lounges? Or attending tail-gating parties? Or finding your niche of disgruntled malcontents? Or having casual coffee with your favorite professor?

On the other hand, online learning broadens the pool of people able to matriculate. Allowing students to engage in asynchronous off-site learning can change the demographics of the target population in the areas of age, work-experience, economic level, purpose and expectations. Responsible college administrators should acknowledge that if they plan to expand their client base, they should include reasonable

Using Social Media in Sales Training

Leave a reply

More from the archives. This originally was posted in November 2009 on The Total Learner Experience. 

This post is a “conversation” held in Google Wave between Dolly Joseph and Brandon Carson (two of our bloggers). Until we can embed actual waves in this blog platform, we will copy-and-paste interesting back-and-forth dialog.

Bwc BRANDON: Typical sales training provides principles, criteria, and process for making the sale. In many instances a a video demonstrating the optimum application of those factors is what is used to try and achieve behavior change. These videos are primarily from the viewpoint of the expert, however. Why not try using social media applications to provide two more perspectives: that of the salesperson being trained, and that of the customer receiving the sales pitch?

Drj DOLLY: Why not use a game? I have some recollection of some game (perhaps from The Sims) as you did different actions you got rapid feedback– so as your avatar smiled or said the other character’s name you earned points, but as you gave negative feedback you would lose those points. How about if as a sales person you didn’t know what the other customer wanted exactly. It’s part of the salesperson/player to figure it out– what their trigger points are. In a sense it’s a sophisticated 20 questions.

Bwc BRANDON: Good point. I like the idea of making it more game-like.

Drj DOLLY: What would be motivating game rewards for sales people? Would they need to IRL rewards, or could they be contained with a larger game framework?

Bwc BRANDON: Sales people have a small window of patience for “games”. However, they do thrive on competition and “winning”. I think reward can be as simple as executive/leadership recognition all the way to financial compensation. A competitive game arcade was used at Intuit as a sales education tool, and the primary reward mechanism was simply the leaderboard and a “weekly top scorer” notification that was sent out to everyone in the sales organization.

Drj DOLLY: Maybe they could earn vacation time. That would be motivating for me. ;). I could see this being a great tool for new hires who don’t have a great deal of sales experience.

Bwc BRANDON: Exactly. The first video can serve as the model. Modeling behavior from the perspective of an expert is a great way to demonstrate to the learner the expected outcomes. New hires need scaffolding to help them practice and understand the complex skills needed for mastery.

Drj DOLLY: Well if we are trying to promote peer to peer learning, then the salespeople should upload their own suggestions or trials. Is there a way for them to do this?

Bwc BRANDON: Sure! Successful salespeople could record their own responses. Have the learner use a webcam or Flipcam to provide their response. Have them upload the response to a video community (YouTube, or an internal “video-on-demand” platform if you have it). Using social media utilities such as ratings and comments, the cohort of learners can view each other’s responses and provide feedback. The experts can also view and rate the practice videos and provide their input as well.

Drj DOLLY: Salespeople who upload videos can receive points. More experienced salespeople could mentor new hires and receive points. If it’s modeled like YouTube, then the more views or comments you get, the more points you get. Of course what we are describing is a closed circuit. You’re not getting any input from the actual customer base.

Bwc BRANDON: Another video can provide perspectives from the client receiving the sales pitch. You can consider using actual customers who have viewed the first and second videos, or you can assemble “actors” and use case studies from the field. I would think it’s key to use real-life experiences whenever possible. Authenticity is a key factor in this type of activity. This unlimited back-and-forth between learners, experts, and even customers provides practice opportunities not possible in a classroom.

Drj DOLLY: What if the best videos became part of the companies marketing plan? Spotlight on top performers. The game makes a genuine leap to real life.

Bwc BRANDON: Great idea! I like that.

10 Ways to Use Facebook for Training

Leave a reply

This post first appeared in December 2009 on The Total Learner Experience. Although dated, I think it still has value. 

Bwc Brandon: You know as I work to create social learning micro-sites on the internal corporate network, it sometimes hits me that in a lot of ways I’m just re-creating what Facebook has already hit a home run with. So, Dolly, the question for you today is: How can I leverage Facebook for training?

DrjDolly: What content is best suited for Facebook? Is Facebook for content or is it for social networking and engagement?

Bwc Well it seems that technical training, soft skills training… really any type of learning content would work. The power of the platform seems to be around the integration of peer-to-peer dialogue and sharing with the context of learning. Facebook users are collaborating by default because they are exchanging comments and creating dialogue interactively. What we need to do is make this engaging activity applicable to work/training.

Drj Shall we brainstorm? What are the features of Facebook are conducive to training?

Bwc OK, so let’s make it a contest. Let’s list 10 ways we can use Facebook for training. I mean we’re professionals right? With imaginations like ours, we should be able to easily list 10 ways! Here’s one way:

1. Discussion threads. People seem to love engaging in dialogue about relevant subjects. A simple discussion board functionality is key in attracting learners to a higher level of engagement.

DrjFacebook recently took out the updates about when people commented on others’ walls. I liked that and was sad to see it go. It allowed me to see what my friends were saying — even if it was directly relevant to me AND it often introduced me to new people that I liked. You never know what tidbit of seeming off-topic information will inspire a new idea.

Bwc  2. Class notes and links to supporting information. In academic settings I’ve seen many professors, trainers and facilitators add their syllabi, notes, and links to websites. I’m using Microsoft Sharepoint now to create a wiki where I will host pre-work content and activities for a workshop I am designing. A Facebook-like platform is great for this, although versioning control would be a nice-to-have.

Drj Sharepoint?

Bwc Sharepoint is Microsoft’s collaboration platform. A lot of people are using it for collaborative learning environments. Check out more about it here.

Drj 3. Course feedback. Students can provide feedback about their favorite or least favorite aspects of the training.

Bwc This is where a “two-way” information stream can provide excellent near real-time feedback. An instructional designer could easily move to a more “agile” design process by using Facebook’s social utilities to gain “instant” feedback on instructional content. I’ve been using similar methods for several years by building database systems. Facebook has this functionality built in!

Drj

4. Student Reflection. Students are required to post some “takeaway” from any training that they just completed in their status, which can help their community at large.

5. Events. If you can get out of the whole departmental competition thing, it could be really interesting to have online “events” that everyone was invited to facilitate cross-company brainstorming for new initiatives.

Bwc 6. Build your personal network/brand. It’s important for individuals to be able to broadcast their expertise and interests. Facebook-like platforms could become similar to an “auctioning” of skills. IBM does this now with their internal Blue Pages. Employees are encouraged to list their skills, expertise, and availability. People can search using keywords to find suitable candidates for their projects.

Drj I like this idea. We haven’t talked about LinkedIn, but it seems like this might be an internal version of that.

Bwc 7. Post-classroom events (extended learning). We used this recently on a leadership training program. Although the primary component of the training course was a 3-day classroom experience, we designed pre-work activities for the cohort to participate in online, and, to ensure retention and reflection we also designed post-classroom activities. The cohort remained engaged, and continued to work on the post-class activities. One benefit were the “breadcrumbs” left behind by the cohorts… subsequent cohorts could learn from them.

Drj  8. Games. We could do a post of different types of games. But clearly all sorts of games are popular on Facebook. I don’t play, but I know that Mafia Wars, Farmville and the like have some real appeal. I could see seeking out people who had certain skill sets to form your teams. You and I play Scrabble and its knockoffs. Facebook serves as a portal to every kind of game imaginable. I could also see some scavenger hunts for content and personnel.

Bwc How many games on Facebook are designed for learning though? Quite a few I think… it’s just not real obvious.

Drj I FIRMLY believe that every activity people participate in teaches or reinforces some learning or knowledge (even if only a physical response). Even if games are not designed with education/instruction/learning in mind, they still have features that can be exploited and minded to fold in formal/planned learning opportunities. I mean, somehow they have made the most inane, tedious tasks on Farmville completely compelling. Who knew? Which leads me to think about another way to use Facebook for training:

9. Quizzes. What kind of programmer are you? What does your desk say about you? What kind of cubicle mate are you? While these are humorous, I could see some actual useful ones, like providing Myers Briggs profiles and the like. Oh, and here’s the 10th (I beat you to it)!

10. Fandom. I was interested to see that the number one corporate identity on Facebook is Coke. Their presence was first created by a zealous fan. The page has become their corporate identity, and hugely popular. A lot of times employees can be the biggest fans of a company and hold the most knowledge. Creating a safe environment to tell the positives of the company can offset some of the griping that we all need to do sometimes.

Bwc Yes, but some companies are paranoid about their “intellectual property” being made public. It’s akin to the music companies obsessing over DRM — the one thing a company needs to understand about today’s information revolution: you can’t control everything … so focus on influence rather than control. Comcast learned this quickly, and adopted Twitter to broadcast network statuses to its customers. Now it uses Twitter for customer support.

Drj  I see we keep returning to the theme that companies need to have a more flexibility in this era of accessible info.

Bwc Well there are 10 ways… but there are many more. The big question for companies: are you going to build internal social platforms, or leverage the public ones? It seems like many companies will want to keep their social networks behind their firewalls. This may work in the short-run, but newer generations of workers (Millennials and the 2020 Gen) will want to be able to blend their public and work networks and profiles. That will make for an interesting dialog… maybe we can discuss that one soon?

What’s the Difference Between a Game and an Educational Game?

Leave a reply

Back in 2009, Google Wave was the next big thing.  Dolly and Brandon used it to discuss their opinions on the difference between a game and an educational game. The distinctions are key for instructional designers to consider when deciding to design a game for a learning intervention.

Drj DOLLY: Games vs. educational games. I know, one’s fun and the other one is not! Really, though, what are the differences? I think intentionality of design is a key point, because I think that all games — indeed all experiences — are educational. Bernie deKoven talks a lot about children’s games providing an opportunity to roleplay and work out developmental issues. I know I learned a lot about what a ruthless sort my brother is when he beat me repeatedly at every board game created. So, if we accept that every game has learning opportunities, how do we make sure to include the fun?

Bwc BRANDON: Both can be fun, right? The basic difference is an educational game is designed with specific learning objectives and should be able to assess whether learning occurs.

Drj

I think assessment is key. That’s the external component that is often artificial. So many times assessment of learning within games have two basic problems:

  1. It’s completely external and separate from the game play.
  2. It’s too “safe”– Alex Trebek doesn’t give you second chances in final jeopardy. It’s FINAL!

I was thinking about that whole shift in board games with the advent of Cranium. Their whole goal was to remove the idea that there was one winner who had all the fun, but that players of multiple intelligences could play and those people could have fun. Bonding, instead of board throwing!
Bwc Hmm… Good point — Cranium could be looked at as a “learning game” although it doesn’t have traditional learning objectives, right?

Drj Well, if you look at their objectives they are met. And their learning objectives fall nicely within affective measures and an arts curriculum.

Bwc For example?

Drj Well, the goals of the game include team cooperation and communication through graphic arts and music.

Bwc “Game” is such a broad term. A game must include some element of Play, I presume. I also assume a game designed with a learning outcome in mind has structured play (as opposed to unstructured play — like when a kid plays with blocks).

Drj Garvey said that a game is institutionalized play. Look at football. It’s a game, clearly. We wouldn’t say that it’s educational, but you can learn it (and those who learn it best find it financially rewarding). Also by using it as a metaphor you can teach life lessons and/or management strategies, or you can teach math from the stats of the game.

Bwc So this leads me to think about the “game perimeter”. Most games have boundaries, right? We step into the perimeter’s “magic circle” to play the game. In the corporate workplace, the “magic circle” is the game’s play boundary the instructional designer/game designer creates. The trick for instructional designers is to craft what learning occurs while the player is in that circle… so does that mean that the assessment needs to occur inside or outside the perimeter?

Drj I think the best assessments are seamless and inside that perimeter. Examples could be performance-, point- or victory-based.

Bwc So are we saying that the differences between a game and a successful educational game can be minimal if assessment can be more embedded and genuine?

Drj Sure.

Creative Effective Training Evaluation

Leave a reply

Drj Dolly: Brandon ranted about ineffective evaluations while I was holed up under the 2 feet of snow here. The snow is still here, but my Internet is finally back. His comments about evaluation made me wonder what I consider effective evaluation to look like. Cumulative evaluation should be as authentic as possible. If you want someone to be an effective salesperson, you’d better identify what competencies are required and then have them replicate and practice as best they can in the classroom environment.

Bwc Brandon: Good point. I concur. However, more and more organizations are hit with the high-cost of travel and the inefficiency of removing people away from their jobs for instructor-led training. We need to provide effective strategies for authentic evaluation in an online format.

Drj So I can think of two ways to recreate these situations in the digital world: virtual simulation and/or role-playing. Learners will have different profiles. Does someone who is technically minded need the same practice as someone who is naturally a people person? Both these individuals need to come out with the same skill sets at the end, but perhaps they need different practice. Formative self assessments combined with flexible course sequencing can allow individuals to focus on their areas of real need.

Bwc That just describes a sales training course I completed using Thiagi’s 4-Door Model. It allowed the learner to “self-adjust” the content based on their own self-leveling of knowledge. It does beg the question though: how do you “branch” an evaluation based on different learner profiles? Can you have an effective evaluation instrument via a dynamic system that presents a contextual series of assessment items based on the learner’s individual profiles?

Drj I think that you can have a self-assessment that gives the learner guidance in their own strengths and weaknesses.

Bwc OK, so you’re talking about self-assessment. What about an actual skills certification? One that can affect a learner’s job status, salary, or in the case of compliance or regulatory situations, a learner’s knowledge that could have life or death consequences? Can a dynamic evaluation instrument provide the appropriate assessment of knowledge?

Drj I don’t know what you mean by “dynamic evaluation instrument”.

 

Bwc “Dynamic pooling” is when the system displays content based on a learner’s input at the time of input.

Drj What you are talking about now is less about the assessment/evaluation used and more the consequences related to that assessment or evaluation. The SATs are pretty weighty and they’ve been using a dynamic response for years. A student gets progressively harder questions until incorrect responses are entered. Then a cycle commences where easier and harder questions are given to the student until the logarithm determines what the student’s level of mastery is. I’d say that stakes are high, but the College Board feels comfortable using this dynamic response.

Bwc Right. So I’m saying we need a similar system in corporate learning where appropriate assessment techniques seem to be a missing factor. In eLearning, there is a cycle of ineffective self-check systems instead of situational problem-based assessments. For example, a colleague was discussing with me the usage of “misconception” problems in assessment. They can present more authentic situations such as posing the problem in a scenario and requiring the learner to identify the parts of the scenario that are wrong or inaccurate. Even using open-ended question types where the system evaluates based on a keyword or series of keywords can be quite effective. How are we truly able to measure whether knowledge or skills are transferred if we’re not willing to properly evaluate?

Drj I like that because some people are just natural or trained to be good test takers. That kind of exercise actually accurately tests knowledge that they have, rather than their ability to suss out a poorly written stem or item. I know you are all about performance. Shouldn’t people’s training be based upon actual deficits in their performance and knowledge? And if they are high performers or can demonstrate mastery of course materials, shouldn’t they be rewarded by being allowed to choose their own training regimen, anyhow?

 

This post first appeared in January 2010 at The Total Learner Experience.