Monthly Archives: February 2012

Creating Losing Designs

Leave a reply
We spend a great deal of time planning for learner success. Do we ever create the ability for learners to lose? Should we?Our society puts a premium on “winning” — nowadays it seems like every team gets a trophy — not just the winner. Americans are culturally bound to the ideal of winning. However, in the real world people lose all of the time: athletes lose games and tournaments, politicians lose elections. In high stakes situations like these, losing has a real consequence. How does this affect learning?Low stakes losing occurs every day. When we lose, we tend to either give up, or continue to practice to overcome the failure. Any casual gamer is familiar with losing and attempting to overcome the loss by quickly replaying the game. Losing, in this context, can be a big motivator, driving the will to practice, which can lead to increased skill. Learning to lose effectively is actually a skill in itself. Resiliency in the face of losing and using strategies to improve is an important life lesson and professional skill. If losing always equates to complete failure, then learners stop striving, stop attempting creative solutions, and see themselves as incapable.In learning design, a primary goal should be to create an atmosphere where losing is acceptable and intrinsically motivates the learner to try again. The potential learning value from this type of experience is measured not by how often the player loses, but by how much they improve through repeated practice.

In your training design include opportunities for low stakes losing before learners are thrust into their high stakes, real-world situations — be it a qualifying exam or on the job performance.

Recommendations for a “losing” design:

  • Make content available on demand so that learners can review material as needed
    • A common mistake is being overly controlling with content. If the goal is for learners to gain knowledge and skills, shouldn’t they have free access to the tools that can make that happen?
  • Communicate how the learner is performing relative to passing scores or via peer-to-peer review as appropriate
    • By communicating how comparatively well they are doing, learners can better sense where they can improve. This also gets their competitive juices flowing.
  • Allow plenty of practice opportunities before the final assessment occurs
    • If the only time that learners can demonstrate knowledge is a final graded test, their opportunities to “lose” are reduced to one high stakes moment. Build in low stakes assessment opportunities that prepare learners for the one that counts.

This post originally appeared in July 2011 on The Total Learner Experience.

Don’t be Half-Assessed

Leave a reply

Assessment is used to measure learning outcomes, but if you see it only as a testing tool you’re missing half its value. Assessment should not be punitive; it should uncover the learner’s strengths and help identify areas for improvement. Well planned and integrated assessment creates opportunities for learners to reflect on their learning, apply new skills and knowledge, and also enables the institution to recognize the value on the learning intervention. Adapting existing models can take you much further than re-creating the wheel. In this post, we examine some guiding principles for effective assessment.

An excellent example of assessment as a way to support learning goals is Adobe’s Certified Associate practice tests. Learners can take a certification exam without any structured practice, but there is also an unlimited certification prep test package. The practice test closely mimics the structure of the final exam. Both combine multiple choice questions and simulations of the Adobe environment. The multiple choice questions require the learner to demonstrate knowledge of design concepts and the production process. The simulations require the learner to complete a task within the simulated software application interface. Keyboard shortcuts are disabled, but otherwise, learners can use any correct method they choose to complete the task, using menus or panels as appropriate.

Another good example is the Sun (now Oracle) Java certification paths. Each path contains a test prep “kit” that includes preparation recommendations, additional resources, a practice test, and a re-take policy. Each path is designed to prepare the learner to achieve a specific level of certification, and used as a benchmark against industry standards. These certifications are recognized by employers and can advance a person in their career.

Accreditation or certification can be used to validate mastery of a topic, but this is not the only way that assessment can be useful. By seeing assessment an integral part of instruction, we can support the learner’s career development while measuring true performance for the business.

Factors that make an ASSESSMENT tool also a useful INSTRUCTIONAL tool are:

  • Authenticity
    • The learner performs the task in context, not recalling theory, but actually demonstrating competency.
  • Open ended
    • Because the end product is assessed, not the method used to get there, learners are able to use whatever menus or panels they choose.
  • Learning while doing
    • Learners use contextual clues and critical thinking to complete tasks. They may not know how to adjust alpha levels in Photoshop, but they may know to investigate the color panel to find them.
  • Self-reporting
    • Learners can mark questions that they’d like to return to if they have time or opportunity.
  • Cumulative time
    • The test is timed with one master clock, not with individual times for certain sections or items.
  • Feedback
    • Learners receive feedback on each item, with notes about the correct answer.
  • Tracking
    • Performance from one practice test to another is tracked.

Assessment is not something that should only occur in a testing situation, indicating pass/fail rates, but it should be integrated throughout instruction to allow the learner to know how they are doing, so they can learn more effectively. You don’t want your learner to leave half-assessed!

A version of this post originally appeared in The Total Learner Experience in August 2011.

Microsoft’s Digital Literacy Curriculum Sucks. Surprised?

Leave a reply

I originally posted this in May of 2009. 

I work as a program director for a non-profit devoted to mentoring and tech literacy in low income 7-12 graders. As such, I am often looking for useful curriculum related to a wide variety of programs, tasks and skills. I am often in debate with myself and others if it is better to provide the students with practical skills related to secretarial-type work or to explore the outer boundaries with projects in graphics, animation and web design. Our approach is to focus on allowing students to explore and produce with the more creative software while providing auxiliary work with students who see the value of learning the more practical Office suite.

Not wishing to reinvent the wheel, I regularly look for clear, self-directed lessons suited for my students. Oh, and free or cheap, because we’re a non-profit. I’ve looked at Microsoft’s Digital Literacy Curriculum several times to see if we can use it to demonstrate proficiency. Now I can hear some of my techie friends slapping themselves in the head, about to eviscerate me because I even mention the Evil Empire. Let me introduce you to the real world outside of your basement and Silicon Valley. Most people out here in the light, don’t care or know about Open Source. They are just grateful that they can reliably turn on their computer and spit out some information. Their focus is on their work, not arguing about the benefits of Ubuntu versus Morphix.

Back to the Digital Literacy Curriculum, it sucks. No surprise there, it is a Microsoft product. Now I know I am late to the bandwagon, but I’ve still got to pile on. Everything about Microsoft sucks. I am a native proud Mac user, but practicality has required me to have a fairly good working knowledge of the Windows world. I work with Microsoft as a recipient of donated software and OSs as part of our non-profit work. Each month when I need to report our usage I curse in my head. It’s almost as though they take pride in having an interface that is clunky and unusable. For a company that’s flush with capital they sure are stingy with any ease of use or follow through to their systems.  The Digital Literacy Curriculum reflects this attitude. It’s almost as though Microsoft is full of idea-men with no one to come back to ensure quality control. hmmmm….

I’ve looked at it before at work, but I wanted to give it a more detailed fish eye if I was going to record my impressions. Here on my mac computer I cannot even log into their curriculum because I don’t have Internet Explorer. Really, I mean, really? Can Microsoft not see the writing on the wall. Nevermind that I’m on a Mac. What about all those people out there who are using Firefox or Chrome? (Too bad I can’t round out that list with Netscape… I like 3s.) Why is it that they are so dead set on protecting their product that they cannot break their walls of paranoia and protectionism to become more welcoming and user friendly?

24 hours later…I had to reinstall IE, or maybe I didn’t. It had been set to “work offline” which took a while to find. They have two tools pulldown menus that do not have the same functionality. It would be cooler to pretend like I don’t have “user error” issues.  Everyone has user error issues; those of us with healthy self-image can own it. Anyway, accessed the curriculum at long last.

I was disappointed. After all my sturm und drang getting it set up, it wasn’t horrible, it was just typically boring. A metallic voice read some prose to me; the sound screeched through my veins like fingernails on chalkboard. There was a lot of tasteful blue and gray on white. It was just dull. And basic. Now, I do have to consider the fact that this is designed for neophytes, people who still need basic computer skills. Just because people are ignorant shouldn’t mean that they should get bad training; that’s perhaps one reason that they are ignorant in the first place.

This material is clearly not current, and likely I shouldn’t even be wasting breath on it. The last update was October 2007 which is so last century in Internet terms. Except it’s not. I get emails regularly from Microsoft touting this curriculum. They just sent me another disk encouraging me to use it with my clients. More of the same didactic lecture and demonstration that is already failing learners all over.

–Dolly

Suck Quotient– 2

usability 1 the actual content was fine, getting to it was an exercise in perseverance and patience.
aesthetics 2 non offensive
innovation 1 no
accuracy 4 the content was correct, just dull
fun 0 none

**Note
I was going to look at version 2 to be all fair and balanced, but while it worked when I first accessed it, once I installed the add ons that Microsoft recommended, I ended up with a black screen, no content. You win, Microsoft!

Word Vine

Leave a reply

While there are plenty of games that get it wrong, I thought I might review a couple of games that get it right. Word Vine is one such game. It’s premise is simple: link words to create inter-connected compound words on branching relationship. An easy level uses tree, sauce, apple, trunk. I’m stuck on a hard level with time, tower, prime, bell, cow, cash, machine, gun, minister, ivory, tower, zone. So what do I like about this game this morning (and last night, for a really long time)?

  • Simple task– It is well within my skill set to make compound words and to click and drag small icons onto a hotspot.
  • “Educational”– I’m not shooting stuff. I wouldn’t be anyway, but I like figuring out this little verbal puzzle.
  • Visual– Solving the harder puzzles requires tapping into some region of the brain that handles visual relationships; it’s difficult to solve the harder puzzle (at least for me) without physically (virtually) dragging the word shapes onto some of the twining branches.
  • Aesthetics– It’s pretty. I like the soft greens.

I’m always trying to figure out how to combine content and gameplay. I can envision a social studies content game which links concepts, dates, and historical figures. How about a branching vine of German philosophers or early 20th Century artists?

 

Serving your Learner

Leave a reply

My natural inclination is to design for the imagination. I’ve long believed that if you give students the best tools to create with, that they then can transfer that knowledge to other applications. By the end of my time at the k-8 school, where I taught the same students for 3 consecutive years, I had 3rd grade students creating Flash animations and writing action-scripting. To me this was a major accomplishment, indicative of incredible potential for these students to become producers of multimedia, interactive projects. Every year, however, the complaint was that the students weren’t learning how to format papers in Word. The parents saw computer class as a support to the other academic classes, not as a curriculum in its own right.

Now that I am working with students who have been traditionally under-served, I wonder about my own obligations to provide opportunities for academic success. Should this curriculum emphasize more immediate academic skills, formatting papers, learning powerpoint? Or should I emphasize their personal expression and storytelling through exposure to professional graphics, audio and video software applications.

I already know where I stand on this. I am still in touch with the majority of my inspirational art teachers. I learned more about how to present myself professionally from them than any of my drudgerous academic classes. I don’t know if I can inspire disaffected students with any software app, but I feel that it’s more likely in Alice, than it is with Excel.

Prior Knowledge and Proper Leveling

Leave a reply

This post first appeared in June 2009.

Yesterday I spent a long time ensconced with Lynda.com working through ActionScripting 3.0 for Flash. Now, I’m not a super-sophisticated Flash user or ActionScripting writer, but I know my way around. If you don’t know Lynda’s training, they structure the courses through a series of 2-5 minute long screen capture videos with voice over. The courses do not offer clearly stated prerequisite knowledge and I wasted long minutes watching videos defining variables, functions and repetitive syntax. I work from the assumption that most people who are working with a particularly specific programming language through an expensive subscription tutorial service likely know what a variable is.

I wouldn’t even be complaining about the slow step by step nature of the lessons if it weren’t that when they got to the portion that I wanted to know, interactive buttons, it suddenly fast-forwarded adding multiple concepts all at once. After overexplaining basic syntax, it underexplained a multi-layer, multi-movieclip while purportedly teaching how to make a button work. Rather than demonstrating how to control action in a simple, reusable manner, they instead embedded it within a much more complex framework, which makes it difficult for this user to transfer it to her own purposes.

So after suffering through information that was overly basic, I was then overwhelmed by the complexity of the next lessons. I had a similar experience at a four day FileMaker Pro training session. The first day was review for me. The instructor went over database structure — topics like “one to many relationships”. I must mention that there were only 2 people in this training, both of whom had a fair bit database experience. Neither of us needed hours of discussion of fields, tables and relationships. Day 2 was moderately useful. The information covered was graspable. Day 3 and 4 however were pure torture. We breezed through layers of proprietary knowledge that was highly specific and highly detailed.

Throughout the four days I kept wondering, who is this training for? There was no database manager who could find all four days useful. I “earned” a certificate, but I didn’t feel like I was well-served by the 28 hours that I sat in that room.

The missing component in both these experiences was the acknowledgment that these skills would not come until we had time to practice them. There was no time for reflection or synthesis built in or encouraged by these training experiences. I understand the culture that creates training like this, but I think it’s wrong-headed to omit time for people to actually ingest what they are “learning”.

Some People Don’t Have Computers

Leave a reply

Originally posted in July 2009 on The Total Learner Experience. 

I am convinced that technology knowledge and fluency is integral to having the fullest range of options in the 21st century. I mean, who’s not?

My concern is how to facilitate the access that all of us techies take for granted. I don’t have any answers, but I do have plenty of questions:

  • Do I design instruction and programs so that my students express themselves and have fun and buy-in to this tech world? or
  • Do I design so that they can get entry level jobs as receptionists and admins?
  • Do I try to sell some Puritan work ethic model (high school, college, 40+ hours a week) that I really don’t believe in, to some people who aren’t likely to buy it?
  • What are the upper middle class kids who do have access to computers since birth doing with them? How will they leverage their innate computer fluency as they grow into adulthood and jobs? How do I facilitate my kids having those casual, but oh so very important, experiences?
  • How can I sell my path to impressionable youth as one to emulate when I owe more money in student loans than I make in a year, drive a used car and work ridiculously long hours?

Suggestions?

Moving to Online Universities

Leave a reply

Originally posted on The Total Learner Experience. 

A friend and colleague approached me because my alma mater is interested in pushing into the world of online courses. I began thinking about what I think is critical about the university and college experience. Reflecting on all the numerous reasons why people go to college, I developed this list:

  • It’s what’s done
  • to make more more money
  • to avoid adulthood, or the real world, or whatever you want to call that delaying tactic
  • to pursue their interests
  • to network
  • to be qualified for the position they desire
  • to remain competitive
  • for the prestige and social status
  • to learn
  • personal satisfaction

I think that these reasons can also be applied to graduate degrees as well. Of course all these factors and more influence which institution a student might choose as well. These variables include:

  • Location
  • Price
  • Flexibility of courses
  • Relative academic rigorousness, relative emphasis on socializing, demographics of student body– including race, age, social status, economic status
  • Trajectory of graduates

I think where you go to school matters, but not necessarily for the reasons people always give. More than anything the college becomes a kind of identifier of the sort of person they might be, just like another accessory. Regionally, I know what a choice to attend a particular college or campus might say. Nationally, we may tend to share those judgments. Reflect on what you might think about a individual you meet who graduated from Berkley or MIT or West Point. Even without knowing anything further about the individual you might make assumptions about their politics, interests, “style”, or major.

For all their supposed rigor, The US News and World Report and other magazines’ annual college rankings feel fairly subjective and ultimately ring a bit false. I’m not super excited to have a veterinarian who claims to have matriculated at Yale perform surgery on my pup. Ultimately, I don’t really care what college a person has gone to as long as they perform their role well.

Others may not share that view. Online colleges need to stake out their expertise and proclaim what their requirements for enrollment are. I’m not impressed by online colleges advertising on every random social networking. Online colleges need to identify not only how they serve their audience, but how their audience, their graduates, serve us.

But also, if we accept that there is some name-brand recognition influencing why people choose to go to an institution, then we must consider how opening up an entire new product line will affect an established brand.

How much of higher education is breathing the rarefied air in the libraries and lounges? Or attending tail-gating parties? Or finding your niche of disgruntled malcontents? Or having casual coffee with your favorite professor?

On the other hand, online learning broadens the pool of people able to matriculate. Allowing students to engage in asynchronous off-site learning can change the demographics of the target population in the areas of age, work-experience, economic level, purpose and expectations. Responsible college administrators should acknowledge that if they plan to expand their client base, they should include reasonable

Pixelvader

Leave a reply

As usual I am curious how to merge the worlds of voluntary online activity with educational outcomes. I just voluntarily spent hours playing Pixelvader for no good reason. When I get caught up in a game or other online activity like that I ask myself several questions:

  • What kept me engaged?
  • What did I learn and how will I use it?
  • And, of course, What am I going to take away for future design?

What kept me engaged?

This game isn’t rocket science (although it did have a rocket ship). You shoot stuff. Typically I am not a huge fan of shooting stuff. I have poor reaction time and seemingly no motor skills. This equals no “twitch” speed which is required by most shooting video games. However this game incorporated strategic purchasing of rocket ship accessories. Once you equipped your rocket with further weaponry, defensive gear or speed, you played a quick shooter with simple arrow key controls (no keyboard combos which I can’t memorize or implement). At the each of the 10 levels you got to buy stuff.  I love buying stuff. You could go down a level and generate quick cash to earn more gear. If after a certain point you realized that your gear strategy was flawed you could reset with no penalty and try another tactic by purchasing differently.

So for a player like meTM the engagement factors are:

  • Multiple modes of play– There’s some strategy and action.
  • Low risk– It’s easy to practice at the lower levels, The game never ends until you choose, If a strategy doesn’t work, it’s very easy to try another one.
  • Low learning curve– There’s not a big story, Gameplay was understandable and predictable.
  • High feeling of success– I beat a good numbers of levels, I understood what I needed to do in order to win when I did not, Consequences were predictable and avoidable.
  • Low brain power– I was tired and wanted to unwind, this game didn’t necessitate huge thinking, but just enough to make me feel smart during late night play.

What did I learn and how will I use it?

Within the game, I remembered that sometimes it’s better to be close to a large object to hit it quickly, but if it has a particular offense, it’s best to step back to have greater maneuverability. Somehow that seems important as a life lesson to me this afternoon. I also determined that personally, avoidance is a more native strategy than aggression. I wish that there was some grand content takeaway that i could reference, but sadly, I haven’t yet thought of a productivity correlation, but I am thinking on it and will update when I think of it.

What am I going to take away for future design?

Obviously I will consider all of the things that makes a player like meTM remain engaged. I also rememembered that not all players are like me. But I think my biggest takeaway is this: Any game that keeps a player playing is successful. We can argue points about genres, playability, modes, all those various game-geek issues players like to argue. But, despite the fact that this game is likely no more than a 3 on imagination, innovation and creativity, I kept playing.

keep playing

Using Social Media in Sales Training

Leave a reply

More from the archives. This originally was posted in November 2009 on The Total Learner Experience. 

This post is a “conversation” held in Google Wave between Dolly Joseph and Brandon Carson (two of our bloggers). Until we can embed actual waves in this blog platform, we will copy-and-paste interesting back-and-forth dialog.

Bwc BRANDON: Typical sales training provides principles, criteria, and process for making the sale. In many instances a a video demonstrating the optimum application of those factors is what is used to try and achieve behavior change. These videos are primarily from the viewpoint of the expert, however. Why not try using social media applications to provide two more perspectives: that of the salesperson being trained, and that of the customer receiving the sales pitch?

Drj DOLLY: Why not use a game? I have some recollection of some game (perhaps from The Sims) as you did different actions you got rapid feedback– so as your avatar smiled or said the other character’s name you earned points, but as you gave negative feedback you would lose those points. How about if as a sales person you didn’t know what the other customer wanted exactly. It’s part of the salesperson/player to figure it out– what their trigger points are. In a sense it’s a sophisticated 20 questions.

Bwc BRANDON: Good point. I like the idea of making it more game-like.

Drj DOLLY: What would be motivating game rewards for sales people? Would they need to IRL rewards, or could they be contained with a larger game framework?

Bwc BRANDON: Sales people have a small window of patience for “games”. However, they do thrive on competition and “winning”. I think reward can be as simple as executive/leadership recognition all the way to financial compensation. A competitive game arcade was used at Intuit as a sales education tool, and the primary reward mechanism was simply the leaderboard and a “weekly top scorer” notification that was sent out to everyone in the sales organization.

Drj DOLLY: Maybe they could earn vacation time. That would be motivating for me. ;). I could see this being a great tool for new hires who don’t have a great deal of sales experience.

Bwc BRANDON: Exactly. The first video can serve as the model. Modeling behavior from the perspective of an expert is a great way to demonstrate to the learner the expected outcomes. New hires need scaffolding to help them practice and understand the complex skills needed for mastery.

Drj DOLLY: Well if we are trying to promote peer to peer learning, then the salespeople should upload their own suggestions or trials. Is there a way for them to do this?

Bwc BRANDON: Sure! Successful salespeople could record their own responses. Have the learner use a webcam or Flipcam to provide their response. Have them upload the response to a video community (YouTube, or an internal “video-on-demand” platform if you have it). Using social media utilities such as ratings and comments, the cohort of learners can view each other’s responses and provide feedback. The experts can also view and rate the practice videos and provide their input as well.

Drj DOLLY: Salespeople who upload videos can receive points. More experienced salespeople could mentor new hires and receive points. If it’s modeled like YouTube, then the more views or comments you get, the more points you get. Of course what we are describing is a closed circuit. You’re not getting any input from the actual customer base.

Bwc BRANDON: Another video can provide perspectives from the client receiving the sales pitch. You can consider using actual customers who have viewed the first and second videos, or you can assemble “actors” and use case studies from the field. I would think it’s key to use real-life experiences whenever possible. Authenticity is a key factor in this type of activity. This unlimited back-and-forth between learners, experts, and even customers provides practice opportunities not possible in a classroom.

Drj DOLLY: What if the best videos became part of the companies marketing plan? Spotlight on top performers. The game makes a genuine leap to real life.

Bwc BRANDON: Great idea! I like that.