Researching the Effectiveness of Virtual Worlds

I attended a very good session at the Masie Conference where folks from HP, Intel, Accenture, to name just a few, discussed the issues around applying virtual worlds in the corporate context.

One of the big issues that seems to be coming out is that corporate decision makers want EVIDENCE that Immersive Learning Experiences are more efficient and effective than traditional learning approaches.

However, one of the issues we consistently run up against is that in order to measure “apples to apples” on the efficiency outcome variable (i.e. can you learn and retain as much in the virtual world as you can via instructor led training or web based training). The underlying issue here is that in limiting ourselves to comparing apples to apples when it comes to knowledge retention we limit the very thing that makes the virtual world so special: Serendipitous, peer-based, discovery based experential learning. Put more curtly, it is why when we take stakeholders into a virtual world they immediately ask “where are the classrooms, I don’t see anyone learning in here?” Rogers classic on the Diffusion of Innovations has taught us that Familiarity is a necessary rite of passage on the road to fundamental disruptive innovation, but I sure hope that the era of digital avatars in digital classrooms watching digital documents will be a very short lived meme.

This evening, after many hours of wrestling with this challenge, things became clear. We need to separate the WHAT (Declarative Knowledge) from the How (Procedural Knowledge). Most ILT and traditional learning is about topic not task. It is about what not how. The virtual world is all about HOW and learning WHILE doing. Most traditional pedagogical approaches are focused on learning BEFORE doing. Thus it is not appropriate to compare ILT to Virtual Worlds because they actually are based in fundamentally different instructional paradigms. ILT is about content while Immersive Learning Experiences are about context. ILT is about Topic while ILE is about Task while ILT is about What, ILE is about HOW. ILT is about Knowing while ILE is about Doing,

If history does not repeat, it certainly rhymes. This should sound familiar to many of us. (Hint: Think Blended Learning). A blend that brings the value of immersive learning to provide an experience where the “WHAT” can be synthesized, understood within the context of task and internalized in an authentic way.

The ILE adds to the ILT as the bridge that finally straddles the Knowlng-Doing Gap that Pfeiffer and Sutton highlited in their excellent book a few years ago.

Finally, as we work on the evaluation it would be remiss of me not to mention some of the very cool work that my friends and colleagues Mitzi Montoya (NCSU) and Anne Massey (Indiana) are doing to develop an instrument that captures Perceived Virtual Presence. I will be using their PVP instrumentation in my evaluations from here on out as those of us in the academic world try to create an overarching evaluation rubric to look at perception, efficiency and effectiveness of Virtual Worlds. You can read about their very cool and innovative research here.

Masie Wrap Up

Two great take aways from Elliott’s conversations with Barak Obama:
1) Nintendo spends more on research than the US does on Education….spend more money on Education

2) Get a CLO for the US Government

BTW: He also sent the same advice to McCain campaign.

On the TeleTrust Game from Champlain College:
Anne’s students came up with a game called Teletrust which is VERY cool and helps people wrestle through the challenges of Teleworking and shows how that impacts the Trust relationship at a distance. They put in 17 hours the first day, 17 the second and 20 the third and the final product was very impressive.

Also, the game is available to all thanks to the Masie foundation. So go out on the Masie site to check it out.

Charles Fadel from Cisco Debunking Read, Hear, See, Say Do Myth
Charles Fadel from Cisco had a great chat with Elliott on the need for true evidence based learning design. He went through the story of how Cisco went about debunking the pervasive myth out there about See, Hear, Do….etc.

They looked at 24 different research studies to come up with a new model Interactive Multi-Modal versus Multi-Modal for Basic Skills versus Higher Order Skills. Here is how things map out. Bottom Line is that you don’t get much bang for your buck on Basic Skills for Interactive Multi-Modal. But if you go to interactive for higher order skills you get the biggest effect.

Bottom line: we need to get more evidence to debunk existing and entrenched learning mythologies.

Charles’ purpose in life is to accelerate the rate of learning because it is the second derivative that changes society itself. Bravo Charles!

Steven MR Covey on The Speed of Trust
Speed of Trust is a universal idea whose time has come. Elevator talk: Three key ideas: Trust is economic not just social because it always effects speed and cost. Trust is the currency of our world today…it is the currency for collaboration. It is the right fuel for today’s times. Trust is a learnable competency. Trust is something you can learn and get better at and develop over time.

eBay as a trust network. The whole premise of eBay is based on Transparency and Trust. What is your manager’s trust rating. Transparency is vital for trust. People do not trust what they can’t see. Don’t fight the transparency trend that the web is diffusing, we need to seize it to increase relationships with stakeholders. Credibility is the foundation of trust. It is a bit more intellectual. Trust is more emotional than intellectual.

If trust is a competency do we hire for it. How do we get our hands around it. We need to get good at recognizing it (go for Character and Competence) and then measuring it and leveraging it. Today it is like we are doing basic math on trust but in 10 years we will be doing calculus.

Social Networks/Trust Networks do not correlate to Hierarchy. So what does this mean for the enterprise of the future? People tend to trust peers more than they do people in responsibility. If we had a TeleTrust account (like bars for cellphone) above people for how trustworthy there were. (We can do that in Virtual Worlds ; ) Augmented Reality.

Recommendations are effectively a transfer of trust. What we are looking for is a combination of credibility and behavior. Credibility flows out of character and competency. Seeking Mutual Benefit on Character Side. Competence is all about Skills. We are better on Competence than on Character these dates. Getting results with behaviors (talking straight, transparency, writing wrong etc..) while depleting trust is not the way to go.

Just like we need evidence around learning we also need evidence around the competency of trust. Trust is the rising tide that raises all boats. Most Mergers do not work not due to strategy, finance etc. First crack in merger is trust. Focus systematically on trust in M&As.

Companies are doing more externally with customers on measuring trust (22%) than they are internally suggesting that this is increasing in importance. Usually companies build a brand but don’t build trust accordingly. Both can bring huge benefits. So we will look at brand like trust.

Advice for CEOs: Begin by making an economic business case for trust. Make the creation of trust an explicit objective for your firm. Start a measurement process. We measure what we value. Get a way to see the trust needle move. Model it from the inside out. Model what trust looks like with the board, executive team. Like Warren Buffett. Trust from time. Always work inside out from own credibility and behavior. Not a nice soft social virtue.

Book: The World is Flat. Describes new reality. Vital for all of us to understand new world we are all operating in.

President: The importance of starting with trust. We find ourself as a nation and world in a visious cycle of distrust and suspision. We need extend trust in politics and organization. When you give trust others can receive it. If you take trust out of a society the country implodes. Leadership without mutual trust is a contradiction in terms (Ken Blanchard Quote).

Last word from Elliott: Think about what you can do to be your own Chief Learning Officer

Over and out from a very cool there days in Orlando.

Clue for DreamCorp ARG

There are some people who are looking for a final clue in the DreamCorp ARG. You may find some insight if you don an AVATAR and head to the following Second Life URL:

http://slurl.com/secondlife/maui%20Beach/22/233/33

When you Login…you should look around and don’t forget to get your dreamcatcher ; )
Best of Luck to you !

Kevin Kelly Q&A

On Collaboration Tools of the Future:
Tools we have right now are minimal. Not good in sense that they dont structure the information we are generating. Info needs structure to be readable and parseable. We’ll see tools come along where the structure of the information is parsed and shared.

Example: Parsing out arguments. If you use the word “Pacifica” for my hometown, the machine should know that…because we know that. Tools today are not aware of semantic structure of an argument. Hello Semantic Web ; ) Awareness of structure and interactions of conversations. Meaning of what we are doing is better captured and parsed.

On Global Mind of Technology Enveloping the Globe:
The notion of virtual awareness. Global cloak of information and computers becoming aware. Tool like Google earth can bring to us an awareness of our globe that we could not do before.
Stuart Brand, turn around the cameras in space and take a picture of the whole earth. What we really want is a real time atlas of the world. How many phones, how much fresh water? Any question at global level is “We don’t know” with exception of population and we are even off by 20% plus or minus on that. Helps us get a sense of who we are as a global species.

Is the Semantic Web Inevitable:
Content is machine readable and your profile travels with you and contextualizes the content to your needs. Definition: Web that is readable by the web itself. It can parse out person, place, contextual cues. Like 10 years ago…web itself is 5000 days old. Everything we have seen: Govt forms, phone numbers, ebay, amazon, google earth….all happened in 5000 days.

Who will do all this for free? That impossibility is here with us. One of the things the web has thought us is to believe in the impossible. History of the last 5000 days suggests that almost anything is possible.

Issue with Semantic Web is that content itself needs to be deconstructed into Tripple and then recombined in huge ways. That is the promise of the Semantic Web.

On how to encourage collaboration in the enterprise and encourage more failure
Hard to change enterprises. Changing established process is very hard. Starting anew may be the best way to go. Having an “existance proof” is a very good way to go. Wikipedia is a great “Existance Proof” as is Flicker, as is Linux. Nobody is as smart as everybody. Some are smarter than others. All amateur photographers together are better than a professional photographer.

You can have a group be dumb. No question about it. There is an art to bootstrapping the positive wisdom of crowds as opposed to the negative stupidity of the mob. Great excitement of this age is that we are in a discovery space…..look at auction spaces. We are in a time of great experimentation.

On the Future of Global Dialogue, Deliberation and Decisionmaking
We are far away from it right now. Democracy and voting is a type of conversation with comment every four years. Voting is the most elementary and not most important aspect of democracy which should be a large scale ongoing conversation.

On Sandboxing outside IT and as service to clients.
Convinced that next 10-20 years we will find whole new structures of collaboration. Experiments in org structure and incentives will proliferate. Lot of how we manage today was defined in a world with very little information. That has all changed now. Organization Lab like a media lab is a fantastic idea. First article he wrote was on how scientists got their information. They all worked by pre-prints before the article was published. Look at how people actually work and spend time is a great avenue of research. How we spend time with instruments and people. Add to it the design approach to life. Do something and iterate. Just go and do it.

On Global Trends for Collaboration on Democratization of Content:
Korea and Japan have a mobile thumb culture that is different that US. Inclination for people to operate from where they are (location). Sitting at a desk in the long term will seem charming and archaic. Quickly we will be released from keyboards. Interacting with whole body will not be far behind. You think differently when you are on your feet. Mobility is much more foundational in the asian culture. Experimentations with hoteling and project rooms etc..

On Early Adopters and Getting to Main Street.
How to distinguish what crosses the chasm. Is this going to take off? Same thing with movies/media product. What gets adopted is a combination of all kinds of things. Sometimes in details of execution. Wired magazine…half of the success was in title of the magazine. Think not Have and Have Not versus Have Now and Have Later. Early adopter overpay for everything. Technology is that which does not work yet. Early adopters subsidize have laters.

On Social Network Analysis
Closest tool that he uses is called popurl. That page aggregates all the aggregators. Digg, Boing Boing. Kinda like Alltop. One page about what the web is thinking about.

On whether or not Technology is Morally Neutral
Babies are good. Technology is good. It is inherently positive as a default. Babies can grow up to do bad things…and so can technology. Weapons are technologies that do bad things. What technology brings is possibilities. Technologies are around us to allow us as individuals with our mix of skills and talent to express ourself. Imagine a world where Bethoven was born before the piano, Hitchcock before film. There are people who are being born where we do not have the technology for their genius to emerge exist yet. Brings us choices so chance for our unique genius to emerge.

On William Gibson
Future is here, just not evenly distributed. Not sure on how we distribute the future faster. Future will erupt in odd places of the world where everything is connected. Otherness becomes the most important thing in a globally connected world. Diversity in the workplace is actually a very productive and economically viable value proposition in an economy of ideas.

On Banking Differently
Lot of legal issues associated with this. It will be a while before we see a revolution in global financial system. Nationalism will be around for quite a while.

On the Singularity is Near
Mythic Idea. By 2040 we make an artificial intelligence that is smarter than humans and it can make a superintelligent computer. When we reach period the world changes so much we can’t really figure out what is beyond it.

Ray Kurzweil want to live long enough to be at that period. Artificial Intelligence is inevitible. Still need a great big Hydron Collider and need to do experiments and living tissue and results. Going to take a very long time. Intelligence is multiple. Calculator is smarter than humans in arithmetic. Computers we make will be different than human intelligence. Google is pretty smart allready. Spells, Translates, but completely different than human intelligence. Therefore it is useful to us.

Idea of singularity: Rapture of the Nerds is Mythic. It will never go away. It is based on exponential curves. Therefore it is always be near but never arrive. Wonderful creative idea that will be in our culture for a long time.

On Blogging to Remember and Open Source our Identity.
Who we are and what matters to us is out there on blogs. Can we leverage that for learning. In general, vertical wikis can harness information. I concur with this conference. People meeting face to face is a tool too. Scenario workshops. Unconferences agenda is set by the people attending. Turn a conference inside out by having it made by folks attending: Barcamp, Foocamp.

Not just always online tools. Online and face to face at the same time. Guidelines for blogging a conference (like I am now; )

Embed and augment virtual world and physical world.

On Disintermediation and Governance
Where is governance going? Nations are too big for small problems and too small for big problems. Nations will release some of the things they do to other mediums. A book never wanted to be a phone directory. They all want to be novels. Internet was better suited to being a phone book. Nations will be relieved at not having to do certain things that global network can better handle. Some elements of sovereignty will move out.

On Science Fiction
Don’t read it as much as used to as a kid. Writing in Science Fiction is not good. Great ideas badly written. Deamon by a first time author. Game designer who sets off in an elaborate AI in the network that takes over the world. Turns out to be a good thing. Gets rid of all parasites like spam. Having a Mafia guy in your neighborhood. Neal Stephenson’s new book called Anathem (monks who build a clock of the long now). Tending for the long term responsibly and seriously. Bill Gibson Spook Country: Augmented Reality on glasses and see the material worlds differently (name, room annotated, lines and coordinates).

On Microchipping People
Not me! I hope I am dead before it happens. Not a tatoo or pierce guy. It will most likely happen. We could do bioidentity with Iris’

On Technology Waning Soon
Technology is anything invented after you were born (Alan Kay). Physical shape of cities will not change much. Material revolution has already happened. Most of revolution is in intangible world.

Blogging Kevin Kelly and Others at Masie

Scattershot notes from Second City
One of the techniques I use to have students experience dialogue is to have two students build a sentence one word at a time going back and forth. That way they need to pay attention to what the other person is saying and build upon it thus establishing what David Baum would call flow.

Second City communications just did a very cool skit called Dr. Know It All, three people lined up answer audience questions one word at a time. I will be sure to incorporate this into my repertoire.

Scattershot notes from Kevin Kelly.

“TV and Phone got married and had a love child called the internet and it is improving our choices and ability to communicate.”

If we can be whoever we want to be then who do we want to be?

Wikipedia was impossible in theory but possible in practice. We can do things with the collective that we could not imagine doing before. We know we can’t do everything that way but we don’t yet know what we can’t do. Can we build an airplane that way?

Historically, knowledge came from an authority. The way we learn things is changing. We have a process called the scientific method to help us make sense of things. I think the scientific method will change particularly on how we agree on what is true. Wikipedia says “consistent opinion is true.”

How to cut through collective ignorance: Wisdom of the Crowd versus the Stupidity of Mob is a delicate balance.

Doing less things as an individual through increasing tools of socialism, but it is not socialist because it is not about making everyone the same. Increasing power of group PLUS increased individual uniqueness and freedom. Cool thing is you can increase both.

Imagine companies with open salaries and open books. Expense Reports: You can do whatever you want but every employee has the ability to review your expenses.

Not about transparency from a governance POV, but this is transparency from a social point of view. So you get to see what is appropriate and it distributes training into the very fabric of the firm.

Science progresses us faster through tools rather than theory. Invention and dissemination of tools is what drives economy.

You clue takes Innocentive model to the masses and is run on Ning.

Kevin is optimistic that we can change on the environment. Response to ills and problems caused by technology should be better technology. We can make cars that get 1000 miles a gallon. So he is optimistic about being at piece in environment.

Designer have an approach to the world and to life that is different. “Build to Think…the first thing they really do is build a very quick prototype and iterate through the process over time. Try and do things as a way of thinking. That way do becomes think.”

Failure is the most important part of learning. What you want to have are lots of little failures to keep them constant and small rather than catastrophic at end. Bring them forward and manage them as you go along.

We tend to be too cerebral in design. We need to actually do it with our hands and our heart not just our head.

How would you redesign the learning function in organization? Learning and teaching are symmetrical. You become one or the other. Get people to become better at communicating what and how they have learned themselves is key. Make everyone a teacher and you have a learning organization.

Learning is like an addiction. He spends a lot of time thinking about his learning process. He tries to appraise the process and iterates on it.

Poptimistic: came from Hollywood. Happy cheerful free-color version of the world. People are basically good, color is free, lets make people happy.

What does technology want? Dawkins did this with the “Selfish Gene.” Helpful to ask technology what it wants. It wants increased diversity, general to specific (same arc as evolution), to become more efficient, increase in complexity. Similar trajectory to live. Technology is the seventh kingdom of live. Very similar in evolution. Technology extinction is never part of the equation.

Book: The Singularity is Near – He disagrees with it but that is why it is provocative.

Place on Earth to study more: China. Not just about cheap copying.

Election Process: Voting is a small part of democracy.

Next Generation Hope: That they will treat us well. Different kind of old. Baby boomers will create a new kind of old.

Movie to Impact World Theme: Optimism leads to better behavior. In the end you have to have hope that things will be better you behave differently.

Wired: Push…end of browser. Got that totally wrong.

His next book: How should I feel about technology? Is there some way to evaluate whether it is good for us or not.

Wow…final chart. The Global Economy is like a human being. Look at financial crisis and think of it as one missed financial heartbeat across the globe:

Doh…I got the number of employees wrong at Wikipedia

Yesterday I had a lot of fun at my breakout session. I got on a riff comparing Wikipedia to Britannica on a number of fronts. Three years ago when I was at one of IBM’s GIO 2.0 meetings I had the pleasure of meeting Jean Baptiste Souffron, who was the attorney at Wikipedia. At that time,he told me it was just “Him and Jimmy” as official employees.

So as I riffed yesterday I talked about how Wikipedia only had two employees which I assumed would be a lot less that Britannica. Later that day Elliott was interviewing Sue and she mentioned that the ranks of Wikipedia have swelled to a MASSIVE 22 full time employees. Still not bad considering that they run the fourth most used site on the web.

The larger point here is that social media principles are rapidly diffusing into the enterprise space and new models of organization where orchestration and coordination of talent and passion around endeavor is trumping command and control of employees around projects they are not energized about is starting to pervade.

None of us is as smart as all of us and we need to start thinking of human capital as a network of talented and capable people who can be tuned to an issue or opportunity at the moment of inspiration or need.

Over and out from Orlando as I ready to hear from one of my true heroes, Kevin Kelly.

Keynote at Masie Conference

This morning I had the pleasure of doing a keynote session with Elliott Masie. True to form we agreed to just riff…and that we did.. I think it went quite well and I had a lot of folks come up after the session to say they got something out of it.

Elliott is pulling out all the stops this year. There is an ARG (Alternate Reality Game) that David Metcalf and his crew are running called Dream Corp. Also, Ann DeMarle’s students at Champlain college are building a game in real time during the conference to deal with the issues of teleworking.

Finally, being the king of last minute, I just signed into the social networking site that the Masie folks have provided and found it to be very user friendly. Here is a screenshot of the social network view. I am the blue dot closest to the middle:

I am very much looking forward to soaking it all in and leaving with more questions than I came…that way I know I have learned something ; )

Over and out for now from Orlando.

Follow

Get every new post delivered to your Inbox.

Join 29 other followers