Increasingly, our everyday lives are influenced by computing algorithms that we cannot see or control.

This is the somewhat alarmist but nonetheless grounded in truth statement by Kevin Slavin in his recent TED talk (shown in the embedded player to the right). It’s not just financial markets, but movie scripts, book recommendations and advertising selections… the online and media world is increasing using software algorithms to tailor itself to what a mathematical equation thinks we want.

I find one of the most alarming examples is Facebook’s algorithm to determine what warrants “top news”. Effectively, Facebook is deciding for you which of your many friends’ updates is most important. And the implications of that are quite scary.. What if a friend thinks you are not listening because Facebook filtered out their update? Or what if you miss an opportunity for a future romantic involvement because Facebook hides a party update from what it thinks is someone you don’t care about?

Increasingly in the future we are going to have to think carefully about what decisions we allow software to make for us, and what things we should keep full control of ourselves.

Watch the TED video here or embedded above, or read the BBC News article for more information.

Related posts:

1. Status update anxiety
2. One step away from lost privacy?
3. Twitter and Facebook attacks highlight the need for a true social network

The sequel to Deus Ex, one of the top-ranked games of all time and a pioneer in the cyperpunk genre, is nearing release. The sequel paints a pretty bleak picture of human augmentation. But this live action trailer goes way beyond promoting a game; it’s nothing short of a short film on the consequences of human augmentation.

Watch the clip. Forget it’s a video game. How likely is this kind of thing in coming years?

Related posts:

1. Mozilla Seabird concept phone
2. What happens when the game is more engaging than real life?
3. An augmented reality geocaching game for children

You know that scene in Bladerunner where Harrison Ford uses a computer to zoom, refocus and travel in 3D space within a photograph? For years we’ve all thought that would be forever impossible, but new technology from Lytro suggests that this sort of thing may soon be possible.

Their forthcoming light field camera captures not just one perspective of a scene, but uses a lenticular array to capture the entire light field, meaning that the 3D space from which the light originated can be explored after the photo is taken – so you can change which part of the scene is in focus, generate 3D images or even peek “behind” foreground objects.

The Silicon Valley startup clearly faces technical and financial challenges to change their prototypes into an affordable consumer product – but the cat is out of the bag on the idea, and we can expect camera manufacturers to race to catch up and enter this brand new market. This is a disruptive technology with huge potential to change the way we think about photography. Soon we may have a completely new kind of camera, which can truly capture a moment in a way we never thought possible. Some are wondering if it will take the skill out of photography, while others are already speculating about what this might do to re-ignite 3D film-making.

Read more details at AllThingsDigital and try refocussing images for yourself in Lytro’s Picture Gallery.

Related posts:

1. Is photography a human right?
2. X-Prize Foundation to launch $100M in new areas including Human 2.0
3. What happens when the game is more engaging than real life?

While you’re doing that thing you’d rather keep secret, a bunch of kids nearby snap photos of each other on their iPhones. You’re in the background of one of their shots. When they upload it, Facebook scans the faces in the photo and automatically identifies you, based on photos you’re already tagged in. The photo is then published to your wall; it is broadcasted to your entire social network. Your “secret” is now plain to see. Your Mum sees you smoking, your girlfriend sees you outside the sex shop, your husband sees you dining with your ex.

If this sounds far-fetched, think again. The pieces are already in place. Import a photo into iPhoto and it will make a good guess about who’s in the photo. Facebook already broadcasts any photo in which you’re tagged to your News Feed. Facebook recently added automatic face detection, making it easier to tag you.

Face detection is just the first step towards face recognition, and it’s likely that Facebook will add that soon – they already have a vast corpus of facially-tagged images of their users. At first, they’ll only tag you in photos from your friends (which you’re likely to know about). The only thing that then prevents random strangers’ photos from reaching your wall is Mark Zuckerberg’s unpredictable whim.

We’ve given Facebook permission to broadcast photos to all of our friends, without asking us, at any time. We have no control over the process. Are you ready to gamble your personal privacy on the chance that no-one will photograph you, and that Facebook will put your best interests first?

Perhaps privacy was just a temporary aberration. The global village is getting smaller every day. In a world of Wikileaks and Google vans, maybe there are no secrets anymore.

——
Related reading: The science-fiction novel The Light of Other Days, by Stephen Baxter and Arthur C. Clarke, provides an excellent exploration of a future without privacy.

Image credit: iFovia on Flickr (Creative Commons)

Related posts:

1. Technology just slightly ahead of legislation
2. Facebook, this abusive relationship must end!
3. Australia to Google: we can block you, but you can’t watch us

1. Posthumanity and digital superpowers

We’ve been blogging about Human 2.0 for over two years now, but it was this April that we split off from Bitcurrent and launched this site. We kicked off with two launch posts… a high-level scene-setter called Welcome to posthumanity:

“We’re becoming a new species–one that can hack its own cognition and edit its own biology. We’re all getting an upgrade, like it or not. This is the most important subject of the century, but it’s still hiding in academia and science fiction. We hope to change that.”

…and a look at some of the tangible ways in which the Internet gives us superpowers:

We may not realize it, but the Internet has given us superhuman abilities. Technology lets us to do things that were impossible 30 years ago – from speaking foreign languages and armchair travel to global messaging and virtual worlds. Welcome, Human 2.0, these are your superpowers.

2. It’s all about the data

Now that we live a significant part of our lives online, we generate data that can be used to create visualizations and learn more about people, for good or for ill. We may reveal details about ourselves that we’d rather keep private. This year we’ve explored this issue from several angles:

Behavioural Analysis and the Age of Metrics explored how HCI researchers are using gameplay data to understand player behaviour and design better games, and examined the wider trend of analyzing human behaviour:

As we enter the age of metrics, analyzing player behaviour in games and sports is just the tip of the iceberg. Already you can track your computer usage, the language you use, and even your sex life! In the future, visualizations of our life data will empower us to make better decisions.

You can also learn a lot by analysing people’s social networks. Here’s what Twitter infers from your followers:

Analysis of social graphs will be part of an arms race similar to that seen in Search Engine Optimization, where unscrupulous marketers try to convince Google to list them in search results.

We covered Ben Shneiderman’s talk on the importance of Human Network Analysis as a skill for the 21st century:

Free network analysis tools are now being used by researchers to deliver valuable understanding and insights to practitioners in a variety of fields, from analysing Senate voting habits to healthcare and counter-terrorism research.

What do your words say about you examined how each of us is now empowered to do our own analysis of authors and public speakers using tag clouds:

We all produce more words digitally than ever before. With free tag cloud generators, anyone can analyze your language and learn a great deal about you in just a few seconds. The age of digital linguistics has begun.

In our first foray into the realms of flash fiction, we explored the impact of music sharing on witness protection in “I know what you liked last summer”:

“Listen, we’ve had six other killings this week. Same M.O. They were all women in their mid-thirties. They were all killed, execution-style, late at night, just as Janet was. And most importantly, all six of them liked the same music.”

In fact, computers might be used to uncover all sorts of patterns in people’s behaviour, some of which they might rather keep private. Periodicity tackled this delicate subject:

Sentiment analysis can decide whether a blogger is angry or content, happy or sad. Given data over time, it can likely recognize patterns of mood, even cycles… Such as those that occur every twenty-eight days.

3. Ownership and digital rights

New technologies call into question the ownership of data and change our rights as individuals. This encompasses a lot of different issues and we’ve only begun to scrape the surface, but here are some highlights:

We looked at the issues around citizens being arrested to photographing police in Is photography a human right? :

Prosecuting these photographers could enable law enforcement agencies to abuse their power. If citizens cannot scrutinize police and hold them accountable for their actions, what is to prevent them breaking the very laws they are meant to uphold? As Plato first asked in The Republic, “Who watches the watchers?”.

It’s surprising how many new forms of online communication are completely owned by corporations. Read the details in Who owns your voice online:

It’s only when we take a step back that the picture becomes evident, and it’s not a pretty one. Almost every form of digital communication is dominated by one company, and locked in to members of that service. We are in a poor state for a free, open exchange of ideas.

Continuing this theme, we looked at how Facebook manipulates its users to better serve its advertisers in Facebook, this abusive relationship must end:

Contrary to popular opinion, Facebook isn’t free – I’ve sold my soul, digitally. On Facebook, you’re the product. Companies pay for your eyes, and the more she knows about you, the more she can sell you for. And that’s about how much she cares about me.

In the field of education, tablets like the iPad could bring about a personal education revolution. But such raw accountability may cause a backlash from teaching unions, as we explored in our four part series, Tablets, unions and education:

When you learn from a tablet, it learns from you. As students work with digital course material, they leave a trail behind them. Teachers can tell whether a particular student isn’t spending enough time on their lessons, or is breezing through them. Maybe, by ushering in an era of cheap, tailored, analyzed learning, tablets can leave no child behind.

Other highlights

You may also be interested to read about:

• Can computers help us remember? (audio interview with Sunil Vemuri of reQall)
• Our guest post by Alexandra Bowyer, Is this the beginning of synthetic life?
• Lifelogging 101: How to record your life digitally.
• New devices that unchain your digital media.
• How distrust of users could be the slippery slope to a dull, safe Internet.
• 2009, the year of Twitter
  (including two of our most popular posts, Twitter’s not a site, it’s a protocol and A better design for Twitter retweets).
• How social networks can cause Status Update Anxiety.

We hope you’ve enjoyed our blog posts in 2010 and we look forward to digging into some more interesting angles on how technology is changing society next year.

To thank you for following us, we have a small gift for you: Human 2.0 is now available on the Chrome Web Store!

Don’t forget to leave us comments or contact @human20 on Twitter to let us know what you think, and what you want to hear more about! Happy New Year, Human 2.0!

Related posts:

1. Bitnorth 2010: The Human 2.0 Weekend
2. Facebook, this abusive relationship must end!
3. Human Network Analysis – a new skill for the 21st century?

These are just some of the questions discussed in this 30-minute audio interview with Sunil Vemuri. Sunil spent 2 years digitally recording his own life while he was a researcher at MIT, and went on to found reQall, a company whose product specializes in helping you remember what’s important as you go about your daily life, with a minimum of effort.

Click here to listen to the full MP3 interview. (Length 32:19, Size 31Mb).

Editor’s note: This interview was recorded back in March 2010, as the first episode in a regular Human 2.0 podcast series. Unfortunately, as Human 2.0 is made in our free time, we’ve had to put the podcast plans on hold for the time being. We’re publishing this as a one-off audio post, but watch this space as we may feature more audio content in the future!

Image (cc) by Rutger Middendorp on Flickr.

Related posts:

1. The e-memory revolution has begun
2. The Ethics of Designer Babies
3. Is Facebook changing our concept of friendship?

Natural language processing and semantic analysis allows us to extract sentiment from documents. Marketing organizations and community managers rely on tools from Scoutlabs, Radian6, and others that try to understand how online communities feel about their brands and products.

As we share more of our lives online, there’s more to analyze. Researchers from Northeastern University and Harvard University analyzed Twitter’s mood over the day. This kind of sentiment analysis can look at someone’s online messages and decide whether they’re angry or content, happy or sad. Given data over time, it can likely recognize patterns of mood, even cycles.

Such as those that occur every twenty-eight days.

(It’s at this point that my dinner companion launched a well-aimed palm at my somewhat scruffy chin.)

Sentiment analysis is a powerful tool, generally aimed at understanding a particular thing.

But we could point the same kinds of tools at people, in an attempt to understand when it’s a good time to sell them something, or ask a favor, or send them an unsolicited message.

The Decade of Sharing is upon us, and public disclosure means public analysis. Knowing how receptive a prospect might be to a particular sales pitch could be invaluable to marketers.

Moods may relate to natural cycles–variances in hormones, day length, weather, and so on. They can occur in both men and women (a point I tried valiantly to make in my defense, without much success.) And they can be triggered by a huge range of causes.

This could have some interesting consequences.

• Wily consumers might try to game the system, posting carefully worded messages in the hope of steering good deals their way.
• Companies might restrict what their purchasing officers can do online, fearing that they might lose out in negotiations.
• Marketing companies could start selling data on individual consumers’ mood patterns.

Because of cross-site cookies, online marketing already targets users, as my friend Julie Matlin recently found out. But in Julie’s case, while the offer of a pair of shoes followed her around the web, it was a dumb offer: it didn’t take into account how she was feeling. Julie’s an unusually transparent online persona, and there’s plenty a marketer could have used to craft a targeted proposition that would have made her likely to buy.

Websites use A/B and multivariate testing to decide which propositions work best. By adjusting a range of variables — color, price, wording — each time someone visits their site, they can optimize their online businesses and maximize their revenues. If marketers can correlate causes with shifts in mood among their target markets, they’ll have tremendous power over consumers. If they then apply A/B and multivariate testing  to those consumers, they’ll quickly learn what approaches work for what moods, taking direct marketing to an uncomfortably personal new level.

Let the slapping begin.

Related posts:

1. Human 2.0 Holiday Highlights
2. Why aren’t there any smiley hashtags?
3. Behavioral Analysis and the Age of Metrics

Understanding human behaviour is vital for good product design. But you can’t just ask people what they need, you have to observe them first-hand. iPods, eBay and TiVo exist because designers watched people, noticed a problem with current products, and designed a solution for a problem people didn’t even know they had.

At OXO Foods in the UK, researchers studied how people measure liquids while cooking, and noticed that most people need to bend down repeatedly to read the markings on the side of the container. None of them reported this as a problem when interviewed. So OXO designed a measuring jug(cup) which could be viewed from above (shown right). This is an example of the growing science of design ethnography – product design based on direct human observation.

How to measure human behaviour “in the wild”?

Observational studies are expensive to conduct, and sometimes distorted because you can’t always observe someone in their natural environment. Fortunately, computers now make it much easier to collect data from “real world” activities. Such data is invaluable – for product designers to better understand their users, and also for us to help us cultivate a deeper understanding of ourselves.

Consider a game designer who wants to improve the interface for her game. At HCI2010, I saw Dr Richard Lilley demonstrate the new i-View X technology from SensoMotoric Instruments, shown in the video below. This tracks the player’s gaze and records what part of the screen they are looking at.

As well as recording the game experience and user inputs, a webcam captures facial expressions. With this system, our designer can examine both qualitative and quantitive data to measure the impact of interface changes – such as repositioning an on-screen HUD – based on how much attention the user gives it and how it influences their play. The designer can develop the game systematically and improve it faster. By watching his own recordings, a player can learn and improve his skills quickly too – and this approach is already being applied in a variety of different sports. For both player and designer, the feedback cycle is shortened considerably.

Visualizing Gameplay Data for Deeper Understanding

At the same conference I met Alicia Dudek, an MSc Design Ethnographer from the University of Dundee, whose research explores ways to gain deeper understanding of how people play games. Your choices when playing a game are a combination of your personal circumstances and the way the game is designed, so both must be explored – but the first step is to identify behaviour patterns.

To do this, Alicia and her colleague Rachel Shadoan analysed 18 months of player histories from the web game Plant Wars. They plotted this data on a graph, with the vertical axis representing time of day, and the x axis representing the passage of time in weeks and months. Different activities in game are marked using dots of different colours and sizes at the appropriate time. Here is the visualization for one player: (Click the chart for a larger version.)

These graphs were used as the basis for player interviews, where key events were annotated with feedback from the player. Having these visual stimuli yielded more detail than relying on memory. Here we can see how the game rules, the player’s daily routine and even the device he used had measurable affects on his patterns of play. The study is detailed in full in this PDF.

From Gameplay to Real Life

Visualizing and explaining behaviour patterns will help designers create better games, but can they help players too? I asked Alicia where this research might lead in future:

“We create vast amounts of data about our behavior patterns, which over time form habits, altering us as people. We need to elevate our awareness of our intangible interactions. We cannot touch emails or feel battles in games. Our minds and ways of thinking are being restructured. Research like ours aims to make it easy to understand your behaviour without specialist training. One day you may be able to know not only what you’re getting good at in a games but also how it affects you.”

As we enter the age of metrics, games and sports are just the tip of the iceberg. Already you can track your computer usage, the language you use, or even your sex life! And the behavioural data we all generate daily is very valuable. Twitter and Google both collect and analyze your web surfing habits. And fraudsters want your behaviour data too. But a new generation of tools are emerging, with which we will explore and learn from our own behaviour data. In the future, our choices will be much better informed.

For example, with my Fitbit I can measure my calories burned and consumed and see it on a graph:

Right now, you need to add your own annotations. But it’s easy to imagine a tool that would let you annotate and interpret patterns like these. Your phone could even alert you when you’ve over-eaten and discourage you from making that restaurant booking.

As more data about our lives become available, we’ll gain new insights into our unconscious habits; computers will increase our awareness of ourselves. They’ll help us understand how music affects our mood, how our diet affects our work productivity, or how our sleeping habits affect our relationships. Life visualizations will empower us to make better decisions. The only caveat: As we use this new power to reduce many decisions to equations, we’ll need to be careful not to sideline those intangible, unmeasurable qualities like love, happiness and instinct.

Related posts:

1. Design Patterns for Social Experience
2. Human 2.0 Holiday Highlights
3. An augmented reality geocaching game for children

In this four-part series, we look at the impact of tablet computing on education: how tablets can save North American students, but how their ability to collect and analyze how students learn will make teaching more accountable — something that unions will oppose aggressively as they try to protect their members’ jobs.

This is a detailed write-up of the Short Bit I first presented at Bitnorth 2010, with lots of background and links to references I found while putting together that presentation. We decided to break it into several parts to make it easier to digest.

Tablet computing is the catalyst that can trigger a classroom revolution: the digital classroom, personalized learning, cheap access to content, and a transformation of how we learn. They promise a shift in education that puts the student, not the teacher, at the center of the learning experience. And tablets can capture and analyze everything about how someone learns.

In other words, tablets make teaching accountable, bringing to it the kind of clarity and can’t-argue-with-that science that has transformed online marketing. But as we saw yesterday, there are powerful forces terrified of what the harsh light of accountability will reveal, as we saw yesterday.

Tablets don’t just display, they collect

Tablets are the ultimate analytical tool. They collect copious amounts of data that can be analyzed, letting us crunch all aspects of a learning experience: What was read, touched, and heard; when and where that learning happened; what was read slowly and what was rushed through. Properly instrumented, a tablet is a window into how a student acquires knowledge. It’s the perfect sensor for educational analytics.

And analytics, as anyone who runs a website will tell you, mean accountability. As we saw yesterday, accountability is something that unions have resisted defiantly for decades.

Tablets, and the digital revolution they bring into the classroom, could radically change the way we learn, and with that, the fate of a society. But they’ll be fought every step of the way by teachers who fear for their jobs. If those teachers win, it’ll be another continent’s turn. So convinced of this was the producer of 2 Million Minutes that, when I saw him speak last year, he admitted to buying a condominium in Mumbai for his retirement because he expected India to have the best standard of living.

Worse, since countries like the U.S. and Canada are huge consumers of the world’s resources — and yet some of the strongest opponents to actually fixing crises that have resulted from overpopulation and mass consumption — it may not just be another continent’s turn. It may be another species’. In an era of globalization, overpopulation, and technology, an illiterate North American electorate is bad for the planet’s health. As Snarkmarket says, forget Skynet; the real apocalypse starts when all the fish die.

A smart population can fix these kinds of problems, particularly when armed with powerful tools like tablets and digital learning. Churchill once said that “America will always do the right thing, but only after exhausting all other options.” Unfortunately, teachers’ unions are stopping the country from doing the right thing.

Unions will come up with all sorts of ways to fight tablets and the digital classroom they usher in. They’ll argue that digital learning is impersonal, and that without supervision and encouragement, students will find ways to trick the system and avoid work. They’ll claim standardization and automation stifle creativity. They’ll argue that the web can’t teach, citing by Nick Carr’s cautionary tale of The Shallows to criticize digital learning. They’ll claim that tablets destroy jobs, just as ATMs were supposed to destroy retail banking and self-service pumps apparently threatened gas stations.

Many of these arguments are contradictory: teachers often teach from the same course material year after year, and are unwilling to update it. Motivated by personal convictions, or trying to hide gaps in their knowledge, they sometimes teach their own version of the truth.

But the real problem here is that educators need to play a new role — one for which they’re woefully unprepared.

The digital classroom is inevitable

The digitization of humanity has shaken the foundations of print, broadcast media, and music. Now it’s teaching’s turn. The democratization of technology puts the world’s knowledge on the lips of every know-it-all in a  classroom. Plagiarism is only a few keystrokes away for any lazy student. How can teachers possibly compete with the best content the world has to offer?

Faced with an onslaught of rich content and a classroom that has grown up connected, some teachers hide behind union rules, sticking to their outdated lesson plans and crying out for protection. But as every newspaper, record label, and broadcaster has learned, hiding from the future won’t stop it.

Just look at how technology has invaded the workplace. In the 1950s, Information Technology involved either counting things, or aiming things. Its primary uses were the census, artillery, and space missions. By the 1970s, the locus of innovation was in businesses and institutions, who used it in pursuit of profit, cost-cutting, and efficiency. By the turn of the new millennium, innovation happened at the edge, on the screens and hips of citizens. The web, mobility, and an unprecedented wave of creation has swept us forever into an ocean of information.

The Kno is a tablet textbook specifically for schools.

Classes are already including online content, but only in a fraction of schools. In a 2008 survey of U.S. school district administrators, researchers found that 75% of schools were offering online or blended courses, and that roughly 1,030,000 K-12 students were engaged in online courses in 2008, up 47% from a 2005 survey.

Sites like Watchknow are filled with student-ready, structured content for kids — in fact, Mashable published a list of online tools for educator just yesterday that underscores how quickly technology can insert itself into classrooms.

This will force teachers to change. No longer will they be the administrators of a curriculum, assigning and grading tests and babysitting classrooms. They’ll become content creators, building lessons and assembling information from many sources. They’ll be stewards of the learning process, using management tools to govern their students’ education. It’s a job for which they — and their unions — are ill-equipped. But it’s inevitable if tablet computing enters the classroom.

A bright light: the new role of educators

In this series of posts, we’ve picked on teachers’ unions a lot. But teachers aren’t unions, and many of them are well-intentioned. They genuinely want to change lives, and they’re frustrated by poor pay and awful classrooms. They’re going to have to learn new skills to manage the digital classroom. And they’re going to have to stand up to their unions, sacrificing protections for true change.

If we can get well-intentioned teachers the tools they need, without the interference of unions, their role will shift. Teachers will manage the “learning dashboard” — air traffic controllers for each student’s learning experience. They’ll define content and tailor it to current situations and each students’ abilities and learning styles. They’ll manage the exceptions, intervening when there are problems and encouraging when there’s excellence.

It takes technical aptitude to do this well. That means making teachers understand and embrace new technologies, rather than hiding from them. Lynn Steen, a professor of mathematics at St. Olaf College in Northfield, MN warns that “an innumerate citizen today is as vulnerable as the illiterate peasant of Gutenberg’s time.” She might as well be talking about a technically illiterate teacher in the era of digital classrooms.

Attempts to analyze students and identify problems have borne fruit. Many post-secondary institutions have Learning Management Systems (LMS) that handle the administration of the student body: class schedules, payment, enrollment, and so on. Increasingly, the LMS is the source of course content and test marks, which makes it a good test-tube for academic analytics.

By looking at which students have accessed LMS-based content, and comparing that to their marks, universities can see correlations. They can then alert teachers to students who are likely to have problems based on their interaction with course content. Imagine what this will be like when every textbook, every notepad, every musical instrument, every lecture includes analytical information.

Another system lets students bet on their grades. It’s a rigged bet — the student can get an A if they really try — but in this case, the end justifies the means. Analytics and measurement make this kind of game theory possible in the first place.

On O’Reilly Radar, Jason Grigsby talks about an iPod Touch educational program that gives students one-on-one interaction with touch devices. Coordinator Joseph Morelock has studiously documented the results of classes equipped with iPods, and the results have been dramatic enough to prompt parents to try and raise money for devices in every classroom.

Perhaps the brightest example of the new teaching model is the NYC School Of One initiative, which uses a “playlist” metaphor and different learning modalities to optimize the educational process.

In their model, each student can learn content in many ways — online tutor, team problem solving, individual study, and so on — and the system analyzes which methods work best for which content for each student. It knows, for example, that you learn Euclidean geometry best through watching a video, and calculus through team problem solving. Then it applies this to future lesson plans, analyzing performance by student, topic, and teaching approach.

It’s early days for the system, which is an after-school pilot project at the moment. But already, students are leapfrogging their traditionally-taught peers.

Some conclusions

As I dug into the subject of tablets and education, I learned more than I cared to about the politics and bureaucracy of learning. North America’s future lies in the hands and minds of its youth, and today, that future looks bleak. Soaring costs and an anti-science attitude won’t have an immediate impact on our lifestyles, but the long-term neglect of education will be hard to reverse.

I asked the folks at School of One what they’d found in their work:

“I’m preparing a presentation for an upcoming conference on tablet computing, education, and the potential tablets have for revolutionizing learning. Much of the power of tablets is personalizing the learning process, something the School of One seems to be all about. At the same time, I’ve read a lot about the resistance teachers’ unions have to accountability, including a recent NPR podcast in which one of the panelists said:

“Right here in New York City, Joel Klein indicated a while ago that he was going to use student test scores as one factor in evaluating teachers for tenure. What did the union do? Now, this is something that Obama supports, that Arne Duncan supports. It’s unbelievable. What the union did is they went to Albany and they got their friends in the legislature to pass a law making it illegal to use student test scores in evaluating teachers for tenure anywhere in the state of New York.”

I’m curious about the tension between these two trends: on the one hand, tablet computing and heavily analyzed student progress (like what you’re doing at the School of One) provides a lot of data for improvement; on the other hand, unions seem to resist that. So my questions are: what objections have unions voiced to the School of One program? Are teachers more or less busy? Will this create or reduce jobs?

They responded with a terse, and politically safe, note saying that unions support the initiative because teachers just have to teach:

“The unions have generally been supportive of our project. Because the computers help so much with organizing curriculum, assessment, and administrative tasks, teachers have a simpler job: just delivering their lessons.” (my emphasis)

In my opinion, this misses the point. Teaching will change. Author Daniel Pink describes one teacher who’s realized that classrooms should be for work, and alone time should be for lessons:

“Instead of lecturing about polynomials and exponents during class time – and then giving his young charges 30 problems to work on at home – Fisch has flipped the sequence. He’s recorded his lectures on video and uploaded them to YouTube for his 28 students to watch at home. Then, in class, he works with students as they solve problems and experiment with the concepts. Lectures at night, ‘homework’ during the day.”

This is far more indicative of what tomorrow’s classrooms should look like. It’s a complete reversal of today’s “show up and speak” mentality, and I suspect it will be far more difficult for the more tenured, set-in-their-ways teachers to embrace.

It’s important to remember, however, that technology in education isn’t a one-time budget item. It’s an ongoing, recurring, constant part of the curriculum, like chalk and recess. Teachers need tools and training to change their responsibilities, and someone needs to develop proper content aligned with the course plan. Simply giving tablets to students won’t change education; rather, developing two-way digital learning and putting tech-savvy teachers in control will.

In the words of Sam Seaborn, Rob Lowe’s character on the West Wing,

Mallory, education is the silver bullet. Education is everything. We don’t need little changes, we need gigantic, monumental changes. Schools should be palaces. The competition for the best teachers should be fierce. They should be making six-figure salaries. Schools should be incredibly expensive for government and absolutely free of charge to its citizens, just like national defense. That’s my position. I just haven’t figured out how to do it yet.

As Julie Johnson, a 3rd grade teacher participating in Morelock’s Canby School District program said, “It’s just so fun to listen to them answer their own questions without my help. I am now the last resort.”

Tablet computing, and the digital classroom of one, gives students access to petabytes of knowledge, tailored to their current situation, ability, and learning preferences. It’s how we can overcome many of the problems endemic in today’s schools. It’ll mean retooling and re-training teachers, equipping them for the student-centric classroom of tomorrow.

As long as the unions don’t get in the way.

Related posts:

1. Tablets, unions, and education – part three
2. Tablets, unions, and education – part two
3. Tablets, unions, and education – part one

In this four-part series, we look at the impact of tablet computing on education: how tablets can save North American students, but how their ability to collect and analyze how students learn will make teaching more accountable — something that unions will oppose aggressively as they try to protect their members’ jobs.

This is a detailed write-up of the Short Bit I first presented at Bitnorth 2010, with lots of background and links to references I found while putting together that presentation. We decided to break it into several parts to make it easier to digest.

At this point in my research, as I explained in yesterday’s post, I’d concluded that learning isn’t a priority in North America — politically, culturally, or economically. It seems to me that tablets — with their access to affordable, tailored education — offer a tantalizing cure to the ills of North American’s classrooms, and a path to the digital classroom that can help us catch up with the rest of the world.

When I started looking into the issue of education in North America, I assumed that military spending outstripped healthcare and education dramatically. That’s how it is in many regions. In San Francisco, for example, 21% of a family’s taxes in 2007 paid for war, while just 5% went to education. Globalissues.org puts military spending — and the financing of past wars — at 44.4% of the US tax haul, with education just under 7%.

In absolute terms, the US pays a tremendous amount for its education (putting aside “special budgets” for specific wars). Universities in the US are the most expensive in the world, and despite spending all that money, the K-12 educational system is dysfunctional.

Comparing standardized test scores, spending on teachers in North America climbed dramatically while performance remained flat.

In other words, despite spending a lot on education, we aren’t seeing good results.

It’s not just lack of money, or inefficiency, which undermines US education. Some regions of the country, like Detroit and New Orleans, are still dealing with economic and environmental collapse. To make matters worse, the country is still trying to decide whether it should teach science or faith, redacting history, evolution, and climate change while adding God to the curriculum. The strangely gerrymandered economics of textbook manufacture means that Texas gets to decide what goes into the books the whole country reads. And Texas has decided to rewrite history.

But amazingly, hippy lefty liberal that I am, I don’t want to complain about any of these problems. And I don’t want to just pick on the US here: similar inefficiencies are creeping into the education system of other Western nations, including Canada and the UK.

Instead, I want to talk about the biggest threat to education: Teachers’ unions. Here’s the argument in a nutshell: When you learn from a tablet, it learns from you. What if it learns that your teacher can’t teach?

What makes great teachers?

The Gates Foundation set out to try and understand why US education was so broken (education is one of the foundation’s core areas of focus.) It turns out that the single most important factor in children’s performance is having great teachers. Simply put, if your teacher is in the top 25 percent of teachers, your test scores go up by 10 percent in a single year.

In other words, if the entire U.S. population had teachers ranked in the top 25 percent, the gap between US and Asian schools would vanish in a single year — and in four years, the U.S. would be far ahead of the rest of the planet.

Think about that for a minute: make more good teachers, fix the future of a nation.

Simple, right? You just need to find and encourage those top teachers: put them in charge, give them money, learn from them. Find out what makes them better, and reward that.

Unfortunately, that’s not how the system works. The foundation’s research showed that the only predictor of whether a teacher is in the top 25 percent — the percent that can save us — is that teacher’s past performance. Factors like whether a teacher has a master’s degree, or how long they’ve been teaching, have nothing to do with their students’ performance. And yet those are two things that have a big impact on teacher salary.

Guess what doesn’t get a teacher a raise? That’s right: being a good teacher. Worse, Gates’ team found that on average, the slightly better teachers leave the system early in their careers, meaning that the longer a teacher has been teaching (and the harder they are to fire), the less likely they are to be good.

Teachers’ unions have numbers and money

Terry Moe, a senior fellow of the Hoover Institution and the William Bennett Munro Professor of political science at Stanford University, says that “unions are and have long been major obstacles to real reform in the [educational] system.” Rod Paige, the U.S. Secretary of Education under the Bush administration, goes one further, calling them terrorist groups.

Unions have strength in numbers. 12 percent of the US workforce is unionized; but with teachers, it’s 38 percent. In New York City, 96 percent of people who teach are in the union. The two largest unions, the NEA and the AFT, have 4.6 million members between them.

They have money, too. The NEA alone had $400M in dues in 2007. As a teacher, you can’t just decide not to be a union member: in California, if you opt out, you get back $300 of your $1,000 annual fee, the union still gets $700, and you still have to play by their rules. They put the money they raise to good use, too: teachers’ unions are the top political spender in the US — contributing more than double what the runner-up did to elected leaders and lobbyists.

What about charter schools?

Some schools are trying to fix this, particularly those known as charter schools. By challenging traditional public school methods and focusing on student performance, charter schools have shown amazing results. These schools tend to embrace teacher data. They show colleagues what works and what doesn’t. They teach in teams. They focus on teaching well. The result? In one case of a charter school that Gates cites, 96 percent of high school graduates — from the poorest regions of America — went to college.

Since they’re a threat to unions, charter schools are under fire. There are 4,600 charter schools in the US and 90,000 public schools. Those charter schools have huge waiting lists — 11,000 students in Harlem applied for 2,000 open slots recently. So why not make more of them? Because the unions don’t like them.

In Detroit a few years ago, a donor offered to spend $200M to set up additional charter schools in what is one of the worst regions of the country. Unions shut down the schools, demonstrated in the state capitol, and convinced politicians to turn down the money.

Lobbying against accountability

It’s not just charter schools that are in the unions’ sights:

• In New York City, when schools suggested they were going to use student test scores as one factor in evaluating teachers for tenure, the unions convinced state legislators to pass a law making it illegal to use those scores in evaluating teachers for tenure anywhere in the state of New York.
• It happens at the national level, too: a recent US House stimulus bill included funding for data systems to track and improve teaching, but the Senate removed it under pressure.
• During layoffs, seniority rules mean that junior people get laid off before senior ones. This purges good teachers from the system, since effective teachers, statistically, are young teachers.
• Unions have even successfully limited the number of times a principal can come into the classroom, even requiring advance notice of visits.
• Jaime Escalante, (played by Edward James Olmos in the movie Stand and Deliver) taught college-level calculus to “unteachable” gang members in L.A. He allowed more students in his class than the union contract allowed, so he was run out of town by the union.
• Bill Evers, a reformer in California, wanted to try a new way of teaching math into his schools. The unions killed it because learning the new teaching method was too hard.

The list goes on. Frustrated parents and administrators have hundreds of stories of the union protecting its members at the expense of students’ education, even when those members’ behavior is indefensible.

When administrators do find a bad egg, unions make it impossible to remove the rot. Unions fight for all kinds of protections that make it incredibly difficult to fire a teacher. On average, it takes 2 years and $200,000, and 15 percent of a principal’s time to remove one bad teacher.

Unions go to great lengths to protect tenured members — so school boards sometimes put alleged sex offenders into a district office rather than firing them. According to Sand, this is commonplace: one union rep admitted to him, “I’ve gone in and defended teachers who shouldn’t even be pumping gas.”

Administrators are terrified of the unions’ power, and capitulate. According to as explained in a recent Freakonomics podcast on charter schools, only 13 percent of teachers in New York can get 80 percent of their students to pass. Yet 99 percent of teachers get a satisfactory rating from administrators.

If only 10 percent of teachers are downright awful — as we’d expect from a normal distribution of teaching talent, were we allowed to analyze it — that’s still 5 million children stuck in classrooms with awful teachers who can’t be replaced.

In the end, unions hate accountability, calling it “scapegoating” of teachers. Yet a 2004 study by the Thomas B. Fordham institute concluded that 21.5% of urban public school teachers sent their own kids to private school, compared with 17.5% of all urban families and only 12.2% of all families in the U.S. A 1983 study of Chicago public schools showed teachers were twice as likely to send their kids to a private school. And it’s not because teachers earn more — indeed, the Fordham study found that “as income decreases, a greater percentage of urban public school teachers choose private schools.”

How can this be good for kids?

Clearly, this is protecting bad teachers at the expense of a nation’s education. But the union’s massive membership numbers give teachers’ unions tremendous power to change how education happens, despite having no accountability for the results. Speak out against them, and you’re a pariah, on the receiving end of hate mail and accusations of class warfare (which this series of posts will undoubtedly earn.)

Both Gates and Moe ask an awkward question: would anyone organize education this way if their top priority was the improvement of young minds? Probably not. Imagine running a company, asks Gates, where you couldn’t use someone’s performance to decide whether they got promoted, and weren’t allowed to supervise your employees or learn and teach what worked best. Sadly, that’s the state of much of North American education.

Teachers know what schools are like, though.

Simply put, unions resist anything that might weaken the stranglehold they have on the future of a nation’s children, and with it, the future of a nation. So guess how they’re going to feel about the accountable, free, extensible classroom-of-one that tablets promise — a promise that could reverse the decline of the North American empire. That’s what we’ll look at next.

Related posts:

1. Tablets, unions, and education – part one
2. Tablets, unions, and education – part four
3. Tablets, unions, and education – part two

In this four-part series, we look at the impact of tablet computing on education: how tablets can save North American students, but how their ability to collect and analyze how students learn will make teaching more accountable — something that unions will oppose aggressively as they try to protect their members’ jobs.

This is a detailed write-up of the Short Bit I first presented at Bitnorth 2010, with lots of background and links to references I found while putting together that presentation. We decided to break it into several parts to make it easier to digest.

Yesterday, we looked at the sorry state of Western education. Now we’re going to consider the ways in which a digital classroom — made manifest by the modern tablet — could reverse the decline.

Digital education isn’t a new idea. The One Laptop Per Child (OLPC) project started shipping cheap, reliable computers to students years ago, and the Web has been a critical resource for many rural and remote schools. But it’s the arrival of ubiquitous tablet computing that can really transform the modern classroom.

If students have their own tablets, they’re equipped with a powerful platform for learning. Here’s why:

1. They’re your entire curriculum

Consider an interactive tablet for education. It’s not just a book: it’s a musical instrument, a fitness trainer, an artistic canvas, a research system, a design tool. It would be an integral part of every class, from science to arts to shop.

The OLPC initiative has put XO laptops into the hands of 1.4 million children worldwide. The results of that experiment? Children teach their parents to read; students are absent from class less often; and teachers report that they love teaching.

2. They’re your library

A tablet is the planet’s biggest book. Armed with a tablet, a student would have the world’s content at her fingertips. The iPad may be first-generation device that doesn’t have a camera, or a microphone, or a GPS, in it yet. But it has millions of resources for students, such as:

• The Elements, an exploration of the periodic table that’s linked to online systems like Wolfram Alpha

• Toy Story, the interactive book that includes read-out-loud and painting modes
• The Khan Academy, a YouTube channel from a retired investment banker that has roughly 1,200 educational videos he’s made
• The MIT Open Courseware project, which puts the university’s curriculum and course materials online for anyone in the world
• Wikipedia, the pinnacle of Clay Shirky’s Cognitive Surplus, with (at last count) 3,391,351 articles on nearly everything

When books were made of atoms, publishers could control their distribution. But that model is in ruins, and we’re creating free, open content at a breakneck rate.

As Shirky points out, the huge gains in free time that came about from industrialization were largely consumed by the television, a passive, one-way device we watch for 200 billion hours a year. But the advent of the Internet has unleashed a huge cognitive surplus on humanity. We created Wikipedia — a true wonder of the online world — in just 100 million hours: a fraction of the time we spend watching TV in a single year.

OLPC’s strategy for their next-generation tablet is to ship each device headed for a particular community with a different set of a hundred books. Then, using peer-to-peer sharing, the community has a vast library: a village with a hundred students would have 10,000 books in its digital library.

3. They’re getting affordable

In The Diamond Age, our heroine grows up in a world of nanotechnology. It’s cheap to make anything, simply by writing the proper code and setting tiny machines loose in a vat of goo. But we’re not so lucky: tablets cost money.

Fortunately, they’re coming down in price. The Indian government recently announced a $35 tablet, which was likely a publicity stunt — the raw components cost $47 — but still shows the shape of things to come. The first OLPC struggled to cost under $200, but now a $75 one may be within reach based on the affordably priced Marvell chipset.

A concept design for the OLPC tablet

There are also Linux implementations on the horizon and both handheld gaming and mobile phones are driving down the price point of portable devices. Today, a $50 phone has a Java engine, camera, speaker, microphone, and more.

Even if tablets are expensive, they’re still cheaper than the alternative. Printed textbooks are a $4.3B a year industry, dominated by a few large publishers who frequently edit their books to encourage students to buy new ones rather than resorting to cheaper used texts. The average high school student has 7 textbooks worth $70 to $90 each; by contrast, a Kindle costs around $260, and an iPad around $600, but eBooks pricing means the total cost of a year’s books is lower than the printed alternative.

Tablets are cheaper than printed texts over their lifetime

The economics are so attractive that the Italian government has just stated that starting in 2011, teachers can only use books that are available partly or entirely online. They estimate a 90% savings of the €200 to €250 a year spent on books per student. Tablets also tackle many of the issues that made student notebooks expensive: they have no hinges, keyboards, or mice.

4. They’re personalized and interactive

Tablets are more than just free course materials. Tablets can tailor the content, format, and rate of learning to each student. That means no more one-size-fits-all learning.

How tablets will connect all the stakeholders in a child's education

Having trouble? Get remedial help or alternate explanations. Dyslexic? Watch a video. A quick learner? Dig deeper or talk to a career counselor. Need more assistance? Hire a tutor, on demand, from anywhere on the Internet.

5. They’re analytical

And here’s where it gets really interesting, because interactivity means analytics. Simply put: when you learn from a tablet, it learns from you.

As students work with digital course material, they leave a trail behind them. The tablet can record what’s being read — and what’s being skipped. Teachers can tell whether a particular student isn’t spending enough time on their lessons, or is breezing through them.

Tablets can be the basis for testing, allowing teachers to assign homework within the device and check it efficiently. Parents can show they’ve checked their child’s work. Students can flag subject matter they don’t understand, so teachers can see where the class is stuck.

So maybe, by ushering in an era of cheap, tailored, analyzed learning, tablets can leave no child behind. But they face one huge, terrifying obstacle. When you read a tablet, it reads you. Tablets are the basis for unprecedented analytical insight into what makes students smart or stupid.

An analyzed, accountable education sounds great, unless you’re the largest political force in the US: Teacher’s unions. We’ll look at that in detail in the next post.

Related posts:

1. Tablets, unions, and education – part one
2. Tablets, unions, and education – part four
3. Tablets, unions, and education – part three

In this four-part series, we look at the impact of tablet computing on education: how tablets can save North American students, but how their ability to collect and analyze how students learn will make teaching more accountable — something that unions will oppose aggressively as they try to protect their members’ jobs.

This is a detailed write-up of the Short Bit I first presented at Bitnorth 2010, with lots of background and links to references I found while putting together that presentation. We decided to break it into several parts to make it easier to digest.

In his novel The Diamond Age, author Neal Stephenson describes a digital book his heroine carries with her. Dubbed the Young Lady’s Interactive Primer, this device is part guidebook, part tablet, and part personal guardian. It’s interactive, changing stories and allegories based on the predicaments our heroine faces. Some of its content is recorded; much of it is prepared, on the fly, by actors thousands of miles away.

Much as he colored in the picture of virtual reality — Stephenson coined the term Avatar as a representation of a virtual self, and his novel Snow Crash is the inspiration for Second Life — he may have nailed tablet computing. With the release of Apple’s iPad, we’re finding dozens of uses for a device we didn’t know we needed. It’s a console, a reader, a movie screen, a musical instrument, a game board, and a window into other worlds.

Beyond all these uses, however, the killer app for tablets could be education. Done right, personal tablets can reverse the precipitous decline of learning in much of the Western world. By putting the world’s knowledge at a student’s fingertips virtually for free, making it interactive, and tailoring it to each student’s abilities and interests, tablets could completely alter the way we teach and learn.

Education is the foundation of prosperity

An illiterate, misinformed population is the hallmark of a failed state. Less educated citizens can’t compete in the global economy, are easily swayed,  and are less likely to make informed decisions. In an information age, an inability to work with information is a death sentence — almost literally, since it correlates with higher rates of infant mortality and lower lifespans.

A Gapminder visualization of the correlation between literacy and life expectancy

Education in the Western world, particularly North America, is on the decline.

One quick note on the scope of this series: As I learned about education, unions, and other subjects, I came across a wide range of research. A preponderance of it was specific to the U.S., but many of the observations about the decline of educational standards apply to a large number of Western nations. As a result, some of the data in here refers to the U.S., some to North America, and some to a broader set of countries. There are several European countries that are succeeding in education, so it would be wrong to lump them into this analysis.

That’s a pretty controversial statement. Many public indicators of literacy are up: the top quartile of America’s universities contain the smartest students anywhere, and standardized testing shows a modest improvement in literacy. The historian Lawrence Cremin argues that “Americans were a more literate population at the end of the 20th century than at any time earlier,” a direct consequence of access to public education.

Look deeper at the state of North American education, however, and things quickly go pear-shaped. Critics caution that the No Child Left Behind Act has simply lowered the bar enough that weak students, ill-equipped to survive in a knowledge-based society, can climb over it. Even Cremin cautions against defining literacy as no more than rudimentary technical skills in reading, ‘riting, and ‘rithmetic — because literacy aims at a changing target.

In a recent TED presentation that didn’t pull many punches, Bill Gates makes a case for fixing our ailing schools. If you’re a kid in the U.S., he points out, you a have 30 percent chance of never finishing high school. If you’re a minority, that’s more than 50 percent. Barbara Amiel recently observed out that one in 31 Americans is involved in the legal system somehow, whether by probation, incarceration, or investigation. With numbers like these, Gates reminds us that if you’re in a low income bracket, you have a higher chance of going to jail than you do of getting a college degree.

Numeracy — quantitative literacy — is particularly bad.

According to Mathematics and democracy: the case for quantitative literacy, “Data from the National Assessment of Educational Progress (NAEP) show that the average mathematics performance of seventeen-year-old students has risen just one percent in 25 years and remains, at 307, in the lower half of the “basic” range (286–336) and well below the “proficient” range (336–367). Moreover, despite slight growth in recent years, average scores of Hispanic students (292) and black students (286) are near the bottom of the “basic” range.”

What’s scary isn’t just that education has stalled — it’s that the rest of the world has kept going. We need more literacy and numeracy today than we did decades ago. Today’s citizen must understand risks of infection and read voting results. He has to screen advertising claims and measure growth rates. He must interpret data and work with computers every day.

Cremin describes two kinds of quantitative literacy: inert and liberating. “Inert” literacy is the basics: comprehending instructions and performing repeated tasks. It’s the kind of literacy we’d expect in a stalled state, where citizens have limited opportunity and aren’t expected to question or change. And increasingly, it’s considered good enough in our educational systems.

By contrast, Cremin’s “liberating” literacy means that individuals can find, analyze, and communicate information. They can apply it to problems and use it to make decisions. The educator John Dewey called this “popular enlightenment” and it’s the basis of a democracy, because only liberated citizens can think for themselves, make their own decisions, and discriminate between truth and lies.

Most U.S. students lack the literacy they need to live well in modern society; they suffer from “math panic” and retreat into faith and bumper-sticker politics. They’re ill-equipped for today’s in-demand jobs. Virtually every college finds that many students need remedial mathematics. We don’t even understand the equal sign properly.

Not fully understanding the “equal sign” in a math problem could be a key to why U.S. students underperform their peers from other countries in math.

“About 70 percent of middle grades students in the United States exhibit misconceptions, but nearly none of the international students in Korea and China have a misunderstanding about the equal sign, and Turkish students exhibited far less incidence of the misconception than the U.S. students,”

Students who exhibit the correct understanding of the equal sign show the greatest achievement in mathematics and persist in fields that require mathematics proficiency like engineering, according to their research. “Chinese textbooks provided the best examples for students and that even the best U.S. textbooks, those sponsored by the National Science Foundation, were lacking relational examples about the equal sign.”

Two million minutes

Students have about two million minutes of high school, and how they spend that time varies widely throughout the world. The documentary 2 Million Minutes looks at this period in the lives of three pairs of students: two from California, two from China, and two from India. It’s a depressing story.

When the filmmakers asked the overseas students what they wanted to be, Chinese students listed scientist, astronaut, or doctor. Indian students aspired to be a computer programmer, or a physicist.

Americans, by contrast, told the filmmakers they simply wanted to be celebrities.

The film underscores the vast differences between learning in Asia and America. In the U.S., if a child shows prowess at sports, they get her a coach. In India, if a child shows promise in school, the parents hire a teacher. But in the U.S., tutors are largely remedial: We don’t encourage intelligence; rather, we try to correct stupidity.

This is a problem that’s finally getting mainstream attention. Several other films are hitting the screens now. Take a look at this trailer for Waiting For Superman, which tackles the subject of charter schools and the collapse of the educational system.

Then have a quick look at The Cartel, a documentary on corruption and obfuscation in school systems.

If you watched those three trailers, you may be wondering what the future looks like. It’s bleak. North America doesn’t value education and science. We put athletes and musicians on pedestals, ignoring teachers and dismissing scientific inquiry in our politics and media.

We’ll pay the price for this attitude, but it will take decades. The effects of a weakened educational system take a generation to show themselves in the workforce — long after the party that weakened the system is no longer in power — so it’s easy for elected officials to turn a blind eye to the problem.

Big classes make dumb students

Education is imperiled for dozens of reasons, and there are no magic cure-alls. One big problem, however, is classroom size. Simply put, teaching doesn’t scale well. The Center for Public Education reviewed 19 well-researched studies, and concluded that smaller classes did better.

They found that:

• Smaller classes in the early grades (K-3) can boost student academic achievement;
• A class size of no more than 18 students per teacher is required to produce the greatest benefits;
• A program spanning grades K-3 will produce more benefits than a program that reaches students in only one or two of the primary grades;
• Minority and low-income students show even greater gains when placed in small classes in the primary grades;
• The experience and preparation of teachers is a critical factor in the success or failure of class size reduction programs;
• Reducing class size will have little effect without enough classrooms and well-qualified teachers; and
• Supports, such as professional development for teachers and a rigorous curriculum, enhance the effect of reduced class size on academic achievement.

The more students you have in a class, the less tailored each lesson can be to individual students. That means lessons are one-size-fits all, watered down by special interest groups and structured to appease the lowest common denominator. Big classes are distracted classes, and bad kids enjoy the same asymmetry as terrorists, because it takes very little effort for one student to interrupt the learning of forty others.

There simply aren’t enough teachers to give every student personal attention, helping those who struggle and accelerating those who excel.

In part two, we look at something that could reverse the decline of education in much of the Western world: tablet computing, and its promise of an interactive, digital, analyzed “classroom of one”.

Related posts:

1. Tablets, unions, and education – part three
2. Tablets, unions, and education – part two
3. Tablets, unions, and education – part four

In the 1970s bestseller The Joy Of Sex, we learn about a man who could only be aroused in a bathtub full of spaghetti. Back then, he probably led a lonely, normal life — albeit one in which he bought a lot of pasta and had a higher water bill than his neighbors. It’s unlikely that he had friends who shared his particular turn-on.

Communities define decency differently. In Canada, certain sexual practices are only legal when two people are present; no spectators. Until a 2003 Supreme Court ruling, many states outlawed specific sexual acts, and many of them remain on the books despite being invalidated by the high court. Communities were defined by their geographic boundaries. That’s why adult sites contain text like this: (and I’m just making this up here (okay not really)):

THIS SITE CONTAINS ADULT MATERIALS OR MATERIALS THAT MAY BE CONSIDERED OFFENSIVE IN SOME COMMUNITIES. YOU MAY NOT ENTER THIS SITE IF YOU ARE EASILY SHOCKED OR OFFENDED OR IF THE STANDARDS OF YOUR COMMUNITY DO NOT ALLOW FOR THE VIEWING OF ADULT EROTIC MATERIALS!

Which leaves me wondering: what’s the standard of my community?

Mass-market pornography died with mass-market media; today, specialization is the key. In the demented tapestry of the modern Web, one can find most any kink or predilection (see also rule 34.) Today, our ravioli romancer can find his pasta porn easily.  And it’s probably porn his geographic neighbors won’t like.

Of course, these days, his community isn’t just the people next door; it’s the people he hangs out with online. The Web holds more than pasta porn for our Fettuccine Fancier — it’s where his friends are.  The more he hangs out with kindred spirits, the more likely he is to conclude that he’s in the majority.

Surrounded by our fellow linguine lovers, or Daily Show fans, or Tea Party protesters, we’re emboldened.

The everyone-else-is-doing-it problem

Our brains are designed for tribes. The Dunbar Limit suggests we can track just under two hundred interpersonal relationships, and this appears to be wired into us: different primates have different Dunbar numbers.

The Web stumps our tribal brain. If many people in our cannelloni Casanova’s community crave soggy noodles the way he does, then soon, he believes himself to be entitled, part of what is — to him — the moral majority. He lobbies for noodle subsidies, marches in the streets in a Lasagna leotard, and raises money for entirely new kinds of food-borne illness.

To everyone else, he’s just weird.

Putting up walls

Many people, each convinced they’re in the moral majority, is a bad thing for consensus. These “moral minorities” emphasize how different we are from one another, rather than exploring common ground. Rather than becoming tolerant of one another, we reinforce the perimeter defenses of our own tribe, something that’s playing out right before our eyes in the zero-sum games and scorched-earth attitudes of (for example) US debates around taxation, health care, and gay marriage.

The ultimate irony of a Web that bring us closer to like-minded citizens is that it splits us apart from humanity. Rather than a smooth fabric of consensus, tomorrow looks like a noisy quilt of special interests and squeaky wheels.

It’s easy to forget that the Internet was built by hippies and hobbyists, idealists who were relatively free from commercial interests and were far enough from the mainstream to escape regulation. We may one day remember the days of Usenet and Napster as the Web’s Golden Age, a time before regulation and governance.

Fast-forward thirty years, and much has changed. The Web is the key to winning hearts, minds, and elections. It can undermine regimes and expose corruption. Because it’s so prevalent, we try to legislate it. We inspect packets, take down content, and put up firewalls.

This medium, spawned by utopian idealists, is looking more and more mainstream every day. Which, of course, collides with the idea of independent communities and squeaky wheels finding a place to live out their alternative lifestyles. It’s not much of a hiding place when it’s streamed into every phone, laptop, and living room. That has important consequences for how we legislate online behavior, and how our culture deals with a commingling of our public and private lives.

Making rules when everyone’s a freak

How should we govern this coming citizenry of moral minorities?

First, legislators must recognize that the Web gives us the right to be weird at scale. Adult content has long placed the burden of decency on the consumer. But the burden is really on all of us, because now community standards are contextual, despite the fact that we hold people to a fixed standard based on where they live and vote.

One day soon, someone will run for office. And someone who doesn’t agree with him or her will leak the history of all the online porn they’ve watched. Christine O’Donnell is dealing with a deliciously ironic witch hunt right now, in fact. We’ll have to decide whether a politician’s behavior is scandalous — because it’s not acceptable to even the most vocal, restrictive communities — or whether it’s his or her own business, and therefore should be discounted.

Canadian Prime Minister Pierre Trudeau famously said, “there’s no place for the state in the bedrooms of the nation.”

But Trudeau also said, “when it becomes public it’s a different matter.” And now, as we live lives in the open, everything’s public.

We have three problems to deal with.

• Everything is available everywhere. The whole world’s kink can leak into anyone’s geographic community over a broadband connection, and you can find support for nearly any idea if you look hard enough.
• We’re enthusiastically public. Social networks, a sharing culture, and like-minded communities mean that we make everything from sexual preference to religion to political bias a matter of public record, often with unbridled narcissism thrown in for good measure.
• We have no common standards for decency. It’s impossible to filter for decency at the source in a truly world-wide Web. Need proof? Just look at what Wikipedia is dealing with when it comes to material that some consider inappropriate.

We can either adopt a “squeaky wheel” standard of decency — banning anything that offends anyone, thereby ensuring that we’ll be led by the blandest, and most secretive, among us. Or we can accept that everyone’s a bit weird, including lawmakers, and legislate from that starting point. Their nocturnal cravings shouldn’t exclude them from a productive daily life.

As citizens of the Web, we must understand that just because we’re in touch with like-minded freaks, that doesn’t mean we’re the majority. The Web will pull back the veil of privacy, and our culture will adjust to a world in which everyone’s a freak, and nobody’s worse — or better — for it. If we give up our privacy, we need to gain some contextual tolerance.

The walled gardens we so often decry are also the edges of our communities. What’s acceptable at home won’t fly at the office, as Facebook’s new Employers Portal should remind us. Right and wrong aren’t as simple as “community standards” any more; we need to weigh someone’s behavior according to the context in which it occurred.

Privacy discussions aren’t really about privacy at all. They’re about the fact that our communities’ borders now change dynamically, according to time, place, and medium. An Internet with garden walls, that forgets, is a good start; after all, good fences and a charitable forgiveness make good neighbors. But a generation that judges decency in the context of others’ communities, not their own, is the basis for a connected, civil society.

Ultimately, we need to rewrite the Golden Rule: Do unto others as you’d have them do unto you, if you had their friends, preferences, and communities. Anything else is just imposing your worldview on the rest of the planet.

Related posts:

1. Welcome to Australia. Can I see your porn?
2. The slippery slope to a dull, safe Internet
3. Who owns your voice online?

Do the benefits of online sharing make the loss of privacy worthwhile?
Julie Matlin extolled the virtues of blogging as a means of coping with stress, making sense of difficult situations and finding support. She explained that since she began writing her intensely personal “Mommy Said What?” blog, she feels an immense sense of freedom to be herself. People judge her less, and understand her behaviour better, when they know exactly what is happening in her life at any given time.

How will we augment our bodies in future?
Donald Donovan explored some ways in which humans could add new senses to the human body by installing sub-dermal magnetic implants. We could detect poison gases or heat signatures, know compass direction instinctively, use lie detection in everyday conversation, or even have truly silent ringtones. RFID chips can be implanted to allow us unlock doors with a wave of the hand. This sparked discussions about the future of the human race. Will we add so many robot parts we will cease to be human? Is the ultimate goal of man-machine interaction a sort of Borg-like collective consciousness or hive-mind? James Duncan also raised concerns about how mankind is using up the planet’s resources of not just oil, but also food and water. Attendees debated whether technology will save us or if we will wipe out civilization before we can organize ourselves to act.

Could tablets save education?
Perhaps our best hope for the future is to improve education. Many people believe that the iPad, and other tablet devices, could revolutionize education, by letting every student progress at their own pace, a true “no child left behind” for our children. What’s more, teachers can then use analytics to see what is working, what areas need more attention, and to tailor their lessons to help every child reach their potential. Alistair Croll explored these ideas and also predicted that teachers’ unions will fight such progress, as it would move teaching (in the US at least) back to a performance-based model of recognition. David Chouinard took this a step further and argued that the entire model of education today is based on outdated industrial-era thinking, and that we should harness the power of the Internet to create more collaborative, digital schools for learning.

Is boxed-in thinking inhibiting the progress of society?
The prevalence of outdated ways of thinking was a recurring theme. Xiao Yu reminded us that we still use the QWERTY keyboard – a layout designed to slow down typing so that typewriter hammers don’t get jammed. The Dvorak keyboard is provably more efficient, but we stick with what we know, even on new types of device like the iPad. I (Alex Bowyer) proposed that the model of files and folders is similarly dated and inefficient. We should start thinking of our personal data as streams, which would not only help us manage information overload, but enable much more powerful applications and computer systems to be built. It seems that human society is highly resistant to change.

Are we losing serendipity?
We don’t just resist change, we seek out things we already know & like. Amazon, Pandora and Google News all encourage you to find content similar to what to you like. Dating sites help you find other people that are just like you. Lenny Rachitsky put forward a convincing case that while having such systems tailor the digital world to us might seem attractive, we are losing something valuable in the process – the happy accidents that allow us to stumble upon new foods we love, fall in love with people with whom we share nothing in common, or broaden our minds by exposing us to new and exciting ideas. Jeremy Edberg and Ross Noble also reminded us of the value of exposing ourselves to people and ideas outside our immediate influences – as engineers, businesspeople, or just as humans. Mike Gero pointed out that sometimes as we inject technology into hobbies like fishing, we lose something of the escapism we originally sought.

How can we model real-world relationships?
In the digital age, new technologies create new problems. But the impact of social sites like Facebook has rarely been as profound as that experienced by Stephane Bousquet. When he and his wife broke up, her simple act of changing her status from married to single on Facebook, broadcast to all of his friends that he was now single, which of course had profound consequences for all his relationships with friends and family. In the physical world, we have much more control over when and how information passes to our friends – in the Facebook age this was the equivalent of calling up 500 friends right after the other and saying “Hey, we’ve split up”. Who owns data that represents our shared relationships and personal interactions – should it be like a joint bank account, controlled by both parties?  Bryan Watson looked at another impact of social networks, pointing out that as we collect more and more online “friends”, we are effectively sacrificing a few strong ties for a larger set of weak ties – in effect devaluing friendships – something that rang true for many of the attendees.

Games, music and storytelling
Not all the “short bits” were what-ifs and thought exercises – there was much fun to be had too. We saw demonstrations of how to change a digital photograph into music (Christine Davis), how to build your own iPad-controlled Arduino robot (Nick Kruchten), and how a wall might be turned into an interactive touchscreen (Theo Ephraim). We were reminded how simple human acts remain just as important today as they were before the Internet; Julie Steele explored storytelling and its role in helping us understand the world we live in, and Gina Minks explored the impact of the digital divide and the dangers of digital recordings being taken out of context – something that is far easier today than it used to be.

Ilana Ben-Ari showed this inspiring video created by Charles Ledbeater, reminding us we are going through a period of great change as a society. Ilana then demonstrated her new communication skills game, Twonicorn – which is a great non-technical example of the mass innovation the video talks about.

Next year, Bitnorth will take place on September 17th and 18th.

Photos by Eva Blue (cc) and diagrams/screenshots are by the presenters.

Related posts:

1. Human 2.0 Holiday Highlights
2. Who owns your voice online?
3. The Impact of Social Models

Kate Lund from the group highlighted the limitations of the few location-based games that have reached the public attention. For example Foursquare and Gowalla have very simple actions and very limited gameplay. The group had took inspiration from geo-caching, noting that is is inclusive and easy to do, but has limited appeal.

They re-invented geo-caching as a game for families and children, creating a new mobile game called “Free All Monsters”. Children can use their creativity to draw monsters, these monsters then get transplanted into the real world, where they and their friends can then use a “Magical Monstervision Machine” (a Nokia N95 running special software) to detect and find monsters in the real world. The display overlays the sensor information and monster pictures onto the real world, much like Layar and other augmented reality applications:

The game reinvents geocaching in a creative, understandable way. For example, the strength of the GPS fix is represented as a “Captoplasm” gauge – you can’t capture monsters if you haven’t got enough. The game reinforces creativity throughout. Children’s monster creations are added to a “Liber Monstorum” (book of monsters), which is used to populate the game world – personalizing the game to the players.

Players also have a “Monster Spotter’s Guide” (which helps encourage teamwork) and have a set of thought-provoking questions for players to answer for each discovery, like “What does this monster dream?” or “Where would he go on holiday?”

The game is also designed to keep players focussed on the real world (which is why the camera augmentation approach is chosen) and favours teamwork and fun over speed and competitiveness.

The game has been used successfully on a small scale at a number of outdoor open days, and will soon be released for use anywhere in the world as an iPhone application (early video here).

There’s more about the group and their research at mobileradicals.com

Related posts:

1. Great summary of the state of augmented reality
2. Farmville entangles players in a web of social obligations
3. Behavioral Analysis and the Age of Metrics

He gave many examples of existing systems that harness the Internet to help with human problems – such as 911.gov, a conceptual site which would allow US residents to report crimes, but more importantly to request and give assistance to each other. For example, allowing a disabled resident being able to find a volunteer to help them get out of the building in an evacuation. Real-world examples included amberalert.gov and nationofneighbors.net as well as the use of Twitter to track the spread of Californian wildfires. Another example was patientslikeme.com which takes a more open view to the sharing of personal medical data than most current medical institutions, but has shown measurable benefits for the participants.

Ben highlighted the nascent nature of such thinking in the public consciousness, and speculated that greater steps need to be taken to help the public see both what is possible but also to give them the tools to make better use of data for good. To achieve this, he said, we will need deep science research to take place which can then be applied to everyday systems and functions.

As an example, Ben introduced SocialAction, a network analysis tool for researchers which can uncover hidden information in human networks. In the following video you can see the tool being used to uncover the strength of relationships between US Senators who voted the same up to 2007. The tool shows that voting is highly partisan – and also identifiers outliers who vote independently, and those who do not have strong voting allegiance to either party.

The research, carried out at the University of Maryland, showed that such techniques can deliver valuable understanding and insights to practitioners in a variety of professions – significant discoveries were made by a political analyst, a bibliometrician, a healthcare consultant, and a counter-terrorism researcher. It can also be used for other problems, such as word sense disambiguation. In one study Flickr photos tagged with the word “mouse” were automatically separated out into those where mouse referred to the input device, the rodent, or the famous Mickey.

Finally Ben introduced NodeXL, a free plugin for Microsoft Excel on Windows that allows anyone to carry out this kind of analysis. With the tool you can import and analyse your own data sources, including your email contacts or Twitter or Flickr contacts. NodeXL is also the subject of a new book by Ben

Ben encouraged everyone to help progress the public understanding and skills in this area, and suggested that in today’s connected Internet age, network analysis could be taught in schools in place of a subject like calculus – as it will be more applicable in future to daily life for our children.

Related posts:

1. The WWW of 18th Century London
2. Behavioral Analysis and the Age of Metrics
3. Twitter and Facebook attacks highlight the need for a true social network

From just 45 phones, they recovered over 7,000 pieces of data including nearly 2,000 images and around 2,700 SMS messages. Over 1,000 pieces of data were personal in nature and over 200 were highly sensitive (including nudity, drug use, pornography, bank account details, health information and explicit material about other contacts on the phone).

Alarmingly, of these 7,000 items 1/7th were “deleted” – their users likely believed them safely destroyed. The team observed that mobile phone operating systems do not give users the ability to completely destroy the data (unlike hard drives which can be reformatted without third-party software). They found that 20% of detectable deleted data could be recovered, and also observed that older phones were considerably worse at securely deleting data, compared to smartphones.

The research, which is still ongoing, is described in this preliminary paper (PDF). In future work they plan to compare different digital forensic techniques and look at the effectiveness of “phone scrubbers”.

As we record and perform more and more of our lives digitally, data housekeeping is just one more skill we will need to learn if we want to maintain our privacy.

Related posts:

1. The Business of Mobility is Broken
2. Rogers takes Android users off the air
3. Ambient awareness – the next step in collaboration

Just as external storage can become a form of prosthetic memory, so computers can become prosthetic decision-makers. If we were to make them understand the dilemmas before us, computer assistants could advise us on the economically rational thing to do.

Would we be able to deal with being told we’re wrong so much of the time?

Evolution isn’t the best designer. While the variety of life is astonishing–prompting many to invoke a creator–a study of biology reveals plenty of inefficient compromises. For example, the Laryngeal nerve runs from the brain, down to the heart, and back up to the vocal cords. This is a terrible, wasteful, error-prone design; but because it arose from an efficient model in fish (which didn’t have long necks), it’s what we’re stuck with.

Here’s a video of a dissection of a giraffe, showing just how inefficient the nerve’s routing is. It’s not for the squeamish.

Our brains, like our bodies, evolved from these kinds of evolutionary compromises. We think we’re smart beings, making rational decisions about things; in fact, we tend to rationalize after the fact, and go with what worked in the past. This made good sense for our ancestors: they shouldn’t sit around thinking about whether that tiger was going to eat them, they should just run. We reinforce patterns that work, because they’re the ones that keep us alive.

One of the side effects of reinforcing our past beliefs is that we’re reluctant to reconsider things, even on the basis of new information. We play all sorts of mental tricks on ourselves to help us stick to our beliefs in the face of evidence to the contrary: adaptive preference formation, cognitive bias, and so on. All of these are attempts to relieve the discomfort of cognitive dissonance, a disconnect between our belief systems and the real world.

So if you’re answering a questionnaire, your early answers may bias you against later answers. Psychologists love this kind of research–generally replacing self-interest with candies, under the assumption that everyone has a sweet tooth.

According to recent research, when you ask a test subject something to do with fairness, their answer will cause them to “dig in their heels”, reinforcing their later behavior. A classic demonstration of this process is the Monty Hall problem: you’re presented with three doors, and behind one of them is a prize. You choose one of the three–but then your host reveals one of the other two and asks if you’d like to switch. This New York Times piece explains the problem, and even provides an online game you can play — though the piece reveals that some of this can be explained by pre-existing biases, not just those acquired during the test.

Whether it’s prejudice, or a learned bias forcing us to reject new evidence, the reality is that humans make decisions badly. The Prisoners’ Dilemma and other experiments show that we often make poor decisions rather than economically or morally rational ones (check out the awesome SMBC explanation of this dilemma.)

Can machines help us overcome these inefficiencies? If a computer could help guide us in decision making, we might overcome these biases. At the very least, we’d make better decisions on TV gameshows. Proponents would point out the tremendous benefits of rational decision-making: a utopian world where we all solved for maximal utility and minimal suffering. Detractors would see this as a great levelling, turning us all into automatons, bland communists with little incentive to try something new or take a risk, stripping away the biases and preferences that make us individuals.

If computers helped us decide, we’d find out that Dan Ariely is right: we’re largely irrational. But can our self-reinforcing psyches cope with being told we’re fools? Or will we reject the rational correction, retreating into costly self-affirmation and embracing our bad decisions?

Related posts:

1. Tablets, unions, and education – part three
2. Does Big Search change science?
3. A new model for paying artists online

Today we photograph more than ever before – and thanks to the negligible cost, we film situations that would never have been captured before. But police and other authority figures do not want to be recorded, and all over the world a battle is playing out between officials pushing current laws to extremes to prevent such recordings, and citizens who fight back with equal vigour to protect their freedom to photograph.

Should photography be criminalized and recording devices banished from any situation where that recording might be used for ill? Or should we assert our right to capture anything we experience as a fundamental right?

16 years in prison for recording a policeman?

In Maryland, USA, the State Police are currently prosecuting motorcyclist Anthony Graber, not for speeding – a charge which he readily admits – but for video recording the officer who arrested him, as shown above. He’s being charged under wiretap laws, the claim being that he illegally recorded a private conversation without consent. If found guilty, he could face 16 years in prison. The attorney defending Mr. Graber points out that “To charge Graber with violating the law, you would have to conclude that a police officer on a public road, pulling someone over, has a right to privacy when it comes to the conversation he has with the motorist. There’s more on the story here, here and here. Many are claiming this is an example of police intimidation.

Is photographing police a crime?

There have been many incidents over the last five years of citizens around the world being challenged for recording police. Here’s just a few examples:

• A man photographed a police car in a London park and was questioned by police under anti-terror laws.
• A press photographer was arrested in France – and allegedly assaulted – after photographing a recreation of a murder scene.
• A 16 year old boy was harassed by police for photographing police cadets at a passing out parade in Romford, UK.
• A freelance writer in Arizona had his camera seized and photos deleted when he tried to capture a skirmish that was taking place at a Mexican border crossing.
• A photographer in Nottingham, UK was arrested for photographing a police armed response.
• In Florida, a man was tackled to the ground and charged with nine misdemeanors after photographing police in a construction zone.
• A journalist was handcuffed and arrested for photographing a road traffic accident near Milton Keynes, UK.

In all of these cases, there is ambiguity over precisely which law, if any was broken. The incidents appear to be more fuelled by individual police officers’ sentiments about being photographed.

Andy Handley, the journalist in the last example, observed when interviewed at the time, “This is a worrying development in the relationship between [the police] and the media.. It is the right of the media to operate unhindered that helps keep the liberty of all in this country.”

With this, he has hit the nail on the head. In both the USA and the UK, police are effectively an extension of the democratically-elected government – ultimately answerable to the people – and while they have authority to enforce the rule of law, they are subject to those same laws themselves.

To prosecute citizens that photograph them would break this equality, and could enable governments or law enforcement agencies to abuse their power. If citizens cannot scrutinize police and hold them to account, what is to prevent them breaking the very laws they are there to uphold? As one commenter on the Graber case said, “The [US] first amendment was specifically intended to allow for dissemination of information regarding improper use of authority”. Or as Plato first asked in The Republic, “Who watches the watchers?”.

A delicate balance

While individuals feel their liberties are being curtailed, clearly police feel threatened by photographers, sometimes even just by their presence. And it’s true that terrorists could gather information for planning attacks by taking photographs of police operations or public gatherings. It’s also true that there are many innocent, legal reasons to photograph those same scenes. It is impossible to criminalize the intent, only the behaviour.

The police are in a difficult situation. When the Olympic flame visited Montréal earlier this year, I personally witnessed riot police being harassed by protestors, who threw snowballs to provoke them, while other protestors stood, “armed” with video cameras ready to capture any police wrong-doing. It was the first time I’ve seen a camera used as a weapon.

What do you think the law should be? Were these police right to intervene, or were the citizens right to stand up for their right to photograph?

Join in with this important debate and add your comments below – or on Twitter with the hashtag #human20.

We’ll be exploring this topic more in future posts, looking at the sensitive issue of photographing children as well as recent grassroots efforts to protect photographers’ rights, so send us your thoughts and we may include them in those posts.

Related posts:

1. Lytro – Start of a photography revolution?
2. Human 2.0 Holiday Highlights
3. Technology just slightly ahead of legislation

They say a picture’s worth a thousand words. But what’s a picture of a thousand words worth? Tag clouds – visual clusters of words where size denotes importance – have been used for everything from navigating blogs to finding popular photographs. They’re a must-have in Web 2.0. But when Jonathan Feinberg of IBM Research created Wordle in 2008, researchers and curious readers gained a new tool. You can generate a tag cloud of any body of text – from news articles or PhD theses to blog posts or whole books; here is a tag cloud of the Bible. The two most popular generators are Wordle and Tagxedo, but there are many alternatives.

Tag clouds can be used to quickly summarize the language used. For example, I generated this tag cloud of the coalition agreement between the Conservative and Liberal Democrat parties in the UK using Wordle:

You can digest some things right away – clearly this is a document about topics the two parties agree on. Also, the Liberal Democrats and their preferred topics (Europe, referendum, welfare) seem prominent – could this indicate they dominated the negotiations?

We must be careful. Tag clouds don’t take into account the context – positive or negative – in which a word was used. The prominence of words like work, tax, or referendum does not tell us anything at-a-glance about the authors’ attitude toward those topics.

However, during the lead up to the US Presidential election, the Boston Globe used Wordle to analyze the blogs of the two main candidates, and were able to draw some interesting conclusions:

Here is their tag cloud of Obama’s blog:

Here is the tag cloud of McCain’s blog:

We immediately see that McCain spent more words talking about Obama than any other topic – and that there are many more negative words in McCain’s cloud.

Tag clouds reveal a great deal about two sources when viewed side by side. A detailed analysis was published by the Globe, and a similar approach has been used to compare historical US presidential speeches.

Today, even people who don’t have a blog produce thousands of words digitally – in emails, on social networks and in text documents. Researchers can now make all sorts of interesting insights by analyzing people’s words in digital form. For example, Agatha Christie may have had Alzheimer’s. These tools are now available to everyone.

Here is a Tagxedo tag cloud generated from Stephen Fry’s blog:

Stephen writes a lot about the current events, such as the new UK government and the launch of the Apple iPad. Without ever meeting him or hearing him speak, you know that politics, technology and people are important to him. This is something employers in particular will find useful. With Wordle, you can even judge someone by the bookmarks they share on delicious - as my colleagues Roo Reynolds and Andy Piper have demonstrated.

Here’s a useful exercise: Find the longest document you’ve ever written, and paste it into Wordle. The results can be enlightening. I generated this tag cloud based on my “Zen of Productivity” post from 2008:

I can tell straight away that I over-use the word “things” and “something” and that I use simple words like “get”, “find”, “work” “want” and “make” heavily (which may or may not be a good thing). I can also tell that I favour analytical words like “think” and “system” over emotive words like “feel”, “love” and “creative”. As a writer, such information can be invaluable – it can help you tailor your language to your audience.

Some words appear more than once – “Happy” and “happy”, “get” and “getting”, etc. Plurals and capitalizations are thought to be different words. For a more accurate tag cloud, you often need to weed your text first by finding and replacing occurrences of similar words with a single form – for example finding all occurrences of “needs” and replace them with “need”.

These simple analyses are just the tip of the iceberg. Already, researchers are harnessing the outputs from this explosion of digital literacy to solve all manner of problems from automatically detecting plagiarism to understanding how children develop syntax. Entire fields of linguistic study are now automated. As computers understand more of our words and the semantic web grows, it’s only a matter of time until tools empower every one of us as linguistic forensic scientists.

Related posts:

1. Introducing the Body Area Network (BAN)
2. Human 2.0 Holiday Highlights
3. The perils of backchannels: Why Twitter should never take center stage