Skip to content Skip to navigation


You are here: Home » Content » The Paradox of Success


Recently Viewed

This feature requires Javascript to be enabled.

The Paradox of Success

Module by: John Kearon. E-mail the author

Summary: The paradox of success is you need to embrace failure to achieve it. This is a paper born out of a speech I gave to 600 17 year olds on entrepreneurship during the UK’s enterprise week in November 2007. It must rank as one of the most nerve jangling speeches I’ve ever done, since owning a teenager of my own I had strong clues that they can be a cynical and challenging audience. However, setting anxiety and cynicism aside, I really wanted to share something of what it takes to successfully invent an innovative new product and commercialise it. The reason for being invited to speak at these events is you’ve supposedly done something successful and the convention is to share this success as if it was the only possible outcome of your company’s brilliance. But something about this teenage audience, over half of whom said they wanted to start their own business, made me abandon convention and share the brutal, unvarnished truth that the essential, hard earned ingredient in successful creativity and entrepreneurship is…failure. Failure is the essential ingredient that nobody talks about or acknowledges and everyone tries desperately and understandably to avoid. But as any inventor, creative or entrepreneur knows, great ideas are not born perfect but are forged in the furnace of trial and error. As Darwin showed, this is the simple but brutal algorithm of life. This seemingly random but amazingly productive cycle of mutation and natural selection, trial and error, has produced the whole of the abundant diversity of life on earth. Niels Bohr, the Nobel Prize winning physicist said about progress in any field, “Mistakes are at the heart of progress, so our challenge as scientists, is how to make more mistakes faster”. Or as I said to my teenage audience, “you’ve spent the last few years being told not to fail by teachers and parents and yet if you really want to create anything genuinely new; you’ve got to start learning how to fail”. Pablo Picasso, who knew a thing or two about originality, had another way of putting it when he said “There’s nothing worse than a great start” having long since learned to celebrate failures and difficulties as key to finding completely new avenues of creative expression.

The Paradox of Success is You Need Failure to Achieve itLearning to Love Failure as Pioneers in the New Frontiers of Market Research

by John Kearon

This is a very personal paper, sharing a personal conviction about the paradoxical truth at the heart of successful innovation, extracted from years of exploration, observation and personal experience.

My hope is to inspire market researchers to recapture the experimental curiosity of the early pioneers, learn to take more risks, make more productive mistakes and create the innovations necessary to increase the growth and influence of the industry. Failing that, I just hope you enjoy reading it.


The paradox of success is you need to embrace failure to achieve it.

This is a paper born out of a speech I gave to 600 17 year olds on entrepreneurship during the UK’s enterprise week in November 2007. It must rank as one of the most nerve jangling speeches I’ve ever done, since owning a teenager of my own I had strong clues that they can be a cynical and challenging audience. However, setting anxiety and cynicism aside, I really wanted to share something of what it takes to successfully invent an innovative new product and commercialise it. The reason for being invited to speak at these events is you’ve supposedly done something successful and the convention is to share this success as if it was the only possible outcome of your company’s brilliance. But something about this teenage audience, over half of whom said they wanted to start their own business, made me abandon convention and share the brutal, unvarnished truth that the essential, hard earned ingredient in successful creativity and entrepreneurship is…failure.

Failure is the essential ingredient that nobody talks about or acknowledges and everyone tries desperately and understandably to avoid. But as any inventor, creative or entrepreneur knows, great ideas are not born perfect but are forged in the furnace of trial and error. As Darwin showed, this is the simple but brutal algorithm of life. This seemingly random but amazingly productive cycle of mutation and natural selection, trial and error, has produced the whole of the abundant diversity of life on earth. Niels Bohr, the Nobel Prize winning physicist said about progress in any field, “Mistakes are at the heart of progress, so our challenge as scientists, is how to make more mistakes faster”. Or as I said to my teenage audience, “you’ve spent the last few years being told not to fail by teachers and parents and yet if you really want to create anything genuinely new; you’ve got to start learning how to fail”. Pablo Picasso, who knew a thing or two about originality, had another way of putting it when he said “There’s nothing worse than a great start” having long since learned to celebrate failures and difficulties as key to finding completely new avenues of creative expression.

Model 5,127No one likes to fail and companies are no different, perhaps worse, but it seems failures are an inescapable part of successful invention and originality. One of the most prolific inventors ever, Thomas Edison, famously said, “I now know over a thousand ways not to make a light bulb” and tellingly the first successful Dyson vacuum cleaner was model number 5,127. What both these inventors embraced is the power of experimentation, where each new mistake teaches you something and the more audacious and new the mistake, the greater the learning. The challenge is to stay afloat long enough to eventually succeed. In my experience, large companies embrace the need for trialling ideas but it’s the failure part they struggle with. Failure is not generally good for careers and the tendency is to be too conservative, to narrow the field too quickly, to keep experimentation to a minimum and make every effort to reduce the risks of failure.

Double-Bind for ResearchReducing the risks of innovation failure is where quantitative market research enters the picture and it also introduces us to what I think of as the ‘double-bind of the quantitative research industry’. When your role is to act as an insurance policy and reduce risk, it’s both important and natural to build a reputation for certainty and not making mistakes. The downside of this for quantitative research is it has encouraged a culture of conservatism and reluctance to change and innovate. Where once quantitative agencies did huge amounts of primary research and experimented with new research techniques, the big agencies’ time and energy are focussed on acquisitions and consolidation, reducing effective innovation still further. The possible merger of TNS and GFK or will it be WPP illustrates the point. It is estimated that in the last 20 years, companies representing over 80% of all research spend have changed hands at least once. In the same period, the industry has grown by single digits each year and is a 1$25 billion industry, compared to the $250 billion Management Consultancy industry2 or still nascent paid-for-online-search industry at $35 billion. I have a feeling the market research industry could be significantly bigger had it continued the bold experimentation of its youth rather than default its energies into the often zero-sum game of M&A.

Research FrontiersTo progress and successfully innovate, the quantitative research industry needs to recapture the experimental curiosity of the early pioneers. It is necessary once again to learn how to take more risks, make more productive mistakes and create the innovations necessary to increase the insight, inspiration and predictive power of research and fuel the growth of the industry. This paper hopes to illustrate what it takes to innovate successfully by sharing world famous ‘failures’, the work of the early research pioneers, the missed opportunity of the large research groups and the new breed of research pioneers once again experimenting, innovating and providing the potential for significant industry growth and influence. My hope is that the developments captured in this paper are just the beginning of a long, sustained and exciting era of research inventiveness.

The Paradox of Success Is You Need to Embrace Failure to Achieve It

Since this paper was inspired by a speech, I thought I’d start with an edited extract from a Harvard graduation speech given earlier this year by JK Rowling, currently the world’s most successful author and also the world’s wealthiest female according to the Forbes Rich List.

Figure 1.

Figure 1
Figure 1 (graphics1.jpg)
3I have wracked my mind and heart for what I ought to say to you today. I have asked myself what I wish I had known at my own graduation, and what important lessons I have learned in the 21 years that has expired between that day and this. I have come up with two answers. On this wonderful day when we are gathered together to celebrate your academic success, I have decided to talk to you about the benefits of failure. And as you stand on the threshold of what is sometimes called ‘real life’, I want to extol the crucial importance of imagination.

These might seem quixotic or paradoxical choices, but please bear with me.

Looking back at the 21-year-old that I was at graduation, is a slightly uncomfortable experience for the 42-year-old that she has become. Half my lifetime ago, I was striking an uneasy balance between the ambition I had for myself, and what those closest to me expected of me. I was convinced that the only thing I wanted to do, ever, was to write novels. However, my parents, both of whom came from impoverished backgrounds and neither of whom had been to college, took the view that my overactive imagination was an amusing personal quirk that could never pay a mortgage, or secure a pension. I cannot remember telling my parents that I was studying Classics; they might well have found out for the first time on graduation day. Of all subjects on this planet, I think they would have been hard put to name one less useful than Greek mythology when it came to securing the keys to an executive bathroom.

What I feared most for myself at your age was not poverty, but failure. At your age, in spite of a distinct lack of motivation at university, where I had spent far too long in the coffee bar writing stories, and far too little time at lectures, I had a knack for passing examinations, and that, for years, had been the measure of success in my life and that of my peers. Ultimately, we all have to decide for ourselves what constitutes failure, but the world is quite eager to give you a set of criteria if you let it. So I think it fair to say that by any conventional measure, a mere seven years after my graduation day, I had failed on an epic scale. An exceptionally short-lived marriage had imploded, and I was jobless, a lone parent, and as poor as it is possible to be in modern Britain, without being homeless. The fears my parents had had for me, and that I had had for myself, had both come to pass, and by every usual standard, I was the biggest failure I knew.

Now, I am not going to stand here and tell you that failure is fun. That period of my life was a dark one, and I had no idea that there was going to be what the press has since represented as a kind of fairy tale resolution. I had no idea how far the tunnel extended, and for a long time, any light at the end of it was a hope rather than a reality.

So why do I talk about the benefits of failure? Simply because failure meant a stripping away of the inessential. I stopped pretending to myself that I was anything other than what I was, and began to direct all my energy into finishing the only work that mattered to me. Had I really succeeded at anything else, I might never have found the determination to succeed in the one arena I believed I truly belonged. I was set free, because my greatest fear had already been realised, and I was still alive, and I still had a daughter whom I adored, and I had an old typewriter and a big idea. And so rock bottom became the solid foundation on which I rebuilt my life.

You might never fail on the scale I did, but some failure in life is inevitable. It is impossible to live without failing at something, unless you live so cautiously that you might as well not have lived at all - in which case, you fail by default.

Failure gave me an inner security that I had never attained by passing examinations. Failure taught me things about myself that I could have learned no other way. I discovered that I had a strong will, and more discipline than I had suspected; I also found out that I had friends whose value was truly above rubies.

The knowledge that you have emerged wiser and stronger from setbacks means that you are, ever after, secure in your ability to survive. You will never truly know yourself, or the strength of your relationships, until both have been tested by adversity. Such knowledge is a true gift, for all that it is painfully won, and it has been worth more to me than any qualification I ever earned.

JK Rowling captures the drama and arc of the innovators story as you’d expect from a gifted storyteller. But this same narrative of personal mission, creative endeavour, and tolerance of failure, evolved solutions, perseverance, elapsed time and eventual success can be found at the heart of all stories of origination, invention and scientific breakthroughs. James Dyson in his autobiography ‘Against the Odds’, tells the extraordinary story of more than a decade of failure, perseverance and 5,127 models before the initial success of his now brand leading cyclone vacuum cleaner. Long odds and hard work it seems are the inventor’s lot or as Benjamin Franklin, a noted inventor himself, said, “Energy and perseverance conquers all things”. Winston Churchill, always good for a quote, had some useful advice, saying, “Success is being able to go from failure to failure with no loss of enthusiasm” and, “never give up, never give up, never, ever give up”.

Charles Darwin’s evolution by means of natural selection, described as the single best idea that anyone has ever had, provides a more scientific view of this same innovator’s story. As depicted in figure 2, it elegantly explains the ‘generate, test and learn’ algorithm at the heart of all creative output and the amazing, inventive plethora of life on the earth. As Richard Dawkins put it, ‘never has so much been explained by so little.’ Daniel Dennett describes it as ‘a universal acid for problem solving’ in his book ‘Darwin’s Dangerous Idea’. The Darwinian algorithm of inventiveness boils down to a simple process of blind variation and selective retention or, expressed in a more familiar way, trial and error!

Figure 2.

Darwin’s Algorithm of InventivenessGenerateTestGenerateTestGenerateTestGenerateTestGenerateTestTest

Figure 2
Figure 2 (.jpg)

Now although inventors, entrepreneurs and large companies alike, are setting out to direct their creativity to a commercially productive end, the reality of how inventive breakthroughs occur is I believe a lot closer to trial and error than usually acknowledged or admitted. This is something I’ll try and demonstrate in the ‘famous failures’ section of this paper as well as with a couple of BrainJuicer Labs case studies to illustrate this in the research pioneers section at the end.

This is not to say that success is random, just that the only way to achieve it is through a massive tolerance for failure and a systematic, iterative programme of trial, error and persistence. And I’m afraid it’s important to note that not all persistent trial and error leads to success. In fact I would hazard a guess that at least 99.99% of all attempts are failures but inevitably we only usually hear the success stories, so it’s tempting to assume that successful inventiveness is easier than it is in reality. There is a phenomena called the ‘even-odds rule’ which acts as a good reminder of how hard creativity and inventiveness really is. Dean Simonton explores the work of various academics in his book ‘Origins of Genius: A Darwinian Perspective on Creativity’, showing that even the most prolific creators in recorded history like, Bach, Beethoven, Chopin, Leonardo Da Vinci, Michaelangelo, Picasso, McCartney produced as much forgettable work as they did works of genius – but we only look at or listen to the good stuff, leaving us assuming, wrongly as it happens, that everything they did was touched with gold.

Perhaps the best way to illustrate the realities and challenges of invention is through the ultimate cheerleader for experimentation and perseverance in the face of failure, Thomas Edison, the most prolific and successful inventor of our time.

Innovate Like Edison

Figure 3
Figure 3 (graphics2.jpg)
Michael Gelb’s 2007 book Innovate Like Edison, provides terrific insight into the ultimate inventor, the ‘Wizard of Menlo Park’ and the man credited with inventing the electric light bulb, the record player, moving pictures and much more. In fact Edison holds the record for the greatest number of patents held by one man [1,093], although as we’ll see this was not the work of a lone genius but as a result of a systematic approach to innovation he developed at Menlo Park, with a wide team of fellow inventors experimenting and inventing on a wide range of ideas.

Figure 3.

Gelb identifies five elements that Edison and his team used in their intensive “hunt” for innovative solutions, which strongly mirror the trial and error algorithm of inventiveness described in the previous section, along with the elements that seem to appear in every innovation story, personal mission, creative endeavour, tolerance of failure, evolved solutions, perseverance, elapsed time and eventual success.

1. Align Goals with Passions

“I never did a day’s work in my life, it was all fun”, was Edison’s way of expressing the close alignment of his goals and passions. He had a deeply held desire to pursue knowledge and provide products and services that improved people’s lives. Contemporary psychological research has confirmed two of the most important traits of successful creative’s are 1) passionate goal directedness, and 2) perseverance through self-control. Dr. John Dacey and Dr. Kathleen Lennon studied decades of work by psychologists including Mihaly Csikszentmihalyi, Paul Torrance, David Perkins, Robert Weber et al, showing that passionate goal directedness helps successful individuals generate “great amounts of energy to invest intensely in their work”. Whilst Edison became hugely wealthy, it’s instructive to note that making money was not his primary motivation, “One might think that the money value of an invention constitutes its reward to the man who loves his work. But…I continue to find my greatest pleasure, and so my reward, in the work that precedes what the world calls success.”

2. Cultivate Charismatic Optimism

“Look on the bright side of everything”. Edison had an extreme way to find the silver lining in every cloud. Dr. E.G. Acheson, a fellow Menlo Park experimenter put it this way:

I once made an experiment in the later part of 1880 and the results were not as looked for. I considered the experiment a perfect failure, and while bemoaning the results of this apparent failure Mr. Edison entered, and after learning the facts of the case, cheerfully remarked that I should not look upon it as a failure, for he considered every experiment a success, as in all cases it cleared up the atmosphere, and even though it failed to accomplish the results sought for, it should prove a valuable lesson for guidance in future work. I believe that Mr. Edison’s success as an experimenter was, to a large extent, due to this happy view of all experiments.

Edison himself said, “Our greatest weakness lies in giving up. The most certain way to succeed is always to try just one more time.” Dr. Martin Seligman, author of Learned Optimism has shown in his work that optimists get better results than pessimists in most areas of life and make significantly more money, even though pessimists are more skilled, he found, in their ability to analyze current problems accurately. Optimists it appears get better results because they remain passionately focussed on finding solutions in the face of appalling odds and apparent results. “It just never occurred to us that this might be an unsolvable problem”, Dr. Donald Keck, co-inventor of optical fibre.

3. Seek Knowledge Relentlessly

“I didn’t read a few books, I read the library”. Edison might struggle to achieve this in today’s information rich word but he never started a round of experiments without first reading everything that was available on the subject. He read widely and voraciously and deliberately including popular magazines as well as technical papers and books on the subject. In NLP terms Edison was a visual/kinaesthetic learner, so naturally pictured things in his mind’s eye and loved to explore the world physically. Edison’s philosophy was, “I see what has been accomplished at great labour and expense in the past. I gather the data of many thousands of experts as a starting point and then I make thousands more,” and, “We don’t know a millionth of one percent about anything.”

4. Experiment Persistently

“The only way to keep ahead is to experiment… When there’s no experimenting, there’s no progress. Stop experimenting and you go backward.” For Edison, experimentation was the practical driving force of the innovation process. He deliberately created an ‘invention factory’ at Menlo Park to fulfil his own challenge of churning out an invention “every ten days, and ‘a big thing’ every six months”. One laboratory employee described a typical but extraordinary ‘hunt’ for the first alkaline storage battery:

When asked how many experiments had been made on the Edison storage battery since the year 1900: “Goodness only knows! We used to number our experiments consecutively from 1 to 10,000, and when we got up to 10,000 we turned back to 1 and ran up to 10,000 again, and so on. We ran through several series – I don’t know how many, and have lost track of them now, but it was not far from 50,000.”

Persistence in experimentation is the sweat that Edison cites in his classic aphorism: “Genius is one per cent inspiration and ninety-nine per cent perspiration”. Charles Batchelor experienced whilst working on exploring possible filament materials. After kneading a lump of lampblack for what felt like an interminable time, he said to Edison:

“There’s something wrong about this, for it crumbles even after manipulating it with my fingers.”

“How long did you knead it?” said Edison

“Oh! More than an hour.”

“Well, just keep on for a few hours more and it will come out all right.”

This crumbly substance turned into a sticky putty and became the essential ingredient in carbonising the spiral filaments that were patented and used in the first ever light bulbs.

5. Pursue Rigorous Objectivity

“The ability to consider data without prejudice is a distinguishing characteristic of great innovators.” Edison apparently had an almost Zenlike quality of detachment when it came to the results of experiments. He trained himself to be neutral about outcomes, seeing each as bringing him one step closer to the answer. In one experiment, Edison sent a single Morse code “dot” through miles of undersea cable generating a 27 foot long printout! This weird result fascinated Edison into experimenting until he found the unintended carbon amplifier effect and three years later when Western Union asked him to improve the quality of Alexander Bell’s poor telephone transmitter, he applied the learning’s from his ‘failed’ earlier experiment to invent the carbon button transmitter which massively improved the phone’s performance and became the industry standard. Edison never tossed away any findings as useless and every experimental outcome represented knowledge, even if the findings were outlandish and without explanation. Edison respected the data and the weirder the results the harder he thought about them.

10 Famous Failures

Something I’ve consistently found researching the origin stories of brands or inventions that originated a category, is a failure, a mistake or problem that gave birth to the solution. Just in case the inference is that innovation is just luck and randomness, I would also say that in every case the inventor had been working on the problem for many years and had what has been referred to as the ‘prepared mind’ to realise the significance of the ‘mistake’ at hand.

Some of these origin stories you may have heard, others will hopefully be new to you but I share them to encourage you to embrace the potential rewards of systematically experimenting and failing and preparing your mind, on the road to innovation success.

1. Rubber4 1843

The "rubber fever" of the early 1830s had ended as suddenly as it had begun. At first everybody had wanted things made of the new waterproof gum from Brazil, and factories had sprung up to meet the demand. Then, abruptly the public had become fed up with the messy stuff which froze bone-hard in winter and turned glue-like in summer. Not one of the young rubber companies survived as long as five years. Investors lost millions. Rubber, everyone agreed, was through in America.

Goodyear, who had a fascination for this ‘mysterious gum’ decided he was the man to ‘bring rubber back’ and set about a series of experiments, adding magnesia and quicklime to improve stability and then nitric acid and sulphur which made the best rubber available but still ultimately failed. In the winter of 1839, he wandered into Woburn's general store to show off his latest gum-and-sulphur formula and whilst excitedly demonstrating his wares, a sticky fistful of gum flew from his fingers and onto a hot stove. When he bent to scrape it off, he found that instead of a melted molasses, it had charred like leather and produced a dry, springy brown rim - "gum elastic" - he had made weatherproof rubber. It took another five years to perfect the ‘manufacturing process’ but vulcanised rubber was in full production by 1843.

2. French Fries5 1853

In the summer of 1853, American Indian George Crum was employed as a chef at an elegant resort in Saratoga Springs, New York. On Moon Lake Lodge's restaurant menu were French fried potatoes, prepared by Crum in the standard, thick-cut French style that was popularized in France in the 1700s and enjoyed by Thomas Jefferson as ambassador to that country. Ever since Jefferson brought the recipe to America and served french fries to guests at Monticello, the dish was popular and serious dinner fare.

At Moon Lake Lodge, one dinner guest found chef Crum's french-fries too thick for his liking and rejected the order. Crum cut and fried a thinner batch, but these, too, met with disapproval. Exasperated, Crum decided to rile the guest by producing french fries too thin and crisp to skewer with a fork.

The plan backfired. The guest was ecstatic over the browned, paper-thin potatoes, and other diners requested Crum's potato chips, which began to appear on the menu as Saratoga Chips, a house specialty. Soon they were packaged and sold, first locally, then throughout the New England area. Crum eventually opened his own restaurant, featuring chips. At that time, potatoes were tediously peeled and sliced by hand. It was the invention of the mechanical potato peeler in the 1920s that paved the way for potato chips to soar from a small specialty item to a top-selling snack food.

3. Quaker Oats 1877

German immigrants to America introduced the world’s first breakfast cereal made of oats, at a time when only horses ate oats. Like many new category brands, Quaker Oats initially attracted much ridicule and the papers carried cartoons about oats making people ‘whinny like a horse’.

New category brands often bring other firsts with them and Quaker Oats was no exception. Quaker was one of America’s first national advertisers at a time when advertising was so disreputable that banks would not lend money to those who advertised. They were also the first to package their product in cardboard packaging, all staples having previously been sold from the barrel.

4. Coca Cola 18866

One urban legend suggests Coca-Cola was a non-carbonated headache remedy, until a soda jerk accidentally used the Coke syrup and added soda water to it. According to the official Coke archives, there are bits of truth in this legend, but the reality was more complex.

In 1886, Dr. John "Penny" Pemberton owned a reasonably successful bottled-beverage company in Georgia. He was driving down the street with a wagon full of his wares one day when his horse was spooked by another delivery wagon that came around a corner too quickly. In the ensuing confusion, the two horses became entangled and smashed through the front window of a dry goods store, their respective wagons close behind.

It was a horrible accident. The horses, hurt by the window glass, panicked and went wild in the store, badly injuring several employees and shoppers. Both wagon drivers were also injured and required medical attention. But amidst the chaos, an amazing discovery was made.

A young boy was passing the scene of the accident. He noticed a puddle of bubbling brownish liquid just inside the shattered store window and, taking care to avoid the broken glass, tasted it. "Wow! This is great!" he said, and invited his little friends over to "have some of this brown stuff and a smile." Apparently soda water from Pemberton's wagon had mixed with kola nuts and powdered cocoa from the other wagon and sugar and other ingredients from the dry goods store's display window and spontaneously created a new beverage. While waiting for someone to help him with his horrible head wound, Pemberton had the presence of mind to write down the names of the concoction's ingredients and he later used those notes to recreate the beverage in a laboratory.

5. Shredded Wheat 18937

A Denver lawyer called Henry Perky invented Shredded Wheat in his pursuit of a cure for his chronic indigestion. He boiled whole wheat and found it edible and good for his condition but not very appetising. He then left the boiled wheat to dry and baked it into a biscuit: Shredded Wheat.

His factory at Niagara Falls was one of the first showcase factories of the enlightened industrialists, providing everything for his employees; landscaped lawns, playgrounds, tennis courts and purpose-built accommodation.

6. Kellogg’s Corn Flakes 19038

Kellogg’s originated a number of new categories. Dr Kellogg, who set up Battle Creek Sanatorium (immortalised in the film The Road to Wellville), created America’s first health foods. These were based on cereals that he believed to have health-giving properties. These cereal products were sold by mail order to patients who had left the Sanatorium but wanted to continue Dr Kellogg’s rather peculiar diet. Since Dr Kellogg opposed the commercialisation of his products, it was left to his brother to create the Battle Creek Toasted Corn Flake Co., which luckily for copywriters soon changed its name to Kellogg’s Corn Flakes.

The rest is history, except to note that Postum, a cereal-based coffee Kellogg tried to sell, did not fare so well. Twenty-three years later, in 1926Kellogg turned his attention to another staple of the Battle Creek Sanatorium, rice. Among hundreds of experiments, the developers noticed one, which made a noise on contact with milk, and Rice Krispies was born. Like Quaker Oats before it, it was ridiculed. One memorable newspaper cartoon showed a cereal manufacturer presenting a bowl to his board saying: It’s the greatest thing since snap, crackle and pop it says grace!

7. Penicillin 19289

Penicillin was discovered by chance, in 1928, after Alexander Fleming accidentally left a dish of staphylococcus bacteria uncovered for a few days. He returned to find the dish dotted with bacterial growth, apart from one area where a patch of mould (Penicillin notatum) was growing. The mould produced a substance, named Penicillin by Fleming, which inhibited bacterial growth and was later found to be effective against a wide range of harmful bacteria.

Penicillin was finally isolated by Howard Florey and Ernst Chain. Fleming, Florey and Chain received a Nobel Prize in 1945, for their discovery which revolutionised medicine and led to the development of lifesaving antibiotics.

8. Post-It Notes 197410

Spencer Silver, a 3M scientist first created the ‘low-tack’ substance when an experiment to make highly sticky glue went wrong. He immediately thought the substance was interesting and spent several unsuccessful years trying to interest colleagues in developing applications.

It was pretty much forgotten when four years later, Art Fry, a colleague at 3M found an intriguing use for it. Fry sang in a church choir in North St. Paul, Minnesota and was frustrated that his bookmarks kept falling out of his hymnal. He had attended one of Silver's seminars, and, while listening to a sermon in church, came up with the idea of using the adhesive to anchor his bookmarks. Fry developed the idea by taking advantage of 3M's officially sanctioned "permitted bootlegging" policy and 3M launched the product in 1977 but it failed as consumers had not tried the product. A year later 3M issued free samples to residents of Boise, Idaho, United States and 90% of people who tried them said that they would buy the product. By 1980 the product was being sold nationwide in the United States and a year later Post-its were launched in Canada and Europe.

9. Sony Walkman 197911

Sony's R&D unit first created an experimental personal tape player for Ibuka, then Sony's honorary chairman, they dubbed the Pressman. He used the player on airplane trips, but he found the player too heavy for everyday use and instructed the tape recorder division to create a smaller version for his personal use.

Kozo Ohsone, who led the division, was eager to avoid having the division consolidated into another division (Sony was going through reorganization at the time) and quickly designed a portable tape player based on Ibuka's modified Pressman player and suggested the name Walkman. Before the new player was available to the public, the press lampooned it. Some claimed that nobody would be interested in a tape player without a record function. Others pointed out that the most popular tape recorder of the time had sold less than 15,000 units, and Sony had produced 30,000. In ten years Sony sold 50 million units

10. AlcoPops 198012

Alco-pops were originated by a lemon-grower in Australia, desperate for something to do with the remainder of a bumper crop he couldn’t sell. He added spirit to the squeezed lemons and created an alcoholic real lemon drink which he called Two Dogs (after a crude Australian joke). The product found a ready market as a refreshing alcoholic drink and an introductory drink for young adults.

In the United Kingdom, a senior Bass executive just back from Australia, is reputed to have said, don’t research it, just do it, and Hoopers Hooch was launched in record time creating a category worth over £300 million in just two years.

Market Research Industry’s Inventiveness and Willingness to Embrace Failure

Since the pattern of innovation seems constant in all fields, let’s take a look at inventiveness in our own field and explore the degree to which the original pioneers, current leading players and the new pioneers have been prepared to embrace failure and experimentation as a means of producing successful innovation.

Let me try and summarise the broad sweep of history that follows. Our research innovation story starts with dramatic experimentation and advances in mathematics between the C17th and C20th, followed by a similar degree of experimentation by the early research pioneers in applying this theory and developing modelling techniques. However, in the 1980’s and 1990’s, experimentation and advances in quantitative research seem to have slowed dramatically, to be replaced by an explosion of qualitative research innovation and a period of intense M&A activity and consolidation amongst quantitative agencies. As the current TNS-GFK-WPP battle illustrates, this focus by the biggest agencies continues to the present day. However, in the last decade, the rise of new technologies, particularly the internet, have seen another explosion of experimentation and advance in what might be better described as ‘quali-quant’ techniques. This clever fusion of new quantitative techniques with new ‘scalable qual’ methods, made possible by a new spirit of experimentation, and new technologies offers the exciting potential for the industry to increase it’s rate of growth and influence in the world.

Let’s start at the beginning.

The Early Quantitative Research Pioneers

It’s a difficult one to decide how far back to go and still feel relevant to a paper on the early pioneers of quantitative market research? But taking ESOMAR’s own superb research history timeline as my guide, I start with innovation of Probability Theory in the C17th, run through the development of statistics in the next two and half centuries, see the flourishing of choice modelling in the 1940s, developed further by mathematical psychology in the 1960s and finish with the growth of modelling in the 1970s13.

1. Mathematics of Probability and the Growth of Statistical Capability

Apart from the early Greek mathematicians and the Arabic innovation of zero, perhaps the earliest detectable innovation in daily use in quantitative market research is Probability theory first introduced in the seventeenth century by Pierre de Fermat and Blaise Pascal. This was soon followed by the probability curve, first put forward by Abraham de Moivre in 1733 and Thomas Bayes novel approach to probability in 1763.

Two major applications of probability theory in MR today are in assessing the reliability of data and predicting the likelihood of commercial outcomes. Governments typically apply probabilistic methods in environmental regulation where it is called "pathway analysis", choosing projects to undertake based on statistical analyses of their probable effect on the population as a whole. Companies do the same thing for their new products and initiatives.

These developments were followed and improved by Charles Freidric Gauss’ seminal work, ‘the method of least squares’ – which forms the basis of regression analysis. This grew out of work in astronomy and the need to combine data sets of different observations to establish best estimate of the true value, so errors decreased with aggregation rather than increased. An early demonstration of the strength of Gauss's method came when it was used to predict the future location of the newly discovered asteroid Ceres. On January 1, 1801, the Italian astronomer Giuseppe Piazzi discovered Ceres and was able to track its path for 40 days before it was lost in the glare of the sun.

Needless to say, these stories simplify the enormous trial and error and perseverance involved in these mathematical breakthroughs.

In the late nineteenth and early part of the twentieth century there followed a series of further statistical innovations that are now in common use in quantitative research today. In summary:

  • 1868 Standard deviation - Francis Galton
  • 1892 Correlation - Karl Pearson
  • 1906 Chi-square test - Karl Pearson
  • 1908 T-test - William Gosset
  • 1918 Analysis of Variance (ANOVA) - Ronald Fisher (and multivariate analysis)
  • 1934 Factor Analysis - Charles Spearman
  • 1937 Spearman’s rank Correlation Coefficient - Charles Spearman
  • 1945 Mann-Whitney U - Frank Wilcoxon

2. MR Application of Statistical Knowledge and Capabilities

This was the era best characterised by the scientific sampling and polling work of Dr George Gallup and what would perhaps be the beginning of the formal Market Research industry as we know it.

As a clue to his future role in the industry, Gallup’s doctoral dissertation was entitled ‘A New Technique for Objective Methods for Measuring Reader Interest in Newspapers’. After teaching at Iowa, he left in 1929 to head the school of journalism at Drake University, subsequently leaving in 1931 to teach and do research at Northwestern University. One year later he joined the Young & Rubicam (Y&R) advertising agency, where he conducted public opinion surveys and became that industry's first market research director. He remained with Y&R for sixteen years. While still at Y&R, he founded the American Institute of Public Opinion in 1935.

In 1936, his new organization achieved national recognition by correctly predicting, from the replies of only 5,000 respondents, the result of that year's presidential election, in contradiction to the widely respected Literary Digest magazine whose much more extensive poll based on over two million returned questionnaires got the result wrong. Not only did he get the election right, he correctly predicted the results of the Literary Digest poll as well, using a random sample smaller than theirs but chosen to match it and you could say the market research industry was born. In terms of failures, there were many but the most notable was in the 1948 election when Gallup wrongly predicted Thomas Dewey would defeat Harry S. Truman in the 1948 election, by five to 15 percentage points. Gallup believed the error was mostly due to ending his polling three weeks before Election Day, which is why today polling continues right up to the moment of the election for improved accuracy.

3. Choice Modelling and Mathematical Psychology

In parallel with the growth of scientific sampling and polling was the development of choice modelling, developed in parallel by economists and cognitive psychologists. The origins of Choice Modelling can be traced to Thurstone's research into food preferences in the 1920s and to Random Utility Theory and basically attempts to model the decision process of an individual or segment in a particular context.

Since its origins in the 1920s it has evolved significantly particularly with the growth of mathematical psychology in the 1950s, when the multi-attribute variant, Conjoint analysis was added to the tool set. Choice Modelling is believed by many to be the most accurate and general purpose tool currently available for making probabilistic predictions about human decision making behaviour. In addition it’s regarded as the most suitable method for estimating consumers’ willingness to pay for quality improvements in multiple dimensions. In fact the Nobel Prize for economics was awarded to a principal exponent of Choice Modelling theory; Daniel McFadden, in 2000.

4. Prediction Models

Finally, there was a burst of experimentation and advance in the development of models to predict the in-market volumes of new product launches, to assess the optimal price of goods and assess the potency and effectiveness of advertising campaigns.

Gabor and Granger introduced their price assessment model of the same name in the 1960s, followed by Van Westerndorf’s ‘price sensitivity model’, introduced in the 1976, enabling companies to explore the optimal pricing for their products.

Then, models like Eskin’s repeat decay curve, introduced in 1973, formed the basis for simulated test markets predicting the volumes that a new product or restaged product would achieve in the market. Nielsen’s BASES has emerged as the dominant player in this field, utilising and adapting the basic theoretical foundation.

And finally in the 1970s came the development of tools to assess the potential effectiveness of the campaign. Probably the greatest contribution in this field came in the Edisonian tradition from the experiments of Maurice Millward and Gordon Brown in isolating the factors that drove advertising effectiveness. Their legacy is that Millward Brown is now the global brand leader in advertising testing and tracking.

The Missed Opportunity of the Large Research Groups

I realise what I’m about to say is in danger of being a gross simplification of all the R&D, papers and advances submitted over the last 20-30 years by the largest global quantitative research agencies. However, it is my contention that given their dominant market position and significant resources at their disposal, they could have and should have introduced many more quantitative research advances than they have. Despite the large, publicly quoted research groups reporting large annual R&D spends, there certainly hasn’t been an Edisonian-like flow of innovation coming from these companies.

I would characterise the innovative performance of the large quantitative agencies over the last 20-30 years following the end of the 1970s Predictive Modelling period as poor. Whilst quantitative research was in a distinctly fallow period, qualitative research was entering its most fertile stage, building on the foundations laid in the 1960s and 1970s and busy applying psychology and social sciences to an exciting array of new research tools; focus groups, in-depth interviews, hypnosis, projectives, ethnography, semiotics and NLP. The history of qualitative research was eloquently articulated by Peter Cooper in his 2007 paper, ‘In search of Excellence: the evolution and future of Qualitative Research’, so I won’t repeat it here. However, we will return to these themes, with a twist, when we get to the New Research Pioneers in the next section.

The irony is that whilst quantitative innovation diminished and qualitative innovation surged, the money spent on quantitative research increased significantly during the period and now accounts for 83% of the almost $25 billion spent on market research1.

Despite the seeming financial success of quantitative research, I would contend that the industry could have grown significantly faster if it had continued the innovative, experimental ways of previous decades. As a comparator, whist the big quant agencies focused on consolidation and M&A, the Management Consulting industry grew many, many times faster during in the same period and now stands 10 times larger than the Market Research industry.

I’d suggest the lack of innovation from the large research groups has been due to three factors:

1. Innate Conservatism

The innate conservatism of the world’s large quantitative agencies is partly due to their dominant position, created largely through M&A (more of which below) but also due to encouraging large clients to buy their services as an ‘insurance policy’ against making the wrong decision. This positioning inevitably favoured safety over creativity, consistency over experimentalism and encouraged an excessive focus on the merits of normative data to defend the preservation of the status quo.

2. Fear of Failure

No one likes to fail but I contend that worse than the understandable human and corporate desire to avoid the failure inherent in constant experimentation, there is a ‘double-bind’ at work on the large quantitative giants of the research industry. When your role is to act as an insurance policy and reduce risk, it’s important to build a reputation for certainty and not mistakes or failure. The downside of this for the large quantitative agencies is it has encouraged a culture of conservatism and reluctance to change and innovate.

3. Focus on M&A

Perhaps because of their innate conservatism, made worse by their fear of undermining confidence in the certainty of their predictions, the large quantitative agencies have focussed the majority of their time, money and energy on mergers and acquisitions. In a presentation given to the MRS in 2005, the Chairman of Synovate estimated that over the previous 20 years, companies representing over 80% of the global research spend had changed hands at least once.

Many industries consolidate and market research is no exception. It may not be remarkable but it does generally indicate a saturated and mature market with single digit growth prospects at best. It is this I take issue with and can’t help feeling the focus on M&A and resulting slower growth rates is self-inflicted. The example of management consultancies suggests there is huge further growth potential for market research if we’re brave, curious and determined enough to recapture the experimental prowess of the early pioneers.

The New Research Pioneers

Figure 4
Figure 4 (graphics3.jpg)

Successful entrepreneurs do not wait until “the Muse kisses them” and gives them a “bright idea”: they go to work. Altogether they do not look for the “biggie,” the innovation that will “revolutionise the industry,” create a “billion-dollar business” or “make one rich over-night.” Those entrepreneurs who start out with the idea that they’ll make it big – and in a hurry – can be guaranteed failure. They are almost bound to do the wrong things. Any innovation that looks very big may turn out to be nothing but technical virtuosity, and innovation with modest intellectual pretensions; a McDonald’s, for instance, may turn into gigantic, highly profitable businesses.

The quote was from Peter Drucker’s book ‘Innovation and Entrepreneurship’ and it captures much of the pioneering, ‘just-do-it-and-see’ attitude of the ‘new research pioneers’ featured in this final section.

The last ten years, largely stimulated by the growth of the internet, has seen something akin to a Cambrian explosion of innovation in market research. Like many revolutions it has not come from the mainstream but from the fringes. It could be characterised as an off-to-online market research revolution but to my mind that fails to capture the full potential of the innovators and the innovations they are unleashing onto the research market.

This ‘research renaissance’ if you like, has already dramatically reduced the time and cost of capturing the same data as traditional, face-to-face and telephone fieldwork methods. It already accounts for over 20% of global market research spend and is still growing fast. Given that online research is around one-third cheaper than the methods it is replacing, this already means the research market has grown significantly more in real terms than the reported 6% value increase in 2007. In fact, I’d argue, that hidden in the figures of another year of modest growth is noticeably more value to the buyers of market research, who like the beneficiaries of the low cost airlines have benefitted from being able to do much more with the same budget.

But this brings me to what I see as the potentially more significant ‘research techniques revolution’ being pioneered on the fringes, largely by these same internet pioneers and which I believe will have a more significant impact on the growth and influence of market research in the long-term. And as Kevin Kelly, founder of Wired magazine said, “Significance precedes momentum”.

I would characterise the new techniques I see emerging, as seeking to capture the insight, inspiration and richness of qualitative research but on a scale large enough to provide a quantitative robustness to the output. In effect, these ‘quali-quant’ techniques massively raise the bar, delivering insightful, robust research solutions that are cheaper, faster and better than existing techniques. The only way to deliver on such an ambition is through massive amounts of experimentation, voluminous cycles of trial and error, hard work, harder knocks and the perseverance required to deliver a step-change in expectations.

This is by no means a definitive list but it is these new research pioneers that I would like to celebrate and congratulate on being prepared to take risks, challenge the status quo and fail from time to time, in their pursuit of meaningful innovation. There are many clients, like Jaroslav Cir at Unilever, Mark Whiting at Moët Hennessy, John Dimopoulos at Fosters, Sion Agami at P&G and Linda Grosjean at Roche, who are sharing and encouraging these developments but for the purposes of this paper I will focus on a necessarily brief summary of the people running the agencies at the heart of this renaissance and the key innovation they’re involved with [see the Rogue’s Gallery in figure 5.]. Since this paper is already long and my deadline for submitting it short, I will share the brief version but I would encourage you to check them all out as they represent an Aladdin’s cave of research innovation that might just deliver against your three wishes.

1. Alex Gofman and Howard Moskowitz - Moskowitz Jacobs.

Howard Moskowitz has been at the heart of many innovations and is a renowned experimenter, intellectual and creative. But it is his latest idea that I would like to focus on, captured in his book, Selling Blue Elephants: How to Make Great Products That People Want Before They Even Know They Want Them, co-authored with his colleague, Alex Gofman. This idea which raises Choice Modelling, mentioned earlier, to an art form is perhaps best expressed in the blurb from their publisher14:

Can you remember the world before the iPod? How about the world before chunky tomato sauce or brown mustard? Many of these products came about not through focus groups and polling, but rather through research and development labs and marketers developing the products they knew customers would want, before customers knew they wanted them. Today your customers can actually help you create your next product.

Rule Developing Experimentation (RDE) is a solution-oriented learning experience. RDE is the systematized process of designing, testing and modifying alternative ideas, packages, products, or services in a disciplined way so that the developer and marketer discover what appeals to the customer, even if the customer can't articulate the need, much less the solution. Read about best practices in the RDE from some of today's top companies: HP, Prego, Vlasic, MasterCard and others. Filled with real-life stories, this book will change the way people think about selling to their present and future customers.

2. David Penn - Conquest

David Penn, a popular and talented speaker at research conferences has been exploring how to turn the latest thinking in psychology into a scalable research tool. I’d like to focus on his latest tool, Metaphorix™ which is designed to capture emotional response to brands and advertising15.

It uses animated visualisations of metaphors to enable consumers to intuitively express what they feel about brands and advertising. It avoids the over-rationalising inherent in traditional approaches and provides real commercial insight hitherto denied to conventional research.

Figure 5. New Pioneers of Research

Figure 5
Figure 5 (graphics4.jpg)

3. Diane Hessan – Communispace

Diane Hessan has created the foremost online customer communities company, which engages customers 24/7 and hardwires the voice of your customers into a business. Interestingly the company deliberately avoids the term research company and prefers to call itself a customer feedback company, since they feel this is both liberating in experimenting how they can add value most and increase the size of the potential market they operate in. Testament to their success, the approaches pioneered by Communispace are now being copied widely around the world.

4. Emilie Labidoire - Repères

Emilie Labadoire and her colleagues at Repères gained certain notoriety in 2007 by establishing the first research agency within Second Life, where they experimented with conducting real research projects with the avatars therein and produced some extremely insightful reports on the back of it.

5. Fiona Blades and Steve Philips – Mesh Planning

Steve Philips is an extremely innovative research pioneer in his own right at Spring Research but together with Fiona Blades they have created a 360o tracking product, called ‘TROI’, utilising the SMS capability of mobile phones to allow people to text the adverts they notice and how they feel about them and utilise an online reporting system to provide clients with real-time, accurate feedback on their campaign. In part, they have achieved on a shoestring and with massive amounts of creativity what the IPA Touchpoints system is hard pressed to deliver with a massive budget and industry wide backing.

6. Fred Reichheld – Bain & Co.

Fred Reichheld’s Net Promoter Score which demonstrated the correlation between people’s stated likelihood to recommend a product or service with its likely growth or decline has been one of the ‘hot potato’ debates of market research for the last year or two. Despite attempts at a demolition job by Tim Keiningham of IPSOS, the simplicity and insight provided by NPS’s one meaningful number continues to be extremely compelling to CEOs around the world.

7. Han de Groot and Jan Willem Gerritsen - MetrixLab

Han de Groot and his partner, Jan Willem Gerritsen have been at the forefront of innovative online developments since 1999 when they started. Of particular note is their work on online advertising research where they have were the first to include attention based metrics in a pre-test, validate an online tool for eye tracking and show the effectiveness of online text based and video advertising16. MetrixLab has won numerous awards for their pioneering, innovative work.

8. John Pawle and Peter Cooper – QiQ International

John Pawle can certainly claim to be one of the founding fathers of the notion of quail-quant research and has worked closely with Peter Cooper translating the best of qualitative techniques into scalable online techniques. The work I’d like to highlight here is the tool they developed to quantitatively measure LoveMarks®, the Saatchi and Saatchi approach to brand equity assessing the degree of love and respect generated by the brand17. This is increasingly being used by multinationals to assess the health of their brands around the world.

9. Laurent Flores – CRM Metrix

Laurent Flores, describes himself as a customer-listening evangelist and has many research awards for doing just that to the benefit of clients. His company CRM Metrix was one of the very first to create a 24/7 online system to track mission-critical customer feedback related to market needs insights, brand affinity, and satisfaction. But like the other New Pioneers of Research, he is a constant innovator and it is his more recent work developing online techniques for generating ideas that I find really interesting. ‘BrandDelphi’, is an online application that helps identify new trends and insights by allowing consumers to enter their own ideas and rate those of others without all the trappings(and expense) of traditional offline focus groups18.

10. Mark Earls – Mark Earls Consulting

Mark Earls, is a very popular speaker on the circuit and was Planning Director for O&M for many years. But it is as author of, ‘Herd: How to Change Mass Behaviour by Harnessing Our True Nature’ that I include him here. The short but challenging version of his thesis is, ‘we’re much less individual than we think we are [and marketing assumes] and we’re much more influenced by what other people actually do, than we would care to admit [or marketers realise]’. And therein lies the potential for a completely new lens through which to view marketing and develop research tools which measure what people are doing rather than what they’re saying.

11. Nadhim Zahawi – YouGov

Nadhim Zahawi has led YouGov to being acclaimed as the UK’s most accurate opinion pollster as well as the most quoted. When the accuracy of internet research was being questioned in the early 2000’s, YouGov was busy proving that not only was their approach as accurate as the established offline pollsters but in fact it was more accurate.

YouGov have introduced a number of innovative products but for me it is their modern version of Dr Gallup’s revolution that puts them centre-stage among the New Research Pioneers [even if I wasn’t able to find a photo of Nadhim or his colleagues to put into the Rogues Gallery].

12. Ray Pointer and Pete Comley – Virtual Surveys

Ray Pointer and Pete Comley have always been innovative researchers but through their company Virtual Surveys, Ray has become a leading commentator on developments in Research 2.0 and Pete has been translating this leading edge thinking into real community and creative tools that are exciting clients19.

13. Dr. Robert Heath – University of Bath, School of Management

Dr. Robert Heath has for many years done pioneering work in exploring the processing of advertising at low levels of attention, and the role of emotion in advertising. His work, which has won numerous awards, aims to improve the understanding of how advertising works, as demonstrated by his latest paper with Paul Feldwick, ‘50 years Using the Wrong Model of Advertising’. This has commercial implications for the improvement of advertising effectiveness, and social implications for the exploitation of sensitive areas and vulnerable groups by advertising.

14. Shelley Zalis – OTX

Shelley Zalis has won countless awards for the research innovation introduced since the company started in 2000. OTX has always specialized in innovative, cutting-edge online technology but of particular note are its customized services specifically for the entertainment industry, in motion pictures, home entertainment, all forms of television, music, publishing, and online20.

The fact you may not have heard of all of these characters and agencies reminds me of author and commentator, William Gibson’s saying, “The future’s here… it’s just badly distributed”.

BrainJuicer Labs

BrainJuicer Labs is our attempt to embed a “perpetual hunt” for innovative research solutions into the very DNA of the company. Our company may still be small, relative to the largest global research agencies but a recent senor hire, involved with innovation at one of those agencies was shocked to discover the level of our Labs experimentation and innovation, saying, “Despite your size, you are probably doing more innovation than the whole of the company I’ve just come from and what you’re doing is certainly a lot more exciting.”

Our goal is to make the complex, simple. Anyone can make the simple complex but making the complex simple is a deceptively hard but worthwhile and rewarding task. And our Lab motto of course is: ‘The paradox of success is you need to embrace failure to achieve it’.

Without realising it, we have intuitively been practicing many of the Edisonian tenets outlined earlier. I’d like to say this was deliberate but unfortunately, I’m not a good enough student to have researched Edison’s ways in advance and the Innovate Like Edison book only came out at the end of last year. But it is hugely reassuring to know that by our own experimentation we’ve arrived in a similar place and if the output is a hundred thousandth of what Menlo Park produced we’d be delighted.

The BrainJuicer Labs innovation programme [see figure 7] outlines the four stages any project goes through, using a software development analogy. Even once a ‘Juicy Product’ is released, there will inevitably be improvements and future releases on this product. The fact there are 35 projects and around 70 people in the company, including Labs collaborators, is hopefully testament to our commitment to innovation. Here’s a little more about how we do it:

1. Align Goals with Passions

Nobody currently works in Labs fulltime; it is staffed on the basis of passion for a particular “hunt”. If there’s an idea someone’s really interested in, they pitch it to the BrainJuicer Labs board and more often than not, they get the time, funding and support they need to pursue it. They become the Labs Leader for the project, tasked with driving and guiding it all the way through our R&D, Alpha and Beta testing, in the hope it becomes a commercialised ‘Juicy Product’. This is akin to the 3M principle that a certain amount of staff time can be allocated to their ‘passion project’ and so far it seems to be working very well.

2. Cultivate Charismatic Optimism

Innovation is difficult, messy, non-linear and infuriating, so it’s easy to get downhearted when you’re trying and failing to get an idea to work. That’s where a culture which embraces failure as part of success, helps put set backs in positive context. And since so many of the BrainJuicer staff are involved in these projects, there’s plenty of support and sympathy on hand when it’s needed.

3. Seek Knowledge Relentlessly

This one feels easy, since researchers are naturally curious and just a little bit geeky and like nothing better than to explore a whole new area and learn something new. To that end, anybody can order any books they need from the company Amazon account and we have quite a library emerging from all the different interests and Labs projects. To share this knowledge with other staff and clients, we encourage people to summarise the books with the ultimate challenge to capture the book in a sentence. In addition we’re experimenting with our own online BrainJuicer Community21 for staff, clients and anyone outside BrainJuicer interested in research to facilitate dialogue and ideas.

4. Experiment Persistently

This is certainly something we’ve embraced in Labs and there has been no shirking of the hard, methodical, iterative loop of ‘generate, test and learn’ at the heart of all Lab projects. Here are just two examples from our published work:

  • FaceTrace™, Orlando Wood’s paper, which won ESOMAR Best Methodology in 2007. This beautifully simple and intuitive tool to measure emotions took two years, numerous dead-ends, 12 photo shoots and 6,500 photographs to create. We are currently undertaking a new series of experiments to create a version of the tool for use in assessing adverts or films as they play.Which of these faces best expresses how you feel about this advert? [See figure 6]

Figure 6
Figure 6 (graphics5.jpg)

  • Predictive Markets, which won ESOMAR Best Methodology in 2005. This challenging supposition that a large diverse crowd of people, buying and selling shares in ‘ideas’ will be as accurate but more discriminating than monadic concept has been the subject of a literally dozens of full scale experiments with hundreds of people each time, over the last five years to explore how, when and why it works. Like everything else we’ve experimented with, there have been many dead-ends and failures which have helped us create a tool that works consistently in all categories and markets and is now fully validated in its success. We are currently exploring a number of experiments to see how accurately the technique can predict market share of new or restaged products.

5. Pursue Rigorous Objectivity

Rigorous objectivity is also something that comes naturally to quantitative researchers. Large scale validation of any potential breakthrough is a given and goes hand-in-hand with the urge to experiment. To illustrate using the same two examples as in the section above:

  • FaceTrace™. For the original paper, we took six IPA award winning adverts and six non-award winning adverts in the same six categories to assess the tools’ ability to capture the potential of effective advertising and how well it correlated to in-market performance. Now a year on, we are about to embark on the ultimate validation of the technique using over 50 of the IPA award winning case studies with in market sales data to assess the correlation with return on investment.
  • Predictive Markets. Over the five years we’ve been experimenting with this technique, we have conducted 215 head-to-head comparisons with traditional monadic concept tests across 12 different categories, as well as more than 50 bettable public events (Big Brother, X factor, Euro 2008, elections etc), where we have placed a small bet to see if our crowd’s prediction can beat the bookies. Suffice to say, we have been making money, so perhaps we’ll end up in a different business to market research!

This is the spirit of BrainJuicer Labs and we hope it continues to produce successful research innovations for many years to come.

Figure 7

The BrainJuicer Labs R&D Programme – July 2008

Figure 7
Figure 7 (graphics6.jpg)


I hope the examples given and the stories shared have helped to convince you of my central premise that, ‘The paradox of success is you need to embrace failure to achieve it.’

Even this paper itself is an experiment to see how successful a long form medium might be, in conveying and convincing people of the paradox at the heart of successful innovation. The fact that only a small number of people typically read such papers would suggest this is an unlikely medium for the successful spread of the message. However, when it comes to experiments, you have to enter each with an open mind and if I’ve learned anything, it’s the most unlikely and unpromising of experiments that often provide the breakthrough. I hope it’s the case here.


Berkum, S. “The Myths of Innovation”, O’Reilly Media, Inc ISBN-10: 0-596-52705-5 (2007)

Cooper, P. “In search of Excellence: the evolution and future of Qualitative Research” ESOMAR (2007)

Dennett, D. “Darwin's Dangerous Idea: Evolution and the Meanings of Life” (Simon & Schuster; ISBN 0-684-82471-X (reprint edition 1996)

Dyson, J. “Against the Odds” An Autobiography, Cengage Learning, ISBN10 1587991705 (2003)

Earls, M. “Herd: How to Change Mass Behaviour by Harnessing Our True Nature” Wiley (2007)

Gelb, M.J. & Miller Caldicott, S. “Innovate Like Edison: The Success System of America’s Greatest Inventor”, Dutton, penguin Group (USA) Inc ISBN 978-0-525-95031-8 (2007)

Heath, R.G. & Feldwick, P. “50 years using the wrong model of advertising”. International Journal of Market Research 50 (1): 29-59 (2008)

Jones C. “Accidents May Happen: Fifty Inventions Discovered by Mistake”, Random House Children's Books ISBN-13: 9780385321624 (1996)

Kearon, J. “Innovation at a Crossroads”, Market Leader (1999)

Kearon, J. “Research Markets – a Fresh Approach to Concept Testing” ESOMAR (2005)

Kearon, J. “Predictive Markets - Is the Crowd Consistently Wise? (Two years and seven experiments on from the paper in 2005) MRS (2007)

Kearon, J. “Creative Sixers; Adding Inspiration to Innovation” ESOMAR Innovate (2006)

Moskowitz, H. Gofman, A. “Selling Blue Elephants: How to Make Great Products That People Want Before They Even Know They Want Them” Howard Moskowitz and Alex Gofman ■ Wharton School Publishing, (2007)

Reichheld, F., “The Ultimate Question: Driving good profits and true growth” Harvard Business School Publishing ISBN 1-59139-783-9

Simonton D.K. “Origins of Genius: Darwinian Perspectives on Creativity”, New York, NY, Oxford University Press (1999)

Wood, O. “Using Faces; Measuring Emotional Engagement for Early Stage Creative” ESOMAR (2007)

The Author

John Kearon is the founder and CEO of BrainJuicer Group plc, and Chief Juicer of BrainJuicer Labs.


Please note that all images are copyright of BrainJuicer.


See ESOMAR Global Market Research 2007, Date of publication:17.09.2007

ISBN: 928310209

2 See IBISWorld Global Management Consultants - Global Industry Report

25 Mar 2008, 65 pages, Industry Code: L6712-GL

3 See the full version of JK Rowling’s Harvard Commencement address

4 See ‘The Strange Story of Rubber’ on the Goodyear website

5 See Panati's Extraordinary Origins of Everyday Thing" ISBN 0-06-096419-7

6 See ‘Urban Legends, The Origin of Coca-Cola’

7 See Food Timeline for History of Shredded Wheat

8 See fascinating facts on History of Kelloggs Corn Flakes

9 See ‘ChemSoc Timeline, 1928, Fleming – Penicillin’

10 See Post-It Notes invention story, "Inventor of the Week: Art Fry and Stephen Silver". MIT.

11 See The Story Behind the Sony Walkman

12 See the Alcopop Origin Story in Encyclopaedia of Consumer Brands

13 See Wikipedia on Market Research, Statistics, Choice Modelling

14 See Selling Blue Elephants, creating great ideas just got easier

15 See ‘Hello and Welcome to Metaphorix

16 See MetrixLab Online Ad Campaign Evaluation

17 See QiQ International; Quantitative Rigour with Qualitative Insight

18 See CRM Metrix BrandDelphi

19 See Virtual Surveys, Research 2.0

20 See OTX Media and Entertainment Insights

21 See the online BrainJuicer Community

Content actions

Download module as:

PDF | EPUB (?)

What is an EPUB file?

EPUB is an electronic book format that can be read on a variety of mobile devices.

Downloading to a reading device

For detailed instructions on how to download this content's EPUB to your specific device, click the "(?)" link.

| More downloads ...

Add module to:

My Favorites (?)

'My Favorites' is a special kind of lens which you can use to bookmark modules and collections. 'My Favorites' can only be seen by you, and collections saved in 'My Favorites' can remember the last module you were on. You need an account to use 'My Favorites'.

| A lens I own (?)

Definition of a lens


A lens is a custom view of the content in the repository. You can think of it as a fancy kind of list that will let you see content through the eyes of organizations and people you trust.

What is in a lens?

Lens makers point to materials (modules and collections), creating a guide that includes their own comments and descriptive tags about the content.

Who can create a lens?

Any individual member, a community, or a respected organization.

What are tags? tag icon

Tags are descriptors added by lens makers to help label content, attaching a vocabulary that is meaningful in the context of the lens.

| External bookmarks