Ethics and Uncertainty

How much knowledge is required, in order to make a proper ethical judgement?

Assuming that consequences matter, it would obviously be useful to be able to reason about the consequences. This is typically a combination of inductive reasoning (what has happened when people have done this kind of thing in the past) and predictive reasoning (what is likely to happen when I do this in the future).

There are several difficulties here. The first is the problem of induction - to what extent can we expect the past to be a guide to the future, and how relevant is the available evidence to the current problem. The evidence doesn't speak for itself, it has to be interpreted.

For example, when Stephen Jay Gould was informed that he had a rare cancer of the abdomen, the medical literature indicated that the median survival for this type of cancer was only eight months. However, his statistical analysis of the range of possible outcomes led him to the conclusion that he had a good chance of finding himself at the favourable end of the range, and in fact he lived for another twenty years until an unrelated cancer got him.

The second difficulty is that we don't know enough. We are innovating faster than we can research the effects. And longer term consequences are harder to predict than short-term consequences: even if we assume an unchanging environment, we usually don't have as much hard data about longer-term consequences.

For example, a clinical trial of a drug may tell us what happens when people take the drug for six months. But it will take a lot longer before we have a clear picture of what happens when people continue to take the drug for the rest of their lives. Especially when taken alongside other drugs.

This might suggest that we should be more cautious about actions with long-term consequences. But that is certainly not an excuse for inaction or procrastination. One tactic of Climate Sceptics is to argue that the smallest inaccuracy in any scientific projection of climate change invalidates both the truth of climate science and the need for action. But that's not the point. Gould's abdominal cancer didn't kill him - but only because he took action to improve his prognosis. @Alexandria Ocasio-Cortez has recently started using the term Climate Delayers for those who find excuses for delaying action on climate change.

The third difficulty is that knowledge itself comes packaged in various disciplines or discourses. Medical ethics is dependent upon specialist medical knowledge, and technology ethics is dependent upon specialist technical knowledge. However, it would be wrong to judge ethical issues exclusively on the basis of this technical knowledge, and other kinds of knowledge (social, cultural or whatever) must also be given a voice. This probably entails some degree of cognitive diversity. Will Crouch also points out the uncertainty of predicting the values and preferences of future stakeholders.

The fourth difficulty is that there could always be more knowledge. This raises the question as to whether it is responsible to go ahead on the basis of our current knowledge, and how we can build in mechanisms to make future changes when more knowledge becomes available. Research may sometimes be a moral duty, as Tannert et al argue, but it cannot be an infinite duty.

The question of adequacy of knowledge is itself an ethical question. One of the classic examples in Moral Philosophy concerns a ship owner who sends a ship to sea without bothering to check whether the ship was sea-worthy. Some might argue that the ship owner cannot be held responsible for the deaths of the sailors, because he didn't actually know that the ship would sink. However, most people would see the ship owner having a moral duty of diligence, and would regard him as accountable for neglecting this duty.

But how can we know if we have enough knowledge? This raises the question of the "known unknowns" and "unknown unknowns", which is sometimes used with a shrug to imply that noone can be held responsible for the unknown unknowns.

(And who is we? J. Nathan Matias argues that the obligation to experiment is not limited to the creators of an artefact, but may extend to other interested parties.)

The French psychoanalyst Jacques Lacan was interested in the opposition between impulsiveness and procrastination, and talks about three phases of decision-making: the instant of seeing (recognizing that some situation exists that calls for a decision), the time for understanding (assembling and analysing the options), and the moment to conclude (the final choice).

The purpose of Responsibility by Design is not just to prevent bad or dangerous consequences, but to promote good and socially useful consequences. The result of applying Responsibility by Design should not be reduced innovation, but better and more responsible innovation. The time for understanding should not be dragged on forever, there should always be a moment to conclude.

---

Matthew Cantor, Could 'climate delayer' become the political epithet of our times? (The Guardian, 1 March 2019)

Will Crouch, Practical Ethics Given Moral Uncertainty (Oxford University, 30 January 2012)

Stephen Jay Gould, The Median Isn't the Message" (Discover 6, June 1985) pp 40–42.

J. Nathan Matias, The Obligation To Experiment (Medium, 12 December 2016)

Alex Matthews-King, Humanity producing potentially harmful chemicals faster than they can test their effects, experts warn (Independent, 27 February 2019)

Christof Tannert, Horst-Dietrich Elvers and Burkhard Jandrig, The ethics of uncertainty. In the light of possible dangers, research becomes a moral duty (EMBO Rep. 8(10) October 2007) pp 892–896

Stanford Encyclopedia of Philosophy: Consequentialism, The Problem of Induction

Wikipedia: There are known knowns 

The ship-owner example can be found in an essay called "The Ethics of Belief" (1877) by W.K. Clifford, in which he states that "it is wrong always, everywhere, and for anyone, to believe anything upon insufficient evidence".

I describe Lacan's model of time in my book on Organizational Intelligence (Leanpub 2012)

Related posts: Ethics and Intelligence (April 2010), Practical Ethics (June 2018), Big Data and Organizational Intelligence (November 2018)

updated 11 March 2019

On the true nature of knowledge

@pickover suggests that these two books, in theory, contain the sum total of all human knowledge. "The Joy of Logic", he remarks (via @DavidFCox).

What They Teach You At Harvard Business School
What They Don't Teach You At Harvard Business School

Why is this wrong? Because knowledge doesn't follow the laws of elementary arithmetic. Adding two lots of knowledge together doesn't give you twice as much knowledge. (Does anyone really think that teaching children creationism as well as evolution will double their education?)

Knowledge is like light. When you add two light beams together, you may sometimes get more light. But you may also get puzzling patches of darkness. This is called interference. In high-school physics we learn that this is because light is a wave. If the two waves are out of phase, they cancel each other out.

(Curiously, uncertainty is also like light. When you add two pieces of uncertainty together, you may get less uncertainty. This is called hedging. Works best when the uncertainty is out of phase.)


Obviously these two books are out of phase.

---

Forget the old joke about the two 'What teach you/don't teach you at Harvard Business School' books comprising all human knowledge

These two books comprise the entire universe pic.twitter.com/3QcZn3FcVk

— davidallengreen (@davidallengreen) May 13, 2021

On The Map
Off The Map

 

Related posts

Does Big Data Release Information Energy? (April 2014)

The Wisdom of the Tomato

Various people have tweeted the following aphorism.

Knowledge is knowing a tomato is a fruit. Wisdom is knowing not to put it in a fruit salad.

Please permit me to quibble with this aphorism. Classifying tomatoes as fruit is merely information. This classification is supported by data, such as the observation that the tomato contains its own seeds. Knowing not to put it into a fruit salad is a culinary best practice, based on a series of social conventions about the proper constitution of fruit salad and its place within a meal. So this is knowledge, or what is sometimes called received wisdom. However, innovation often involves disobeying social conventions and surprising those who rely excessively upon received wisdom. For example, how did chefs discover that it was okay to put flower petals into salads (next practice)? So courage is knowing that you are not supposed to put tomatoes into fruit salad, but doing it anyway. And real wisdom is not inflicting such gross culinary experiments on the wrong people at the wrong time in the wrong way.

---

Wikipedia attributes this aphorism to the Irish rugby captain Brian O'Driscoll. Various interpretations can be found in the comments to Brendan Cole's blog What did BOD mean? (Feb 2009)

On the Unbelievable Truth (Series 10 Episode 5), the @RealDMitchell rants about whether a tomato is a fruit or a vegetable. He claims that the US Government taxes tomatoes as vegetables, and regards this as more authoritative than mere science.

See also my post Co-Production of Data and Knowledge (Nov 2012), Alternative to the DIKW Pyramid (Feb 2020)

Updated 29 January 2013. Link added October 2025.

Ethics and Intelligence

@flowchainsensei (Bob Marshall) argues that All Executives are Unethical (pdf).

More precisely, he argues that it is unethical to believe things without proper evidence. (He is particularly interested in beliefs about product and software development, but the argument applies more generally.)

As far as I can see, there are three steps in this argument.

1. People are ethically responsible for their beliefs. (According to Bob, this was the basis for a controversial paper presented to the Metaphysical Society by William Kingdon Clifford in 1876.)

2. An unfounded belief is unethical.

3. A person who holds unfounded beliefs is unethical.


Let's look at step 1 first. This appears to entail an ethical obligation to subject one's beliefs to some kind of "due diligence". However, most of our beliefs are based, not on evidence that we have personally collected and analysed, but at least partly on evidence that has been filtered through other sources. We may have reasons to trust certain sources more than others, but if it is unethical to believe things without proper evidence, it would also surely be unethical to trust things without proper evidence. We may accept an ethical obligation to subject our beliefs to "due diligence", but this is normally a collective obligation rather than an individual obligation.

Step 2 asserts that any failure to ground beliefs in proper evidence is an ethical failure. People are rightly held accountable for failing to act in certain circumstances (for example failing to save someone from drowning), but ethical censure generally assumes both awareness (knowing that someone needed rescue) and capability (being able to swim). So the problem with Step 2 is that the more complex the beliefs are, the greater the intellectual power (intelligence) that is required to appreciate and thoroughly investigate these beliefs. If the management team isn't individually or collectively intelligent enough to understand what proper evidence would look like, then believing things without proper evidence is a consequence of insufficient intelligence.

Does being stupid count as an ethical failure? (Being deliberately or avoidably stupid might, but most instances of stupidity are not deliberate.) Appointing people and teams who don't have enough intelligence might be unethical, but only if the appointment was deliberate or avoidable, and so on along the responsibility chain until we can find someone who should have known better.

Step 3 assumes that we can categorize people as ethical or unethical based on incidence of ethical or unethical behaviour. Once we have a hard-and-fast concept of sin, then we can define a sinner as a person who has committed (and not yet purged) at least one sin. The trouble with this is that if we are all sinners, the category of "sinner" ceases to have much value except for the purposes of hellfire rhetoric. Labelling all executives as unethical (and why stop at executives, by the way) becomes merely a rhetorical gesture.

---

So where does this leave the virtues of diligence, responsibility and probity? Firstly, I hold that these are collective virtues - executives display moral character in a particular organizational setting, and we may not know how their ethics would stand up in a different setting.

Secondly, I think character and intelligence are distinct virtues. We should not automatically suppose that intelligent people are more ethical than less intelligent people, and therefore we should not define "ethical" to mean something that only very intelligent or highly educated people can comply with.

Thirdly, there is a widespread belief (especially among consultants) in the value of knowledge (although I don't know exactly what would count as proper evidence for this belief - if executives are unethical, I dread to think where this leaves consultants). If we define knowledge as justified true belief, then knowledge is degraded to the extent that it is unjustified or untrue, or for that matter disbelieved. If it is unethical to believe something without proper evidence, it may sometimes also be unethical to disbelieve something. Sometimes excessive scepticism shades into cynicism and negativity, and maybe this can be just as unethical as unjustified optimism.

 

---

Related posts: Intelligence and Governance (February 2013), Ethics and Uncertainty (March 2019)

Knowledge Claims

@JDeragon blogs about the emergence of a "know" profile. He identifies four types of knowledge - intellectual, social, creative and spiritual - and advocates the construction of individual profiles that express our individual "knowledge inventory" across these four types of knowledge.

The metaphor of "knowledge inventory" is based on his assertion that people are containers of knowledge. But what if people are NOT "containers" of knowledge, asks @EskoKilpi, who argues that one of the main challenges for knowledge management is bridging the gap between knowing and acting.

I fully concur with Esko's objection to the "container" metaphor, and I agree that the relationship between knowing and acting is important - in fact it's a critical connection in my model of organizational intelligence. However, I think we have to be careful not to imply that the gap between knowing and acting can ever be completely closed. There is always a need to act under conditions of uncertainty.

Even if we were willing to regard knowledge-as-content, Jay Deragon acknowledges that this knowledge would need to be measured and vetted over time. So the best that we could possibly expect from a set of knowledge profiles is a collection of knowledge claims, together with some information that would allow us to evaluate a given claim. Does this person really know everything about project management? Does this medical researcher really know that this procedure is safe and effective?

When I pointed out that I can claim knowledge about all sorts of things, and asked who is the best judge of how much I really know, @oscarberg replied that "real life is the best judge".

But since we don't have a reliable way of interrogating real life in real-time, we must surely treat all knowledge-claims with caution.

Puzzles and Mysteries

For my second post inspired by Malcolm Gladwell's latest book "What the Dog Saw, and Other Adventures", I want to turn to the chapter "Open Secrets" (originally published in the New Yorker, Jan 2007).

In his analysis of different kinds of intelligence, Gladwell picks up the distinction between puzzles and mysteries originally proposed by Gregory Treverton. (See Curtis Frye's review of his 2003 book Reshaping National Intelligence for an Age of Information. See also Risks and Riddles, published in the Smithsonian Magazine, June 2007. And see my post on Making Intelligence Relevant.)

A puzzle is characterized by having a definite answer, if we can only find it. A puzzle is difficult only because we don't have enough information. For example, Gladwell and Treverton classify the question "Where is Osama Bin Laden?" as a puzzle. If we knew exactly where Bin Laden was, then this would cease to be a puzzle at all. So the purpose of intelligence here is to get relevant pieces of information that help narrow down the field of search.

A mystery is characterized by ambiguity and uncertainty. For example, the question "What is Osama Bin Laden up to?". There is perhaps no shortage of information here, there may even be too much information, the difficulty is in interpreting the information correctly.

Gladwell uses the distinction to discuss the Enron case. He argues against the popular view that Enron management concealed their dealings, and points out that the information was freely available to those that took the trouble to wade through the documents. In contrast to the Watergate case, which was only revealed because an insider (the famous "Deep Throat") leaked information to Woodward and Bernstein, journalists investigating the Enron case simply downloaded the information they needed from the Enron website. Indeed, several years before Enron fell, a bunch of MBA students had carried out a pretty accurate analysis, based merely on the published accounts. In Gladwell's opinion, Enron therefore counts as a mystery rather than a puzzle.

This is consistent with an assertion made by Harold Wilensky in his 1967 book Organizational Intelligence (which Gladwell has cited elsewhere), that a sophisticated reporter working with open sources may achieve more than an agent working with top-secret information. Wilensky highlights the distorting effects a doctrine of secrecy can have on intelligence: one example in Wilensky's book concerns the possible consequences of an American invasion of Cuba, where reporters read the situation more accurately than the CIA experts.

Dishonest people can create puzzles simply by withholding important information. Enron executives went to prison because their conduct was judged dishonest. Gladwell agrees that Enron was reckless and incompetent, but defends the company and its executives against the charge of concealment. The state of Enron's finances was too complicated even for its own executives to understand; we might imagine that some of the executives consciously took advantage of this complexity, but we could equally imagine that this was a situation that Enron blundered into without any deliberate strategy. Hence the mystery,

Characterizing Enron as a mystery also goes some way to explaining why the auditors were useless in detecting the fraud - if it was indeed a fraud. The audit process is designed to spot errors and omissions in the financial accounts, which might indicate dodgy dealings somewhere in the organization. The audit process is not designed to spot excessive complexity or risk, and auditors do not generally practice the kind of ratio analysis that the MBA students used. (Auditing is therefore one of those "best practices" whose flaws are exposed by complexity.)

Another lens through which the Enron accounts could reasonably have been viewed was the taxation lens. The fact that Enron wasn't paying much corporation tax (in several years it paid no income tax at all) might have been seen as an important clue to its lack of real profitability. However, those who wanted to believe in Enron's profitability could easily convince themselves that the low level of tax payments represented clever tax avoidance - in other words, interpreting it as evidence of the smartness of the accountants and/or the stupidity of the tax authorities. (Thus the accountancy lens was used to discredit alternative lenses that might have revealed alternative truths.)

Another way of thinking about the difference between puzzles and mysteries is that puzzles are about people (deliberate conspiracies) while mysteries are about systems. As Gladwell tells the story, Enron wasn't about a handful of bad people misleading everyone else, it was about a system that led everyone astray. Working out a mystery is not a question of collecting more information, but about finding a frame or lens for systematic analysis, to make sense of the information we already have.

Does it make sense to divide intelligence problems into puzzles and mysteries, as Treverton and Gladwell do? I'm not convinced there is a simple either/or, but I'm not sure that's what Treverton and Gladwell are claiming anyway. I  think what is important here is not to identify which problems count as puzzles and which ones count as mysteries, but to acknowledge that at least some problems do count as mysteries in Treverton's terminology, and therefore we need an intelligence capability that helps us to make sense of too much information, and not rely solely on an intelligence capability that merely gathers more information in the hope of resolving something. Examples of this need can be found both in national security and in business.

In terms of organizational intelligence, this means achieving a good balance between two capabilities - the information gathering capability (Perception) and the analysis capability (Sense-Making) - and linking effectively into the remaining capabilities (Decision, Action, Learning). Sometimes merely collecting more information doesn't help solve the problem, especially if we don't have the capacity to interpret the information we already have, or if the new information merely provides an excuse for further procrastination.

(This is similar to the point some of us were discussing recently on Twitter: whether it is always good to produce more ideas, or whether it is sometimes possible to have too many ideas, especially if you don't have the capacity to use them effectively.)

Was Enron intelligent? Enron certainly did some clever and innovative things, but the spectacular failure of Enron suggests that there were some flaws in the thinking. Here are two suggestions. Firstly, a collective failure within Enron to appreciate the scale of its exposure to risk indicates a weakness in sense-making - an insufficiently robust way of seeing beyond the complexity of the accounts and understanding the true financial state of the company. And secondly, a collective refusal to learn from the voices of doubt coming from external critics. When a company is convinced by its own cleverness, this conviction can become a barrier to learning, and therefore a limitation of true intelligence.

---

Other examples of the puzzle/mystery dichotomy

• Bob Rosner (ABC News Feb 2007) suggests that puzzles are left-brain and mysteries are right-brain
• Larry Ferlazzo suggests that puzzles are data-driven and mysteries are data-informed
• Security blogger Gunnar Peterson suggests that Authorization is a Puzzle, Authentication is a Mystery
• The Moonrider blog looks at accidents versus collisions in Motorcycle Safety
  
• Brett Miller quotes Gary Kasparov on Life versus Chess: Uncertainty is far more challenging
• In the 1950s, Frederik Buytendijk described men as puzzles and women as mysteries (via Annemarie Mol).
  

Related posts: Connecting the Dots, Explaining Enron, Making Intelligence Relevant

Linking Facial Expressions

Two studies about facial expressions have been reported by the BBC in the past few weeks.

Facial expressions 'not global' (14 August 2009). In research carried out by a team from Glasgow University, East Asian observers found it more difficult to distinguish some facial expressions. (Findings published in Current Biology journal.)

Delinquents 'misinterpret anger' (19 September 2009). A Japanese study of young offenders found they often misread facial expressions. (Findings published in Child and Adolescent Psychiatry and Mental Health journal.)

I have not read the academic papers, but it looks as if there might be an interesting link between the two studies. There are some missing perceptions in some people (perhaps by culture and/or personality type), and these are linked to patterns of behaviour. The Japanese researchers are looking at personality type, while the European researchers are looking at cultural differences.

But how do such links ever get identified, especially as it is perfectly within the bounds of possibility that there is nobody who reads both of these journals? I only spotted it myself, because I read the second story and recalled having read something similar not long before, and because I was able to find my way back to the first story.

The first general point to pay attention to here is the process of memory retrieval, in this case involving a collaboration between my brain and a simple internet search.

The second general point is about the fragmentation of knowledge and the "architecture" of joined-up research. How do such accidental links influence not only what we happen to know, but also what becomes available to be known?


Related post: Affective Computing (March 2019)

Hard science

Found an extraordinary exam question in a post called GCSEs are dumbed down and getting worse, by Cabalamat, taken from an actual physics exam (Edexcel GCSE Physics P1b reference 5010 taken on 9 November 2006) (via Ben Goldacre).

Our Moon seems to 'disappear' during an eclipse. Some people say this is because an old lady covers the Moon with her cloak. She does this so that thieves cannot steal the shiny coins on the surface.

Which of these would help scientists to prove or disprove this idea?

A - collect evidence from people who believe the lady sees the thieves
B - shout to the lady that the thieves are coming
C - send a probe to the Moon to search for coins
D - look for fingerprints

I have read this question several times, and I am still unsure what answer they are looking for.

A - Well, this is exactly the kind of thing that social scientists would probably do. The question doesn't specify what kind of scientists it is talking about.

B - Well, this is a good experimental approach. If shouting affected the outcome, and if shouting about thieves produced a significantly different outcome to shouting about other things, then this would be good evidence in support of the hypothesis. However, if shouting didn't affect the outcome, this wouldn't help to disprove the hypotheses because there is a vacuum between the Earth and the Moon and sound doesn't carry in a vacuum. The old lady might have cybertronic ears, but then again she might be deaf.

C - Finding or not finding coins doesn't really help us much. If there are coins, it could mean that the old lady has outwitted the thieves, or that the thieves thought it would be unlucky to take all the coins, or that there aren't any thieves. If there are no coins, it could mean we are looking in the wrong place, or it is the wrong time of the month, or Fred Goodwin's got them.

D - Fingerprints. Same as coins. By the way, are we looking for fingerprints on the coins, or fingerprints on the cloak?

I suspect that any child who really understands science and the scientific method will waste more time on this question than a child who hasn't a clue. So this isn't just dumbing down, it is levelling down.

Thinking with the Majority

A.A. Milne "wrote somewhere once that the third-rate mind was only happy when it was thinking with the majority, the second-rate mind was only happy when it was thinking with the minority, and the first-rate mind was only happy when it was thinking". (War with Honour)

I wrote somewhere once that "thinking with the majority" is an excellent description of Google.

'The suggested improvements (in Google) are just great for those people who want to ask the same questions as everyone else, and get the same answers. Google rankings already depend on the clicks of previous websurfers, and this dependency will become more sophisticated. Google will therefore support, with ever-greater efficiency and effectiveness, an intellectual activity characterized by A.A. Milne (author of Winnie-The-Pooh) as "Thinking with the Majority". '

And as Titus-Armand points out, it is also a good description for

'reliance upon authority in which the “authority figure” is represented by the entire population rather than a single individual or a particular group'.

What about thinking with the minority? There is a popular meme known as "thinking the unthinkable", and I think this is what the third-class mind supposes the second-class mind to be doing.

If you are one of those who are happiest when thinking, please subscribe to this blog and also follow me on Twitter.

---

Update: within a few minutes of this item's being syndicated on Twitter, Al Chou reminds me of a related quote: "A great many people think they are thinking when they are merely rearranging their prejudices." Was it Churchill or William James? The authority of the majority (aka Google) prefers the latter; who am I to argue?

Negative Evidence

@snowded blogs on Negative evidence and the village idiot syndrome


People who should know better (so-called scientists) make this problem worse by using the phrase "no scientific evidence". For example "no scientific evidence that eating infected meat carries any risk to humans" or "no scientific evidence that mobile phones cause headaches".

This creates the impression that there may actually be lots of evidence, but we can safely ignore it because it hasn't been collected or approved by somebody in a white coat.

Just as some kinds of evidence are inadmissible in a court of law, so some kinds of evidence are inadmissible in a scientific journal. Among other things, this leads to publication bias, where people perform calculations based only on the data that have passed through some publication filter, which is then systematically incomplete.

See my comment to Science isn't about Checklists

Learning from Experience

Under the heading Big Fish Eat Little Fish, Dave Snowden posts a series of photos, telling a visual story with a dramatic conclusion. (Go on, look at the story, I can wait here until you come back.)

An obvious lesson to draw from this story is one about learning from experience.

However, there is a further twist: the final photo in the series turns out to be faked. (There is further explanation of this on Snopes.)

In addition to learning from experience, another lesson we could draw from this story is to be a little wary of situations whose narrative structure is too good. (When there are forces within a story that make us so want it to be true, maybe that's the time to switch logical levels.)

But of course, even if the story is part-fiction, that doesn't stop us learning from it. We do need to learn from experience, and experience includes fiction. The story is therefore True (at some level) because it is Relevant and Meaningful.

Dead Cert

BBC Radio Four broadcast an excellent programme on political doubt and certainty last night.

"Doubt seems a dangerous thing in politics. If possible, you don't admit it; not about your values, nor your analysis, nor the policies that will magically bring about the change that you are certain is needed. Confidence, by contrast, thrives: confidence in the power of our own analysis, of who is to blame and why, the strident confidence of politicians or business people in their preferred remedies. In this edition of Analysis, Michael Blastland asks whether these common assumptions might actually have their own dangers."

The programme will be repeated on Sunday 9th November at 21.30 GMT; podcast and transcript are available from the programme website for limited period only.

I shall post a review later.

You don't have to be smart to search here ...

... but it helps. 

I'm prompted to write this post by a throwaway remark from David McCoy, in a post on election statistics: "You don’t have to be smart to search nowadays - all you have to do is enter the key snippet."

Ah, but how do you find the key snippet? 

My son had a school essay to write comparing two films, so we thought it would be worth looking on the internet to find some analysis. But if you just search for the names of the films, you just get endless cinema listings and DVD sales, plus a few fairly superficial newspaper cuttings. 

So we tried another tack. Who are the key figures (film theory, media studies, sociology) that might be name-dropped in a serious essay? 

Let's start with Lacan. When we added "Lacan" to the name of one of the films, the search engine suddenly unearthed an entirely different set of web pages, including a bunch of blogs apparently created as part of a high school project (sixth-form) and talking about a set of related films including the two we were interested in. Could we have found these blogs any other way? 

Okay I admit it, my son hasn't read Lacan, hadn't even heard of him, but he had a bit of parental help. The point I'm making here is that sometimes the more knowledge you can put into the search, the more useful the results. 

Even Microsoft sometimes misses important stuff when it searches the internet - for example when checking a brand name. See my post on Google and Longhorn.

Internet search looks rather like a P v NP problem. It's fine for checking unoriginality: for example, if the teacher suspects a student of plagiarism, she can put a suspiciously well-phrased sentence into an internet search engine and confirm that the sentence is not original. It is also fine for finding well-structured material: if you want to check Missouri voting statistics, you can probably find something relevant. (See Missouri loses bellwether status, November 2008.) 

But if you want to find an unusual thought, you will have to find an unusual combination of search terms. You do have to be smart to search here.

 

---

Update March 2023

What I described in this post is now known as prompt engineering. The importance of this has become much more apparent since the emergence of AI tools such as Dall-E and ChatGPT. See James Bridle, The Stupidity of AI (Guardian, 16 March 2023). (The Wikipedia entry on prompt engineering was created in October 2021.)

Meanwhile while there are now automated tools to help teachers detect basic plagiarism, these are not much use against students who try to cheat using AI tools. See my post on Reasoning with the majority - chatGPT (January 2023).

Which develops a view of the limitations of Google I have talked about elsewhere: Thinking with the majority (March 2009)

Confirmation Bias

Adam Shostack has a couple of posts on Confirmation Bias. I've added some comments on Adam's blog; here's a digest of the discussion.

Things Only An Astrologist Could Believe

Adam picks up an astrological analysis of a recent action by Google: apparently the timing of the Chrome release was astrologically auspicious.

Vedic Astrologer: "Such a choice of excellent Muhurta with Chrome release time may be coincidental, but it makes us strongly believe that Google may not have hesitated to utilize the valuable knowledge available in Vedic Astrology in decision making."

Adam: "This is a beautiful example of confirmation bias at work. Confirmation bias is when you believe something (say, Vedic astrology) and go looking for confirmation. This doesn't advance your knowledge in any way. You need to look for contradictory evidence. For example, if you think Google is using Vedic astrology, they have a decade of product launches with some obvious successes. Test the idea. I strongly believe that you haven't."

Myself: "What our Vedic friend is actually telling us is that Google "may not have hesitated" in its use of Vedic astrology. To be honest, I also find it hard to believe that Google executives sat around dithering about whether to use Vedic astrology or not."

In further comments, the Vedic astrologer argues that the astrological method is no different from other forms of observational science, using the scientific method, which requires the prediction of future results.

• Hypothesis: The sun comes up every 24 hrs.
• Method: I will time when the sun crosses the horizon.
• Results: I successfully predicted 50 sunrises with a 100% degree of accuracy. This is further evidence that my hypothesis is correct.
• Caveat: Although, I note that since 24 hrs is the period between sunrises by definition of a day, this is circular.

Actually, the statement that the sun rises exactly in 24 hour intervals is only believable if you live near the equator and you know nothing about astronomy, or if you adopt a solar method for measuring the length of an hour.

What confuses me about the hypothesis posed by our Vedic friend is whether he is trying to predict the decision-making behaviour of Google executives or the successful outcome of their decisions. Even if Google executives are making auspicious decisions, this could be "explained" either by the fact that they are employing the services of an astrologer, or by the fact that Google happens to have good (= astrologically blessed) executives. Or something.

(See my post Does Fortune-Telling Work?)

More on Confirmation Bias

According to an old article by Michael Shermer in the Scientific American [The Political Brain, June 2006], "a recent brain-imaging study shows that our political predilections are a product of unconscious confirmation bias". Devan Desai concludes that "hardcore left-wing and hardcore right-wing folks don’t process new data".

When I first read that line about "hardcore left-wing and hardcore right-wing folks" quoted in Adam's blog I assumed it was talking about serious extremists - communists and neoNazis. Turns out it was just looking at people with strong Democrat or Republican affiliation. Maybe any party affiliation at all seems pretty hardcore to some people.

As far as I can see, the study only actually looked at people with strong political opinions, and didn't compare them with any control group. Like, er, the middle-of-the-road folks who fund and write up this kind of research.

I wonder whether anyone would get research funding or wide publicity for exploring the converse hypothesis - that people with strong political opinions are actually relatively open-minded, and that the people who have the most entrenched opinions are the bureaucrats who staff the research funding bodies and the people who write popular articles for Scientific American.

(Of course I'm jumping to conclusions myself here, that's what bloggers do isn't it?)

I'm not saying I believe that bigots are more open-minded than wishy-washy middle-of-the-roaders. I'm just saying we need to be mistrustful of studies that are designed to confirm the prejudices of the researchers, and suspicious of people who latch onto these studies to prove a point. The problem is that there may be confirmation bias built into the way these kind of pseudo-scientific studies are funded, organized and then publicized. Not surprising then if "the FMRI findings merely demonstrate what many of us already knew".

As director of the Skeptics Society, Michael Shermer latches onto a study showing that people are biased. Shermer himself has a particular set of bugbears, including evolutionary psychology, alien abductions, prayer and healing, and alternative medicine. Are we really to imagine that he approaches any of these topics with a truly open mind? And why should he anyway? The rest of us don't.

Peer Review in the Dock

Tonight's Science programme on BBC Radio 4 was critical of the peer review process, in which scientific articles are filtered for publication according to the comments of other researchers in the same field. [Peer Review in the Dock, 4 August 2008]

The purpose of peer review is to give us confidence in the quality of published scientific research. Like many other social institutions, it has well-known weaknesses as well as strengths. [BBC News, Science will stick with peer review]

I have often been asked to provide peer reviews on articles for journals and conferences. Sometimes I find I know much more about the subject of the article than the authors, or at least some aspects of the subject. Even when my knowledge is less, I can usually find some areas of weakness or confusion in the article, demanding (in my opinion) either a significant re-write or complete rejection.

Having gone to the trouble to provide these reviews, I used to be shocked when I discovered that papers sometimes slipped through to publication without the identified flaws being adequately corrected. Experienced authors (or their supervisors) know how to game the system, and most journals and conferences simply don't have the resources to prevent these games. Some years ago I wrote a critique of this process and identified a number of negative patterns.

The BBC programme this evening identified several more, including the famous institution bias and the publication bias. The latter is particularly important for research that involves sophisticated statistics (such as medical research), because if only publishable data are included in the analysis, then the publication criteria may themselves distort the findings. The publication bias also affects the opinions of so-called experts, whose assumptions will have been reinforced by the papers they have read.

---

Related post: Trahison des Clercs - AntiPatterns of Peer Review (July 2004)

Update - Further links

Liam Kofi Bright and Remco Heesen, The Perfect Time to Reform Peer Review (BSPS 2022)

Remco Heesen and Liam Kofi Bright, Is Peer Review a Good Idea? (British Journal for the Philosophy of Science 2021) 

Memory and the Law

Rebecca Fordham writes:

"Many experts are challenging the view that eyewitnesses recounting what they saw is the best way of tapping their memory. Some think brain scans could be the way forward." [Memory Mixup, BBC News Magazine, 17 June 2008]

We already have technology that is supposed to detect discrepancies between what the witness remembers and what the witness says - it's called a polygraph or lie detector. Now we apparently need another technology that detects discrepancies between what the witness consciously remembers and what is buried in the witness's unconscious.

The lie detector has been controversial ever since its invention, and features in a Chesterton story called "The Mistake of the Machine". (Of course it is not the machine that makes the mistake, as Chesterton's hero Father Brown points out, but the people using the machine who misinterpret its output.)

• Chesterton and Friends: Lie Detectors
• David Wallace-Wells: The Big Lie (Washington Monthly, April 2007), via AntiPolygraph.org

Of course humans sometimes lie, and sometimes this can be detected by the polygraph, but that doesn't make the polygraph an instrument of truth. (For that matter, people sometimes blurt out secrets under the influence of alcohol or torture, or get artistic inspiration under the influence of mind-bending drugs, but none of these are reliable instruments of truth either.)

And human memory is sometimes unreliable, but that doesn't make the brain scan an instrument of truth either. Constructing evidence from the unconscious contents of a brain is no more reliable than constructing history from an archaeological sift through a mediaeval rubbish tip. It may be possible, and may yield some intriguing results, but the results are always speculative and uncertain.

Meanwhile, our "common sense" understanding of the brain and its contents is probably less accurate and less coherent than our understanding of mediaeval waste disposal. That's why psychoanalysts make more money than archaeologists. They do, don't they?

Update

"India has become the first country to convict someone of a crime relying on evidence from this controversial machine." [Source: New York Times, via Bruce Schneier]


Related posts: The Dashboard Never Lies (February 2020), Lie Detectors at Airports (April 2022)

See also

Sarah Scoles, Polygraphs Aren’t Very Accurate. Are There Better Options? (Undark, 25 March 2026)

Solitary Thinking

Hackers get busted. Dan Cvrcek raises a very interesting question about the flaws in thinking committed when bright people try to solve problems in isolation.

Science and Censorship

Dark materials. Nuclear scientist Joseph Rotblat campaigned against the atom bomb he had helped unleash. Is it time for today's cyber scientists to heed his legacy?

"There is an ever-widening gap between what science allows, and what we should actually do. There are many doors science can open that should be kept closed, on prudential or ethical grounds. Choices on how science is applied should not be made just by scientists."

Essay by Martin Rees
President of the Royal Society
[Guardian, Saturday June 10, 2006]

---

We cannot allow the terrorists to terrorise us. Scientific research shouldn't be halted simply because it might fall into the wrong hands.

"The scientist's job is to shine light in the darkness, and if we occasionally burn our fingers on the candle, so be it. Lord Rees can choose the darkness if he wants. I'm not going to."

Response by Ross Anderson
Chair of the Foundation for Information Policy Research
[Guardian, Tuesday June 20, 2006]

---

I am uneasy about both sides of this debate. Should science be restrained - either by scientists or by society. Do politicians represent the interests of society, or is there a better and more democratic way for society's interests to be represented?

The fact is that science is already restrained by all sorts of social and commercial forces - above all the willingness to fund particular kinds of research and not others. The choice is not simply between a risk-averse establishment (represented by the Royal Society) and a risk-seeking free-thinking radical alternative (represented by the Foundation).

Of course Anderson is right to be wary of the distorted perceptions of risk by politicians and the non-scientific public. But the proper response to this is a properly constituted debate. Meanwhile, politicians will often seek stupid measures to use and abuse scientists.

As I reported last year (Research Under Fire), scientists and engineers at the University of Berkeley are wary of academic restrictions imposed by the US Federal Government in the name of national security. Thankfully this isn't the kind of restraint Rees is advocating.

A reasonable percentage (3)

One piece of intelligence was accurate.

A man described as Abu Musab al-Zarqawi's "spiritual adviser" inadvertently led US forces to the spot where the militant leader was finally located and killed, the US military says.

Major General William Caldwell said the operation to track down the most wanted man in Iraq was carried out over many weeks, before he was killed after two US air force F-16s bombed a house in a village north of Baghdad.

"The strike last night did not occur in a 24-hour period. It truly was a very long, painstaking deliberate exploitation of intelligence, information gathering, human sources, electronic, signal intelligence that was done over a period of time - many, many weeks," Gen Caldwell said on Thursday.

[BBC News]

---

One piece of intelligence was flawed.

Anti-terror police raided a house at Forest Gate last week after saying they received "specific intelligence" that a chemical device might be found there.

Scotland Yard later said they had "no choice" but to act while the prime minister said it was essential officers took action if they received "reasonable" intelligence suggesting a terror attack.

Tony Blair said he backed the police and security services 101% and he refused to be drawn on suggestions that the armed operation had been a failure.

[BBC News]

---

It's a reasonable percentage. (Previous posts: April 9th, April 18th.)

But that's part of the problem with intelligence - it delivers probability rather than certainty. Perhaps the outcomes are the right way around this time - the presumed-guilty man was killed, and the presumed-innocent man merely injured. (So we shouldn't complain, should we? Imagine the complaints if it had been the other way around!)

But over the long run, are there too many errors? (Difficult to tell, as we only know of some of the better publicized successes and failures.) Should we be uneasy about the errors of intelligence, and the consequences of acting upon erroneous intelligence? There are fundamental questions here about the relationship between knowledge (or ignorance) and action (or inaction).

Conflicting Opinions

Disagreements are unsettling. In a stable world, we like our experts to provide simple and authoritative truth.

Science isn't really supposed to work like that. Science should be constantly open to new discoveries, and new interpretations and explanations of old discoveries. [Kant, Peirce] Science is supposed to work with conjectures, proofs, refutations and paradigm shifts. [Popper, Kuhn, Lakatos, Feyerabend] These are essentially architectural notions. [See my earlier post on Open Architecture]

But the institutions and bureaucracies of science don't conform to this stereotype. There is a closed loop of research funding and journal publication, based on so-called peer review. In a recent report the Royal Society, Britain's most prestigious scientific club, deplores the popular media coverage of science and calls upon scientists to exercise greater self-restraint in publicizing findings that have not undergone the proper peer review process. [Source: John Kay, Financial Times, May 22, 2006]

As John Kay argues, this is essentially an appeal for scientists to be dull and boring. Peer review is inward looking, inhibits new and radical ideas, and serves as a kind of professional censorship. He concludes

"Any form of censorship, including self-censorship and censorship by fellow professionals, encourages complacency and discourages innovation. The history of modern scholarship is that, more slowly than we would wish, truth and new knowledge emerge only from a cacophony of conflicting opinions."

Technorati Tags: architecture Kant knowledge open Peirce uncertainty