Jeff Jonas

Virtual Reality: There Is No Place Like Home

Increasingly over the last year, I have been asked to share my thoughts about virtual worlds (e.g., Second Life and World of Warcraft).  After repeated provocation, I took a peek into these interactive 3D La-La-Lands to see what is up.  Here are a few of my core conclusions:

1.    Virtual realities will end up consuming the attention of a substantial number of humans and this will happen more quickly than most may think.
2.    Data synchronization between the real world and virtual worlds will increase the relevance of virtual worlds.
3.    Along with the eyeballs of transacting consumers will come increased corporate investment thus driving more relevance and more growth.
4.    Investors in virtual world physics re-engineering will possess a distinct advantage in virtual worlds.
5.    As with any tool, a very small percentage of the population will use virtual worlds for criminal activities.

Here are a few details related to these points.

Virtual reality: soon serving the masses.  As these alternate worlds become more immersive (i.e., ability to hold ones attention when in the virtual space) and accessible (think One Laptop Per Child), I think it is possible that a half billion people show up.  How soon?  In six to ten years – maybe faster.  Why?  Because there are a lot of people on Earth that would rather exist in a synthetic world as opposed to their real world.  Hmmm … shanty town, nagging spouse, or insurmountable odds versus a stimulating environment with near limitless potential to reinvent oneself.

Cross-reality synchronization.  Imagine taking heat sensors in a real-world data center and publishing these into a virtual space which is physically configured like the real data center.  The difference being that the immersed person can now physically visualize the temperature distribution in the data center.  This is already being done.  Then move something in the physical world and it moves in the virtual world at the same time, automatically.  I wouldn’t be surprised if this is already happening in some context, somewhere.  As well, the inverse: There is supposedly a fellow who has an island in Second Life with a surveillance mechanism implemented that sends him a real-world email or text message whenever someone steps foot onto his island.  Long story short, if one wants to look at something specific, if the real and virtual worlds are synchronized, it’s going to be cheaper, faster and will burn less carbon if one takes the virtual option.

Real economic growth in unreal worlds.  Have you heard about the real estate developer in Second Life that has made $1,000,000 (real) US dollars?  True.  Linden dollars, the monetary unit in Second Life, have their own currency exchange to US dollars called the LindeX – a currency market apparently moving millions of dollars a month between the virtual and real worlds.  Point being, if a billion people show up in virtual spaces, each on average spending only eleven cents ($0.11) a month – this amounts to a real growth market which will trigger further industry investment.  Consequently: more people arrive. 

Virtual world physics re-engineering.  Through serendipity, careful study, and/or experimentation it is possible to develop capabilities within virtual reality that other participants, or in some cases even the creator, cannot fathom.  The first such instance I heard of five or six years ago involved a player who figured out he could climb walls by mounting deactivated explosive devices on a wall.  Placing one above the other, the avatar cleverly scaled the wall.  He climbed so high that he moved beyond the rendered space … into a region of the game where all that was visible was texture-less grid lines.  In a more recent example, in Second Life an individual created a covert listening device the size of a single pixel -- then placed this pixel inside some object.  Later when the object was near a conversation, all communications were echoed to a third party unbeknownst to the victims.  Prior to this event many players in Second Life would not have considered this possible.  Game physics re-engineering is also happening in World of Warcraft (where it is called "Theorycraft").  In this virtual world, one expert explained it to me this way “[we are] unwrapping the mathematics and developing a perception of space and time in relation to the virtual world, that determines which combination of attacks or defenses have the greatest efficacy.”  This info is then shared with colleagues in password protected chat rooms.  Using this knowledge delivers extraordinary advantage, namely lethality per second optimizations, hence the importance of keeping this specialized knowledge to a privileged few as long as possible.  By the way, let’s not forget that we are re-engineering the physics here on Earth in a similar manner.  Heck, ten thousand years ago, who would have conceived of the possibility that spaceships could be devised to take man to the moon and back! 

Tools are tools.  Are virtual spaces dangerous?  Well, is a phone, the Internet and email dangerous?  Nope, not for the most part, in fact the opposite, as the social and economic values of these technologies far outweigh the consequences of misuse.  Sure bad actors will continue to use the best tools they can get their hands on too.  And with this behavior, as more bad actors show up … the folks paid to “protect” us will venture into these virtual spaces in an effort to detect and preempt.  Hence some of my quotes in this recent Washington Post story “Spies Battleground Turns Virtual.”

And finally, how will you know virtual worlds are starting to collide with your own real world?  Watch for this sign: someone wants to chat with you while showing you something and they explain the best way to do this efficiently is for you to “step in” [to the virtual world that is].

RELATED POSTS:
Ghost in the Machine?

The Vegas Asymmetric Threat

If you have not already seen my post entitled Takin’ Vegas, I recommend you read it first.

Guns don’t kill people. People kill people. And so it is. In Vegas, behind every scam there is a "who" … and it is always a who (or many whos) that lays out strategies, develops plans, recruits actors, trains, adapts, rehearses and ultimately executes the mission – all in hopes of taking home some loot. Undetected.

How do they do this?

For starters, if the actor believes he has already become known to casinos as a subject of interest, he will attempt to defeat recognition through the use of a disguise e.g., changing his hair style, growing a beard, sporting a bandana, slipping into some motorcycle apparel, etc. Some go so far as cross-dressing or pretending to be wheelchair-ridden.

Beyond a new look, opportunists will claim false identities and carry false credentials. There is one particular subject I am familiar with that has used over 30 very different aliases and a handful for different social security numbers and dates of birth. [On a funny but related note, check this out: Be Anyone in Vegas: Get Help Creating a Cover Story Here]

When actors want to expand their business, they recruit. Recruitment can be local or distant. New recruits are trained in rogue facilities so that by the time these new actors appear on the scene, they have superb tradecraft despite having never laid foot in a casino.

One insidious form of recruitment is the recruitment of a casino’s longstanding, high roller to participate in a coordinated covert scheme … or worse, the recruitment of the casino’s own employees. I know of one case where a dealer was coerced into assisting the scammers after being told his wife and children were going to be hurt.

The more significant the opportunity, the more organized the group, which means strategy meetings, business plans, capital raising, operating procedures, training guides, and so on. Attention to detail is essential to increased revenues and the sustainability of the group effort.

Larger teams may establish counter-intelligence measures to reduce the risk of being infiltrated by the establishment or scammed themselves by scammers. Their own team members may be placed under covert surveillance to ensure these trusted insiders have not gone bad over time as trust has a half-life.

Large teams have cells, sub-units that are expected to work together. Some cell members are members of several cells. As with any organization there are conflicting purposes and ideologies, thus membership in cells tend to morph over time.

Often, there is no apparent center of power. New groups emerge simply based on the inspiration of a movie, book, online chat rooms, etc. Groups form, merge, split and dissolve. Some groups just specialize in card counting while others specialize in a certain cheating methods e.g., "bet pressing" – adding to a bet after the outcome is known or the manufacturing of illegal cheating devices. Some groups start with one particular brand of cheating and then branch out into other specialties.

Imagine coordinating a team with 50, 100 and sometimes even more actors! What a challenge!

Well, actually, thanks to the Internet … training, recruitment, soft targets, etc. are at one’s fingertips. Some web sites maintain a level 1 area with limited revelation and benign chat rooms. There are level 2 areas which require the user first undergo a background check before being provided access. At level 3 the member is not only required to have a background check but is also required to have committed a felonious activity. And it is this group that shares even more sensitive information … for argument’s sake this might be viewed as equivalent to a government "Top Secret" clearance. Above this level, much like a military’s Special Access Program, very specialized knowledge is controlled to a minimum and known universe of folks. When the identities of the 12 Nevada dealers with subtle procedural defects in how they dealt hand held blackjack were discovered, such secrets are worthy of such extraordinary controls. The principle: if too many people knew about this weakness, it may be exploited at a pace that would have resulted in casino defection, prompting the casino to quickly close this vulnerability.

The latest challenge? Poker.

Poker is the problem these days. You think you are playing at a table with a fair shot. But unbeknownst to you, there is a team on that table working together against you – even though they make no outward appearance of knowing each other.

Is that slight twitch in the eye "natural" or "a signal"?

RELATED POST:

Takin’ Vegas

IEEE Spectrum Story: Vegas 911

IEEE Paper: Threat and Fraud Intelligence – Las Vegas Style

Web 2.0 – Al Qaeda’s Most Effective Force Multiplier

Found: An Immutable Audit Log

An immutable audit log is a tamper-resistant recording of how a system has been used – everything from when data arrives, changes, departs, to how users interacted with the system. Each event is recorded in an indelible manner - even the database administrator with the highest level of system privileges cannot alter the past … kinda like the paper tape on an adding machine tape, etched in stone … only more high-tech.

I think (and hope) tamper-resistant audits will become common place in settings ranging from health care patient records to government surveillance systems. The primary value being twofold:

a) Accountability. Enable policy folks charged with oversight and accountability to validate that a computer system has been used within policy and law: and,

b) Deterrence. The "chilling effect" caused by the knowledge that a tamper resistant audit log is in place – deterring a corrupt person or two from bad behavior.

Well, good news. I stumbled onto a software company in Spain called Kinamik which has been dedicating its technical resources towards the creation of … a tamper-resistant audit log!

Now what? What if no one wants to pay for one? Will tamper resistant audit logs need to be built-in to commercial off-the-shelf systems to reach the market? If so, will organizations actually pay for the additional disk space and processing requirements to turn such a log on? Or, will they simply turn the feature off?

This is important technology and one that really needs to see the light of day, especially in conjunction with non-transparent government systems.

If any of my readers have thoughts as to what kind of incentives or levers will be needed to make such audit logs a reality, I would love to hear from you. As well, if you discover any other companies selling tamper-resistant logs, please let me know. I would like to compile a list.

RELATED POSTS:

Yesterday’s Technology Review Story: Blinding Big Brother, Sort of

Immutable Audit Logs (IAL’s)

More Death Cheaper in Future

The difficulty and cost of delivering death and mayhem are dropping so fast, there will come a time in which the ill-will of a few evil men could ruin the day for millions.

Technological advances in physics, engineering and biology coupled with the Internet and the dynamics of Web 2.0 have contributed to unprecedented social progress and overall improvement of the human condition. In many ways … and in most places … it is better now than ever before; hence my recent post "The World is Not a More Dangerous Place." At the same time, these same phenomena are accelerating the lethality potential per unit of human effort.

Example 1: The difficulty required to build and deliver the first few 10-kiloton nuclear devices in the 1940’s involved 130,000 people and cost two billion dollars ($23B in 2007 dollars). Today, graduate students are building viable detonation systems … albeit lacking the enriched uranium or plutonium. But unlike the 1940’s when enriched uranium did not exist – every ounce having to be produced – today this nuclear material exists in stockpiles all over the world.

Example 2: Recent biological advances have made it possible to reanimate the 1918 Spanish Influenza. Did I say "possible?" Sorry, I meant to say "this has already been done!" Between a couple of tissue samples left over in a military hospital and a deceased Alaskan Eskimo preserved in the permafrost, the virus has been successfully reconstructed and its DNA sequenced. Researchers then proceeded to inject this virus into mice with the human immune system. The result – unprecedented death – the most deadly flu virus ever tested. [story here] While nuclear material is hard to acquire, I was told the DNA sequence of the 1918 Spanish Influenza was already in the public domain. Hard to believe, so I asked a friend in the biological community for a copy of this DNA sequence. So it appears that I now have a copy on my laptop, but what would I know!

While advances in technology are a big part of this trend, other factors contribute as well including population density, dependence on mobility, the tightly coupled interdependencies in which the world operates (e.g., from just-in-time supply chains to your just-in-time access to cash and food) and media-driven sensationalism. Factors such as these have a force multiplying and amplification effect even upon traditional means for mayhem. For example, consider the death and mayhem created by Malvo and Muhammad, the two Washington DC-area gunmen. They were able to turn an investment of a few thousand dollars (car, gas, gun, bullets) into an instrument of terror which not only killed a number of people but also created so much panic the regional economy lost an estimated half a billion dollars ($500,000,000).

And so it seems, as time marches forward fewer people are able to create more damage cheaper and faster.

RELATED POSTS:

The World is Not a More Dangerous Place

The Only Way to Actually Win the (Long) War on Terror

Web 2.0 – Al Qaeda’s Most Effective Force Multiplier

The World is Not a More Dangerous Place

Back in the days when I had my company, Systems Research & Development (SRD), I prevented anyone from pitching my software using "the world is a more dangerous place" as the set up pitch.

Two reasons: (A) I think it is safer to be alive now than ever before and (B) I hate the idea of using the "fear card" to sell.

Before you call me crazy, consider the following: In the 1300’s the Black Death killed an estimated 75 million people – including a third to two thirds of Europe’s population. The 1918 Spanish Flu killed 50 – 100 million in just 18 months making by far the most destructive pandemic on record.

The average life span at the end of the nineteenth century in Western Europe was thirty seven. Today the average lifespan in the world is sixty seven! [Ref: Life Expectancy]

In short, you are more likely to grow older today than any time in the history of man.

Here is another point of reference: Even if America sunk into the ocean the 300 million deaths would be ~4.5% of the world’s current population (~6.7B). The 75 million lives lost to black death amounted to ~17.4% of the world’s population at that time (~432MM). Thus, if you were standing in America and discovered it was going to suddenly fall off into the ocean in the next few minutes, although this makes for a very bad day for you personally, overall the world still would be a less dangerous place as compared to the mid-1300’s.

Nukes complicate this equation. The two primary nuke scenarios being: a) one-se-two-se nuclear detonations carried out by stateless criminals; and b) a full scale global nuclear war causing the annihilation of mankind.

While periodic unscheduled 10-kiloton nuclear detonations would be very very bad, until such events exceed a few a year (or they go thermonuclear) – in the grand scheme of things us Earthlings are still safer than the 1,300’s. (True. If all of these events happen in a single geography, then while the world at large would still not be a more dangerous place, that specific geography would certainly be a more dangerous place!)

The scenario involving a full-scale nuclear exchange of large numbers of thermonuclear weapons deserves special attention. True, the risk of global nuclear annihilation was absolute zero before the 1900’s and today this risk is no longer zero. But, this risk ebbs and flows. One way to consider how this risk changes over time is the Doomsday Clock. Remember that? The idea being, the closer this clock is to midnight, the greater the risk of global annihilation. Its keepers calculated the time period 1953-60 as the closest the world has yet come to a doomsday event (2 minutes till midnight). [Note: the Doomsday clock was not adjusted in 1962 during the Cuban Missile Crisis as this incident came and went faster than the group reconvened and reset the clock.] Then from 1991 to 1995 the Doomsday Clock was rolled back to 17 minutes until midnight suggesting times were the safest since the inception of this clock in 1947. Notably, the clock shows that since 1995 the safety of the world has been declining. Nonetheless, at this point in time even when considering nuclear Armageddon, the world is less dangerous today than 1953-1960.

So in all fairness, when considering whether the world is a more dangerous place one would also have to ask "as compared to when?" and "as compared to where?" For example, if you called Chernobyl home on April 27th, 1986" you were definitely in a more dangerous place.

And one more thing … when the world seems like an incredibly dangerous place ... you can probably thank some of the media for that. The media’s ability to take every bad thing that happens on the planet and package it up for maximum sensation plays a huge role in spreading fear. It’s not their fault of course it is yours (and mine) as sensational news is what draws us into the media. And as our attention gives them higher ratings they justifiably work even harder at finding, packaging and delivering up even more of this bad news for us. [So, I propose we fix this by only directing our attention to "good news" stories from now on ok? Wait, is that a smoke plume on CNN … get out of my way … I gotta see this!]

Honestly, if you could pick another time to live, would you really trade living in this age for an earlier century? I wouldn’t. Oh, and I wouldn’t want to trade it for 100 years in the future either – I think the future has a chance of being really messy.

These could be the golden years!

PS: Before you get too excited one way or the other about this post, take this into account: This is my Yin post. Stay tuned for my forthcoming Yang post which will be entitled something like "More Death in Future Cheaper."

RELATED POSTS:

The Only Way to Actually Win the (Long) War on Terror

Web 2.0 – Al Qaeda’s Most Effective Force Multiplier

How to Use a "Glue Gun" to Catch a Liar

"People lie. How are you going to account for that?"

This question used to make me crazy. I always wanted to blurt out, "And the sun is going to consume the earth someday – deal with it!"

I never said this, of course.

Anyway, I have a more thoughtful response these days.

Try this on for size. Yep. People are going to falsify information. In fact, you may have experienced this in your life. Let’s say you had a friend – or so you thought. Over time you discovered that this person was in fact dishonest. How did you discover this? The answer is simple: you collected more observations over time.

Observations add up.

I have seen this play out in real data. For example, there was this very big database (billions of table rows describing hundreds of millions of unique people). In this particular database there was this one fellow who was repeatedly lying about his identity. He did a good job, in fact such a good job that despite Semantic Reconciliation processing he appeared to be six different people.

The guy was a liar and no one knew ... that is until future observations (created by his own actions) flushed him out.

[Skip this next paragraph, if you are speed reading or want to stay out of the weeds.]

Here is how this happened. Imagine six apparently discrete identities. Some name similarity, but that never matters at this scale. Then one day this fellow decides to use one of these identities (using previously reported features e.g., same name, phone, SSN, date of birth, etc.), except this time he introduces a new address, one that had never been previously associated with this identity. So this new record is identity resolved to the existing identity – the identity he wanted to present). This caused context accumulation – in this case the new address enhanced what was known about the person he was being today. Sequence Neutrality processing then fires-up to make sure earlier identity resolution events are still valid. During this process another identity was located that shared the new address (the one just learned) and other matching features (e.g., similar names and more). The identity he was trying to be had now become conjoined to one of his other identities – one he was trying to distance himself from. [Technical note: I am specifically using the term conjoined as opposed to merged. Think of conjoined like being rubber-banded together versus merged where two records become one. This is essential for many reasons e.g., retaining the ability to change one’s mind later. More about this in a future post.]

When two identities collapse into one identity – this new conjoined identity now has more context. As something new had just been learned, sequence neutral processing immediately determines if there are any further assertions of the past to fix (e.g., more identities that can be conjoined, or in some cases, disjoined).

Long and short, his six discrete identities collapsed into one … thanks to the arrival of two new records.

Knowing this, one thinks about what data sources are better than others. Some data sources are so good … they work like "glue guns."

From a national security and privacy point of view, it is the above behavior that makes it so important to debate what perceptions (observations) are fair game for context construction, and when.

RELATED POSTS:

More Data is Better, Proceed with Caution

Ubiquitous Sensors? You Have Seen Nothing yet

Accumulate Context: Now or Never

To Know Semantic Reconciliation is to Love Semantic Reconciliation

Web 2.0 – Al Qaeda’s Most Effective Force Multiplier

A few weeks ago someone told me that the Cold War was won because we were able to master silicon first. Well, I don’t know, but this could be true, in part, I suppose. The next thing that popped into my head was:

If the Cold War was won with silicon, we are currently losing the war on terrorism due to Web 2.0. Ouch!  Web 2.0 enables these folks to communicate, share tradecraft, recruit and synchronize at a velocity and resiliency that is unprecedented.

This is certainly one of the key mechanisms enabling the viral-like growth of Al Qaeda-aligned missionaries.  What evidence, you might ask? Check out this story ("Web of Terror") written by Rita Katz and Josh Devon.

RELATED POSTS:

The Only Way to Actually Win the (Long) War on Terror

The Only Way to Actually Win the (Long) War on Terror

I think the only winnable long-term counterterrorism strategy involves lowering ill-will.

No matter how many holes in the dam and how many prophylactic technologies are fashioned as fancy fingers to plug these holes in the dam … if ill-will continues to grow, the water will be coming over the top of the dam.

If ten people on earth want to see democracy fail, the risk to our free society is relatively low. When this number becomes ten thousand, a million, or ten million, catastrophic damage can be rendered. And this damage will be some combination of physical harm and the eroding away of privacy and civil liberties. Dark days.

Some number of years ago, Jim Simon, a former CIA executive, once told me "They can blow up our buildings and kill our people and we still don’t lose. But the day we have to change our Constitution in response to terrorism … we lose."

This sent a shiver up my spine. And this happens to have been the first time I really began to ponder a previously unfamiliar concept to me … something called "privacy and civil liberties." (Although embarrassing, better a late bloomer than never a bloomer.)

In any case, I think about this a lot. And as a technologist making efforts to balance national security and privacy, I can’t but help think, where are the folks developing a workable strategy to lower global ill-will?

While this is obviously outside of my area of expertise, it is clear to me that a well executed strategy to lower global ill-will is significantly more important to a brighter future than any of my work.

"Need to Know" vs. "Need to Share" – A Very Fine Line Indeed

I kept thinking I should title this post, "Knock. Knock. Who’s there?" But I scrubbed that idea.

This post is related to national security, intelligence, classification and information sharing. If this is not your domain, my comments below will make no sense.

Back in pre-9/11 days, those holding classified data used to apply the "Need to Know" model when considering information sharing. Following 9/11, there has been a call for improved information sharing, which has resulted in the new mantra "Need to Share."

What do you think is the difference between "need to know" and "need to share"? When push comes to shove, most people I speak with in the mission cannot quite articulate the difference. Not good.

In part, "need to share" involves a new mindset. This new mindset was highlighted in the third report issued by the Markle Foundation’s Task Force on National Security in the Information Age. For example, in this report we called for an increased use of tearline reporting and a decreased use of ORCON designations. (see report pages 44-48)

There is another aspect, however, that, while referenced in our report (see report pages 46 and 61), may be lacking the attention it deserves. And this is the subject of data indices. These are so fundamental to implementing a functional information sharing program, that I might hazard to say that without data indices … there is little to no hope information sharing will ever be solved. Let me explain.

If someone is the custodian of a highly relevant data item how will they "know who needs to know?" And conversely, if someone else is in need of this highly relevant data item how will they "know whom to ask?" Basically the problem is: who needs to know what? Example: How will the folks working on counter-proliferation know they have a record that is directly related to another team specializing in anti-money laundering? The chances these two groups (even if working in the same building ... ouch) will actually recognize they have related data points is close to Z E R O. If there were just these two groups, the problem would be trivial and could be worked out. But in the real world, organizations may have hundreds of isolated data sets. On whose door shall I knock?

In this earlier post I introduced the Information Sharing Paradox. This paradox basically states that if everyone cannot share everything with everyone else, and everyone cannot ask everyone else every question every day … then how is someone going to find something?

The answer of course is one must first solve "discovery," i.e., knowing who to ask for what. All large scale discovery problems are solved by central indexes (data registries with pointers). Be advised, discovery is not solved by a federated search where one broadcasts searches across the enterprise. And if you hear that federated search is the solution, be afraid, be very afraid. [I explain this in some detail in this post here.]

In order for "need to share" to fulfill its full potential, data custodians must first publish (limited) metadata to the central index. More precisely, when I say "publish data," in actuality they will need to use data tethering to ensure all adds, changes and deletes are properly reflected in the index. At libraries, index metadata about new documents includes subject, title and author. In your business this limited metadata is more likely to be something like who, what, where, when, etc.

As central indexes will be the means by which information discovery challenges are solved, this becomes a way to begin focusing the privacy and civil liberties debate.

One privacy related tension will be defining exactly what kind of data should be discoverable, i.e., placed in the index? For example, in counter-terrorism information sharing programs, there would be significant controversy over, say, including pharmaceutical prescription information of all US citizens; whereas, including foreigners banned from traveling to the US would probably cause little to no concern. The subject of discoverability (i.e., selecting which data will live in the central index) deserves much debate.

On the good news front, solving discoverability via central indexes brings with it a few useful privacy protections including: a) urges to share more data with more parties is replaced by transferring less information to one place (the central index), b) who is searching for what and what they found can be logged (e.g., using immutable audit logs) in a consistent manner thus facilitating better accountability and oversight, and c) information sharing between parties is now reduced to just the records that they need to know and need to share (sharing less by sharing only information that must be shared), and d) it is now possible to make the index anonymized (see: Anonymized Semantic Indexes), which means the risk of unintended disclosure of even the limited metadata in the index is drastically reduced.

Whether living in the "need to know" world or the "need to share" word, one must first be able to answer the question "who" and "what"; otherwise, this dog won’t hunt.

RELATED POSTS:

Discoverability: The First Information Sharing Principle

Information Sharing: Got Directory?

No Need to "Over Share" – Thoughts on Information Sharing

It’s All About the Librarian! New Paradigms in Enterprise Discovery and Awareness

Intelligent Organizations – Assembling Context and The Proof is in the Chimp!

Federated Discovery vs. Persistent Context – Enterprise Intelligence Requires the Later

Predicate-based Link Analysis: A Post 9/11 Analysis (1+1= 13)

Had I been blogging in 2001, this would have been posted then. And although I posted something similar to this on May 16th of last year [here] … I am posting this now as some of the reporting to date around my work in this area has been overstated and/or inaccurate.

Following September 11th many newspaper and magazine stories began showing how the hijackers were related to each other and ultimately to Osama Bin Laden. And with these pictures came suggestions that this event could have been prevented had the government had access to much more data (e.g., health care records, banking records, communications, etc.). As well, there appeared to be an emerging consensus that by studying merely the shape of the 9/11 network, one may able to locate similarly shaped networks – thus, detecting and preempting future events.

I disagreed with this thinking. In fact, it was my opinion that, at least in the case of 9/11, neither more large data sets to graph the nation nor hunting for similar network shapes in this graph would have been necessary (or even useful) for detecting and preempting this event.

Ever see someone standing in front of a giant graph? Imagine a picture with millions of nodes connected via millions of lines each with varying thickness and color. Think spaghetti-fest ready to feed 10,000 people. While very impressive to look at … looking at such is not useful in establishing a starting point.

Networks are useful when one has an entrance point. From a specific vantage point one has a string to pull. And in the case of criminal investigations, these starting points are "predicates." By this I mean knowledge about something or someone that meets some threshold on the scale of reasonable and particular (calibrated with respect to the crime i.e., a different threshold for a deadbeat dad versus a nuclear threat), and that justifies some further action.

From this predicate one begins an investigation or inspection – pulling the string and marching down the path toward the ultimate fact: whether someone is planning something, or has done some bad act. When an investigation is started without a sufficient predicate, or starting point, one risks rampant false positives which not only waste resources, they bring investigative attention to the innocent – which results in unnecessary intrusion on our privacy and worse our civil liberties.

While the question of "what is a predicate" is worthy of a longer conversation and debate, in the case of the 9/11 hijackers, there were two perfect starting points. Both Nawaf Alhamzi and Khalid Al-Midhar were already known to the US government to be very bad men. They should have never been let into the US, yet they were living in the US and were hiding in plain sight – using their real names.

When 1+1=13: Starting with these two guys I drew from various public sources (e.g., investigative journalism, grand jury indictment, etc.) to demonstrate how the network would have looked. In short, with basic investigative procedures, I demonstrated that at least 13 of the 19 could have been exposed.

So, back in the day when running SRD, I created a series of PowerPoint charts to illustrate exactly this point.

This was first published on page 28 of the Markle Foundation’s report entitled "Protecting America’s Freedom in the Information Age." Since then, it has found its way into a number of other publications (e.g., Newsweek: Geek War on Terror).

From time-to-time, though, this work has been characterized incorrectly. For example:

It has been said that the data was run through NORA to develop this analysis. Nope. I never had this data. Rather, I just analyzed the open source and told the story – which required no computational power at all.

It has also been said that had NORA been in use by the US government, 9/11 would have been prevented. Ha Ha! The whole point of my 9/11 analysis was that the government did not need mounds of data, did not need new technology, and in fact did not need any new laws to unravel this event!

Just to be clear, I am not saying better technology and better laws would not be helpful. Obviously, our government needs both. I am simply saying that according to my analysis 9/11 very possibly could have been averted without either. I attempted to make this point in my most recent paper entitled "Effective Counterterrorism and the Limited Role of Predictive Data Mining." In this paper my co-author Jim Harper of the Cato Institute and I were able to draw upon new insights revealed in the 9/11 Commission Report to more clearly describe just how effective predicate-based link analysis would have been in the context of 9/11.

One more thing: I am often asked about how false positives would have effected my 9/11 analysis had such an investigation been carried out in the real world. The relationships selected for this demonstration involved solely shared addresses, phone numbers and frequent flyer numbers. When constrained by date ranges, the number of additional parties would likely have been minimal, unless the addresses and phone numbers on the plane reservations were actually those of the travel agency (which was not revealed in open source documents). As such, I posit that the investigation would have produced a small universe of subjects and would have exposed the likes of Mohamed Atta.

RELATED POSTS:

Sometimes a Big Picture is Worth a 1,000 False Positives

The Six Degrees of Kevin Arbitrary

Hunting Bad Guys, Phone Records and a Few Good Dead Men

What is Data Mining? Depends Who You Ask ...

110th Congress Debates Data Mining