I have been watching the public statements from our leadership, media reports, the blogosphere, and emails galore related to the failed December 25th, 2009 terrorist attempt.
In my opinion, this recent intelligence failure has more to do with a lack of imagination than anything else. I am not talking about the imagination of the intelligence analysts – they have been begging for help, instead they face endlessly deep alert queues where the top item is clearly not the most important of the day. And I am not talking about a lack of imagination of our senior leadership – they have been dedicating a vast amount of money for years now toward programs that should have already addressed this recent intelligence failure. Lawmakers even have been changing policy following advice from such organizations as the Markle Foundation’s Task Force of National Security in the Information Age (of which I am a proud member). Our leadership is rightfully miffed by the state of the union despite these substantial investments. Don’t blame the analysts or the leadership this time. The blame belongs elsewhere.
Boiling down the Christmas event to its most simple form – Abdulmutallab applies for a multi-entry visa. The terrorist database (TIDE) is checked and found to contain no such record. The State Department issues a visa. Later, a TIDE record for Abdulmutallab is added to TIDE. Abdulmutallab gets to keep his visa, although his renewal in a few years would have been a problem.
Systems that assume the data will show up before the query are flawed. More about this here: What Came First, the Query or the Data?
The December 25th event is a classic case of enterprise amnesia. Enterprise Amnesia is the condition of knowing something on one hand and knowing something on another hand and never the two data points meet. This disease presents this way: after something bad happens everyone looks in their pile of puzzle pieces and brings to the boardroom table a small, hand-culled selection of data points. And right there before your eyes, it is so obvious. So obvious it can make an organization look incompetent or worse … negligent.
Contrast enterprise amnesia with Enterprise Intelligence. In this model, every time a new record is added, changed or deleted the organization has learned something. At that very split-second one must ask: how does this relate to what the organization already knows (its historical observations) and now that this is known, does it matter, and if so to whom?
Enterprise intelligence roughly translates to making sense of the situation (situational awareness) and then appropriately reacting at that moment (situational reaction). Jump. Duck. Sell it something. Shoot it with a laser from space.
What would analysts and policy makers expect from an "intelligent" system? Abdulmutallab applies for a multi-entry visa. The terrorist database (TIDE) is checked and found to contain no such record. The State Department issues a visa. Later, a TIDE record for Abdulmutallab is added to TIDE. The split-second this record is added to TIDE, the State Department is notified the visa may need reconsidered. (Was there enough evidence for revocation?) I believe when the dust settles and the forensics analysis is completed, whether it is open source or other intelligence collection, it will be clear Abdulmutallab would not have made it onto that plane, so long as this additional fodder was made discoverable.
Devil in the details. For all this to work, the system needs to realize that despite name variations and inconsistent data, the identity in the terrorist database is the identity in the visa system. Recognizing when two people are the same despite having been described differently is sometimes called Identity Resolution or Entity Resolution or more broadly Semantic Reconciliation. Whether one is solving national security challenges, identity theft faced by the financial institutions, or improving health care outcomes, figuring out two identities are the same within and across piles of data is essential to make sense out of the data. Hence my lifelong obsession with such technology.
As for the “Nigerian in Yemen”: Hardly a signal at all. To know if these fragments really mattered, one first must understand the entire universe of weak signal at that time, and how these weak signals have been changing. Gut tells me on any given day there are thousands upon thousands of such dots hovering around. While such chatter has some value – it will rarely, by itself, be a basis for immediate promotion to top-of-queue for the analysts. While this chatter was not essential to detecting and preempting this event, next time, when the signal is truly weak; such chatter may make the difference.
Now what? We must envision systems whereby analysts are not hopelessly pinned down in apathy by information overload … rather as volumes of data increase and signal gets weaker, the analysts get more efficient – producing higher quality and faster decision-making.
Question: Why is it when you put a puzzle together at home the last few pieces are as easy as the first few, despite the fact there is more data in front of you than ever before? More about this interesting phenomenon here: The Fast Last Puzzle Piece and how it is done here: Puzzling: How Observations Are Accumulated Into Context.
And it ain’t no Manhattan project. Let’s get ‘er done right this time, people.
OTHER RELATED POSTS: