My Photo

Your email address:

Powered by FeedBlitz

April 2018

Sun Mon Tue Wed Thu Fri Sat
1 2 3 4 5 6 7
8 9 10 11 12 13 14
15 16 17 18 19 20 21
22 23 24 25 26 27 28
29 30          
Blog powered by Typepad

Become a Fan

« Privacy by Design in the Era of Big Data | Main | G2 | Sensemaking – Two Years Old Today »

November 09, 2012


Feed You can follow this conversation by subscribing to the comment feed for this post.


Brilliant post, Jeff. I particularly enjoyed your "Beware of Social Media" advice. Thanks for putting your thoughts in print!

Ed Smith

I think it is important to consider something else, Jeff.

You don't have to solve the entire problem using data alone. Your data can influence your procedures. For example, if your job is to find bombs in shipments, you no doubt have some capacity for inspecting shipments.

You can do anomaly detection among shipments and manifests. Use this to target inspections. Track what kinds of anomalies led to which findings.

You will find lots of false positives, like legitimate manifest errors, illegal activity that isn't bombs, etc. However, you will be much more likely to succeed in your overall mission.

From there you can do even more targeting. For example, maybe a majority of shady shipments are coming from just 15 ports around the world. Target getting more data about what is happening there.

Hyoun Park

Oh, I thought you were talking about BOMs (Bill of Materials) at first. Bombs are much cooler!

But reality is that the proliferation of data sources often just means there are many more wrong ways to find an answer than ever before.


Very funny and very familiar.

There's a variant on this story too, where the customer's first conversation is with the sales guy who then passes it on:

Customer: “That’s it.”

Sales guy: “Sure we've got this great software that will find all your bombs.”


To me the root of the issue is those who created unrealistic expectations for customers like yours. This customer didn't made it out of the blue. It is probably the consequence of reading too much hype marketing material, don't you think?
Your post is really about common sense and realistic expectations. We need more of this down to earth messaging to counter Big Data exagerations we read every day.


Right on. In my own self initiated analytic projects I go back to the adage: "KNOW YOUR DATA." What it can and can't do, what biases or emphases it may have based upon who is collecting and reporting it, and whether there is enough of it. If no, cut your losses and move on.

Doug Wood

I'm in agreement with many of the comments. The customer knows only that they want to find bombs. Based upon how we as vendors present ourselves, they believe we can help them.
Unfortunately, the industry's pervasive 'catchphrase-trumps-technology' mindset does little to create realistic customer expectations and has yet to find a single bomb.

The comments to this entry are closed.