My Photo

Your email address:

Powered by FeedBlitz

April 2018

Sun Mon Tue Wed Thu Fri Sat
1 2 3 4 5 6 7
8 9 10 11 12 13 14
15 16 17 18 19 20 21
22 23 24 25 26 27 28
29 30          
Blog powered by Typepad

Become a Fan

« Privacy by Design in the Era of Big Data | Main | G2 | Sensemaking – Two Years Old Today »

November 09, 2012


Feed You can follow this conversation by subscribing to the comment feed for this post.


Brilliant post, Jeff. I particularly enjoyed your "Beware of Social Media" advice. Thanks for putting your thoughts in print!

Ed Smith

I think it is important to consider something else, Jeff.

You don't have to solve the entire problem using data alone. Your data can influence your procedures. For example, if your job is to find bombs in shipments, you no doubt have some capacity for inspecting shipments.

You can do anomaly detection among shipments and manifests. Use this to target inspections. Track what kinds of anomalies led to which findings.

You will find lots of false positives, like legitimate manifest errors, illegal activity that isn't bombs, etc. However, you will be much more likely to succeed in your overall mission.

From there you can do even more targeting. For example, maybe a majority of shady shipments are coming from just 15 ports around the world. Target getting more data about what is happening there.

Hyoun Park

Oh, I thought you were talking about BOMs (Bill of Materials) at first. Bombs are much cooler!

But reality is that the proliferation of data sources often just means there are many more wrong ways to find an answer than ever before.


Very funny and very familiar.

There's a variant on this story too, where the customer's first conversation is with the sales guy who then passes it on:

Customer: “That’s it.”

Sales guy: “Sure we've got this great software that will find all your bombs.”


To me the root of the issue is those who created unrealistic expectations for customers like yours. This customer didn't made it out of the blue. It is probably the consequence of reading too much hype marketing material, don't you think?
Your post is really about common sense and realistic expectations. We need more of this down to earth messaging to counter Big Data exagerations we read every day.


Right on. In my own self initiated analytic projects I go back to the adage: "KNOW YOUR DATA." What it can and can't do, what biases or emphases it may have based upon who is collecting and reporting it, and whether there is enough of it. If no, cut your losses and move on.

Doug Wood

I'm in agreement with many of the comments. The customer knows only that they want to find bombs. Based upon how we as vendors present ourselves, they believe we can help them.
Unfortunately, the industry's pervasive 'catchphrase-trumps-technology' mindset does little to create realistic customer expectations and has yet to find a single bomb.

Verify your Comment

Previewing your Comment

This is only a preview. Your comment has not yet been posted.

Your comment could not be posted. Error type:
Your comment has been saved. Comments are moderated and will not appear until approved by the author. Post another comment

The letters and numbers you entered did not match the image. Please try again.

As a final step before posting your comment, enter the letters and numbers you see in the image below. This prevents automated programs from posting comments.

Having trouble reading this image? View an alternate.


Post a comment

Comments are moderated, and will not appear until the author has approved them.

Your Information

(Name and email address are required. Email address will not be displayed with the comment.)