Jeff Jonas

Your Movements Speak for Themselves: Space-Time Travel Data is Analytic Super-Food!

It doesn’t matter who you say you are!  Where you are (space), when you’re there (time), and your movements over time (travel) are closer to the truth.

I’ve seen a lot of data in my life, and I’d like to think I have a decent grip on what can be accomplished with data and analytics.  However, I recently stumbled upon some facts that have radically reshaped my understanding of the world we are living in.  What I thought was years away is already here! Our toes are dangling over the edge of a very different future.

Now, before you get all worked up, remember: You have helped create this, most folks love this, and most will continue to eat this up despite the obvious consequences.

Mobile devices in America are generating something like 600 billion geo-spatially tagged transactions per day.  Every call, text message, email and data transfer handled by your mobile device creates a transaction with your space-time coordinate (to roughly 60 meters accuracy if there are three cell towers in range), whether you have GPS or not.  Got a Blackberry?  Every few minutes, it sends a heartbeat, creating a transaction whether you are using the phone or not.  If the device is GPS-enabled and you’re using a location-based service your location is accurate to somewhere between 10 and 30 meters.  Using Wi-Fi?  It is accurate below10 meters.

Fancy.

It should be no surprise that all this data lives in the coffers of the cell providers.  Lots of people know that.  What is new, at least to me, is that this data is being provided to third parties that are leveraging specially designed analytics to make sense of our space-time-travel data. 

With the data out and specialized analytics emerging, this infant industry is already doing some pretty amazing work. Your space-time-travel data makes where you live and where you work self-evident, and it reveals your most frequent, periodic, infrequent and rare destinations. 

The data reveals the number of co-workers that join you Thursdays after work for a beer, and roughly where you all go. It knows where these same co-workers call home, and just exactly what kind of neighborhood they come from (e.g., average income, average home price) … information certainly useful to attentive direct marketing folks.

Large space-time data sets combined with advanced analytics enable a degree of understanding, discovery, and prediction that may be hard for many people to fully appreciate. Better prediction means a more efficient enterprise and nifty consumer services. 

Cellular companies are now receiving essential insight about their customers (e.g., to better understand and predict customer churn).  Major retailers can now better understand changes in consumer behavior (e.g., how far their customers are traveling on average this month compared to previous months).  Consumers are benefiting by getting real-time traffic information so they can avoid congested roads.  (I have a colleague that thinks he is saving two to four hours a week in commute time due to this service!)

Tip o’ the iceberg.

I can barely get my mind around the ramifications. My concept about what comes next shifts almost daily now.  A government not so keen on free speech could use such data to see a crowd converging towards a protest site and respond before the swarm takes form –  detected and preempted, this protest never happens.  Or worse, it could be used to understand and then undermine any political opponent.

A stalker might be questioned just days after he starts and before his victim is personally aware of it – detection previously beyond human capacity.  Maybe it’s not a crime in this case, and it turns out to be just a private investigator with poor tradecraft hired by a suspicious husband.

Such a surveillance intensive future is inevitable, irreversible and as I have said before here … irresistible.

Why?  Companies must be competitive to survive and consumers have quite the appetite for almost anything that optimizes their life, especially if it’s cheap or free.  For example:

Tuesday afternoon your [free] Gmail account advises you that your buddy Ken is going to be 15 minutes late to the pool hall this coming Thursday, unless he leaves work 15 minutes early … which he has only done twice in seven years.  Brilliant!

Your Starbucks drink of choice (a grande vanilla soy latte in my case) is handed to you the instant you pull up, and you did not call ahead nor did they ask.  Priceless!

When powerful analytics commingle space-time-travel data with tertiary data, the world we live in will fundamentally change.  Organizations and citizens alike will operate with substantially more efficiency.  There will be less carbon emissions, increased longevity, and fewer deaths.

I think people should know about this imminent new age we are marching into.

[Theatrical pause.  Breathe.]

Now I’m going to step back and address some questions you may have, using the good news/bad news format.

Good news: The space-time-travel collected by the cellular network carriers is de-identified when provided to these third parties for privacy reasons in that it does not include your name, address, phone number, etc.; rather, unique identifiers are assigned to transactions from the same device so that trends can be measured.

Bad news: If you were to provide your home, work and one other address (e.g., gym, school) in most cases, with just these data points, you are re-identified.  With just a few days of space-time-travel activity, your top three or four more frequently visited destinations become self-evident, and without a whole hell of a lot of effort you could be re-identified through a tertiary data set like a credit header.

Good news: There is so much data being produced, a lot of transactions are tossed aside, are sampled and summarized to make the computational effort feasible.  Historical data also falls off the back of the wagon (ages off the system) rather quickly.

Bad news: The competitive nature of this emerging business model will likely require these organizations to make more sense of more data faster.  Cloud computing and new classes of algorithms will make it possible to keep more transaction detail, keep it longer, and commingle it with other large and very interesting secondary data sets (e.g., phone books and property records).

Good news: So far there are only a handful of companies already entrusted with this data.

Bad news.  It may not be good news that only a few companies do this.  If only one company can monitor the consumer foot traffic of all Nordstrom stores in near real time,  this would be an unfair advantage in terms of predetermining its financial condition before anyone else.  As I learned from countless conversations with my friends at the ACLU, very powerful tools in the hands of a few is not often a good idea without one hell of a lot of oversight and accountability. And even then, this is no panacea.

Good news: Some of the organizations holding space-time-travel data are fully aware of the privacy consequences and are offering consumers the ability to opt-out – meaning, if they get a transaction about you it will be permanently removed from the system and all future correlation.

Bad news: If by chance a snapshot of sufficient detail had been sold off to another party before the opt-out request, then the toothpaste is out of the tube.  Data tends to replicate, more about this here.

Good news: Not any old mom and pop operation can get into this business.

Bad news: That won’t be true for long.  Suppose an aspiring entrepreneur makes a compelling proposition to a number of parties holding space-time-travel data.  Anticipating free analytics and a cut of the future action, the parties work a deal. For computing power this entrepreneur simply hops onto Amazon’s EC2 cloud and partners with a data aggregator to get some tertiary data and what do they have?  An ultra-sexy prediction engine.

Good news: People tend to appreciate location-based services, which is why they are opting in.

Bad news: Sensitive information about people is no longer under their own control.  As well, a number of well held secrets (e.g., your hideout) evaporate overnight.

Good news: If you want to escape the consequences of having your space-time-travel being graphed by others, here are some options that come to mind:

(a) Stop using mobile devices;

(b) Use multiple devices e.g., use one device only at work, and only a land line at home – all mobile devices being off at all other times (never moving around with a device on) – being sure these mobile devices are registered to someone other than you – and if you need to use some kind of device while on the move or at other locations. see (c) below;

(c) Unregistered, cash-purchased, disposable devices – used once then discarded (or recycled!) – although in some cases you can use the device a few times, but you better let some fancy software (which I may have to invent) advise you what is safe usage and what is not.

(d) If you can figure out locations on earth where only one cell tower exists (and you are not moving between towers and never using GPS or Wi-Fi) you will probably live safely under the radar – unless you are a way bad mofo and others know it, in which case, you are ‘going down’ anyway because there are more tricks (expensive) which will be levied against you.

Bad news: Few are willing to be this inconvenienced.  And if only a handful of innocent, clean living folks go to this same effort that the bad guys MUST employ … well crap, that in itself may be considered by some to be signal.

Net Net: My guess is most consumers don’t fully realize how their space-time-travel data is accumulating and congealing.  I hope consumers come to appreciate how all of these nice conveniences of life are delivered. And I hope they will continue to enjoy these while they make better informed decisions, especially with respect to their privacy.

However, without a feedback loop consumers may never fully appreciate what can be gleaned from their space-time-travel trail. Therefore, one way to enlighten the consumer would involve holders of space-time-travel data to permit an owner of a mobile device the ability to also see what they can see:

(a) The top 10 places you spend the most time (e.g., 1. a home address, 2. a work address, 3. a secondary work facility address, 4. your kids school address, 5. your gym address, and so on);

(b) The top three most predictable places you will be at a specific time when on the move (e.g., Vegas on the 215 freeway passing the Rainbow exit on Thursdays 6:07 - 6:21pm -- 57% of the time);

(c) The first name and first letter of the last name of the top 20 people that you regularly meet-up with (turns out to be wife, kids, best friends, and co-workers – and hopefully in that order!)

(d) The best three predictions of where you will be for more than one hour (in one place) over the next month, not counting home or work.

I think Google’s Android and Latitude products might be able to move on something like this first.  It would then be cool if other holders of space-time-travel data followed.

On the subject of privacy and civil liberties consequences, privacy by design is essential.  And for those of you with ideas in the area of policy or technology, I would be most appreciative if you would share these thoughts with me … sooner rather than later.

I will continue sharing perspectives about these ideas and the apparent consequences with my many friends in the privacy community, the defense/intelligence community, and media.  (Surprisingly, their feedback so far has been quite similar.)  I am also speaking with the organizations amassing and analyzing this space-time-travel data to learn more about what is possible.  From the perspective of the analytic engines I create, this space-time-travel data looks like “super food.”

USEFUL REFERENCE:

Internet Society: Wireless Location Privacy: Las and Policy in the US, EU, and Japan

 

RELATED POSTS:

Six Ticks till Midnight: One Plausible Journey from Here to a Total Surveillance Society

Prediction: Channel Consolidation

More Data is Better, Proceed with Caution

How to Use a "Glue Gun" to Catch a Liar

Prediction: Channel Consolidation

As we live life, our actions are recorded across countless channels, e.g., text messaging threads versus ATM transactions and so on.  Channel separation is why your bank doesn’t know where you were physically located yesterday and your doctor doesn’t know the contents of your work emails.  While we take channel separation as a given, channel consolidation is the trend and our society is heading in this direction at warp speed.

Channel consolidation is an essential ingredient to improving accuracy in prediction (e.g., your on-line retailer wisely inferring your interests).  For the most part, consumers love this as it makes life more efficient.  Businesses equally like this as better prediction means more efficient operations.  And for all these same reasons the public sector (intelligence and law enforcement to social services) are just as keen to enjoy improvements in prediction as well.

Facebook makes for a great example of channel consolidation.  All your emails, instant messages, status updates, past/present/and future travel, annotated photos, your social circle, memberships, self–expressed interests, and more … all bundled together in one nice little package, under your user account.  Traditionally such life details are expressed on diverse channels – unobservable to any single entity.  No more.  Facebook, with this panoramic view of its users, now likely has a substantially more complete picture of a person than almost any other single entity.

How powerful is this?  Here is one example: if you are a Facebook user maybe you have noticed the increasingly (spooky smart) relevant ads.  I get ads that read “Are you 44, a triathlete, and want abs like this?”  Or a well-timed ad over the summer when I was in Southern California that read: “Are you looking for a triathlete coach in the Orange County area?”  It is so relevant I find it very hard not to click on the ad!  (Be assured I do resist.)

The more sense Facebook makes of users, the better the service, the more folks will find Facebook irreplaceable, the more users will flock to the platform, and last but not least, the more advertisers are willing to pay.  Everyone seems the winner.

Hence, channel consolidation is inevitable primarily because it is irresistible.  [More here]

Consumers actually demand this. For instance, you expect that your healthcare provider will channel consolidate your data (lab work, prescription history, etc.) to properly care for you – or you may sue them for negligence! 

Nonetheless, it takes no leap to realize this very big and very important question: ‘who consolidates which channels and for what purpose?’

Law and policy will inevitably determine which entities can access and commingle which data (channel consolidation) and under what condition.  At the same time, I worry that the technical means to enhance privacy (e.g., Immutable Audit Logs that facilitate accountability and oversight) are not being adopted at an appropriate pace to keep up.

One more interesting tidbit: People often use and then abandon email accounts.  And I bet most of these folks consider all those communications (e.g., associated blog comments) effectively clipped off – like a tail – and no longer of record.  However, if the data lives on, and if there are features in that data that enable channel consolidation (e.g., your name and one or more additional distinguishing features) … then it is quite possible that these bodies could be raised from the dead.  Hummm…

How to prevent channel consolidation and the resurrection of your clipped tails makes for interesting conversation – but that will have to wait.

And on the lighter side: Facebook, by the way, makes use of a fraction of what they know and uses basic algorithms at best.  I realized this when not long ago I commented to my girlfriend about how damn useful the Facebook ads are getting and she pointed out that one of her recent ads stated “Is your boyfriend gay?”  What the hell!  And then a few weeks later I get an ad that says: “Is your girlfriend cheating on you in Vegas?” 

Touché!

The less likely alternative is that Facebook is using most of the data and very smart algorithms … so smart in fact that they have an advertiser intentionally making us suspicious of each other with the intent of soon dishing up a new ad that says something like “Need a private investigator to watch your girlfriend?”

Bastards!

RELATED POSTS:

Six Ticks till Midnight: One Plausible Journey from Here to a Total Surveillance Society

More Data is Better, Proceed With Caution

Puzzling: How Observations Are Accumulated Into Context

Trust Has a Half-Life

Nation At Risk: Policy Makers Need Better Information to Protect the Country

Last Tuesday, March 10, The Markle Foundation Task Force on National Security in the Information Age released a report titled: Nation At Risk: Policy Makers Need Better Information to Protect the Country. (PDF here)

Members of the Task Force who prepared this report included: William Crowell, Bryan Cunningham, Jim Dempsey, John Gordon, Slade Gorton, Jeff Jonas (me!), Judith Miller, Jeffrey Smith, Abraham Sofaer, Rick White, and Richard Wilhelm.

We made the following five recommendations, calling for the accelerated creation of an information sharing framework:

1. Reaffirm Information Sharing as a Top Priority

2. Make Government Information Discoverable and Accessible to Authorized Users by Increasing the Use of Commercially Available Off-the-Shelf Technology

3. Enhance Security and Privacy Protections to Match the Increased Power of Shared Information

4. Transform the Information Sharing Culture with Metrics and Incentives

5. Empower Users to Drive Information Sharing by Forming Communities of Interest

The report is relatively short and to the point, just 27 pages long. It includes a handy four page appendix summarizing all of the recommendations (pages 22-25).

Here are a few elements I would like to bring to the attention of my readers:

Recommendation 2 speaks to "Discoverability." Our Task Force built on our earlier recommendations (in prior reports) to use data indices – much like the card catalog at the library. Using indices, users can locate data in the enterprise and if qualified for authorized use, they can limit access to the records of relevance. Notably, this model means less data is being transferred around, therefore less data must be kept in synch. The risk of unintended disclosure is mitigated to a degree because fewer copies of data are being made. In short, indices allow users to locate just what they need and no more.

In recommendation 3, the Task Force makes a number of specific security and privacy recommendations. One of my favorite examples is:

"… including implementation of real-time audits of user compliance and behavior and immutable audit logs that record how a system has been used …"

These Immutable Audit Logs are clever little inventions that allow oversight and accountability groups to see exactly how a system has been used. Even a system administrator cannot secretly change the past by altering the log. Imaging every time someone peeks into the card catalog whether they find something, or not, this is recorded in an indelible manner. What cards they saw, what books they looked at, and so on … all accounted for with no way to hide the facts.

The use of the term "metrics" in recommendation 4 is also worth special mention. One such metric the Task Force would like to see is reporting what percent of a system’s information is discoverable i.e., percentage of records that have corresponding cards in the card catalog. Case in point; what is the value of a book on the shelf at the library if there is no related card in the catalog?

Finally, as indices are going to be the way information sharing gets accomplished … in my opinion, the essential policy debate must immediately begin considering the following:

1. How many indices? There are benefits for fewer. And there are different benefits for more. One index for law enforcement and one for intelligence? Or, one index for foreign collections and another for open source? One index, 20 indices, or 100?

2. Where will these indices physically reside? At Google? (Note: Google is an index already used for open source).

3. Which key attributes will be placed in the index? While the library uses subject, title and author … maybe one index will need to contain who, what, where, when.

4. How much latency between data changes in source systems and their corresponding change in the index? For example, if a watch list record is deleted in a source system, what is the maximum amount of time one would want to wait until the same record is redacted from the card catalog?

5. When a user searches the card catalog, who do you notify when an index card is found?

a. Only the inquirer?

b. Only the owner?

c. Both?

d. Neither, only a third party is notified?

e. No one can be told?

6. When there is any notification, what information is revealed about the index card to the inquirer and/or data owner?

a. All the attributes related to the index card?

b. Some of the other attributes on the card (e.g., search author and see title)?

c. No other attributes are revealed?

d. The custodian organization of the data record?

e. The custodian source system of the record?

f. The actual record number used to identify this piece of data in the source system?

g. The user, if any, associated with this record, e.g., the analyst’s name and phone number?

7. When there is a notification to a data owner, what information is revealed about the inquirer?

a. All the attributes related to the search?

b. Some of the other attributes on the search?

c. No other attributes are revealed?

d. The inquirer’s organization?

e. The inquirer’s source system?

f. The actual session number used to initiate the inquirer’s search?

g. The inquirer’s name and phone number?

8. What user audit standards and processes will be required to ensure the system is being used in accordance with law and policy?

9. What metrics will be kept and who can see which metrics?

RELATED POSTS:

Discoverability: The First Information Sharing Principle

Information Sharing: Got Directory?

No Need to "Over Share" – Thoughts on Information Sharing

It’s All About the Librarian! New Paradigms in Enterprise Discovery and Awareness

Federated Discovery vs. Persistent Context – Enterprise Intelligence Requires the Later

Immutable Audit Logs (IAL’s)

Found: An Immutable Audit Log

Full Attribution, Don’t Leave Home Without It

Out-bound Record-level Accountability in Information Sharing Systems

Data Tethering: Managing the Echo

To Anonymize or Not Anonymize, That is the Question

“Macro Trends: The Privacy and Civil Liberties Consequences … and Comments on Responsible Innovation” – My DHS DPIAC Testimony, September 2008

On Wednesday, September 17th, 2008 I testified before the DHS Data Privacy and Integrity Advisory Committee (DPIAC).  The testimony transcripts were recently posted on the DHS web site.

Transcript Here  (My testimony starts on the bottom of page 9.)

In this testimony I covered a number of topics including:

I. Macro Trends

1. The world is not a more dangerous place – average lifespan continues to increase (More here: The World is Not a More Dangerous Place)

2. Fewer people can create bigger effects faster … in both directions – destruction or value creation (More here: More Death Cheaper in Future, Ludicrous Speed Billionaires)

3. Competition is driving technology adoption – notably the organization who receives the data first has the advantage (More here: Ubiquitous Sensors? You Have Seen Nothing Yet)

4. Data is being created faster than organizations can make sense of it – to this degree organizations are getting dumber (More here: Why Faster Systems Can Make Organizations Dumber Faster)

5. Commingling data for enhanced context will drive a new generation of smarter systems (More here: Smart Systems Flip-Flop)

6. The surveillance society continues to build momentum as consumers find it irresistible (More here: Six Ticks till Midnight: One Plausible Journey from Here to a Total Surveillance Society)

II. Las Vegas: What Can Be Learned

1. Las Vegas is one of the fastest growing cities in the Unites States of America (before the economic crash of course)

2. Casinos have a minimal security and surveillance budget

3. Casinos have a legal obligation, in some circumstances, to determine your identity (e.g., under age gambling, crossing a winnings threshold which necessitates IRS reporting)

4. Public and private watch lists are used – some compulsory, some elective

5. There are a fair number of scams that are tried on casinos – some scam artists are using false identities and disguises (More here: The Vegas Asymmetric Threat, Takin’ Vegas, Be Anyone in Las Vegas, Get Help Creating a Cover Story Here)

6. When employees go bad, this is particularly problematic

7. Most ‘tripwires’ are generated by alert employees who are watching, not by computers

8. An organization called Griffin Investigations provides information sharing of cheaters and advantage players (e.g., card counters) – they system is called Griffin GOLD (My old SRD company build this system)

9. One example of a watch list and insider-threat detection system known as Non-Obvious Relationship Awareness (NORA) was built for the industry (My old SRD company build this system too) (More here: IEEE Spectrum Story: Vegas 911, IEEE Paper: Threat and Fraud Intelligence – Las Vegas Style)

10. A patron can still enter a casino and enjoy a degree of activity without exposing their identity

11. There is no “predictive data mining” to spot unwanted behavior

12. Most vulnerabilities are remedied with processes not additional electronic surveillance

13. Humans are in the security decision loop – computer systems do not make security decisions, rather only promote items of interest

14. Facial recognition technology is not used at doors looking for watch listed persons

III. Towards Responsible Innovation

1. Technologists should more regularly engage the privacy community (More here: Responsible Innovation: Staying Engaged with the Privacy Community) 

2. Information sharing systems must have information attribution (More here: Full Attribution, Don’t Leave Home Without It, Out-bound Record-level Accountability in Information Sharing Systems) 

3. Data destruction requires careful planning and execution (More here: Decommissioning Data: Destruction of Accountability) 

4. Limit data transfers and use indexes to make information discoverable (More here: Discoverability: The First Information Sharing Principle) 

5. If data changes in a source system replicate this change through the information eco system (More here: Data Tethering: Managing the Echo) 

6. Where possible obfuscate data (More here: To Anonymize or Not Anonymize, That is the Question) 

7. Build high assurance accountability into systems (More here: Immutable Audit Logs (IAL’s), Found: An Immutable Audit Log) 

8. Data mining is not always good or bad, it depends on the circumstances (More here: Effective Counter-Terrorism and the Limited Role of Predictive Data Mining, Data Mining, Predicate Triage and NSA Domestic Surveillance) 

9. Link analysis, especially predicate-based, has some usefulness – although it is also wise to ‘prune’ early (More here: Hunting Bad Guys, Phone Records, and a Few Good Dead Men, Predicate-based Link Analysis: A Post 9/11 Analysis (1+1= 13), Sometimes a Big Picture is Worth a 1,000 False Positives) 

10. Low fidelity watch list entries (identities with few attributes, e.g., name only) are problematic (More here: Precision in TSA’s Terrorist Watch List, Comments on the TSA No-Fly and Selectee Watch List Process)

IV. Closing Statements

1. There are going to be more sensors, more data.  This data will be commingled for greater accuracy to serve consumers and to protect countries.  What data is collected and when … will be the debate.  Once data has been collected, the holder has the obligation to make sense of it.

2. The most fundamental principle I have synthesized from my many conversations with folks in the privacy advocacy community is this one point: “Avoid consumer surprise.”  (More here: Where Possible … Avoid Consumer Surprise)

3. Finding professional bad actors involves the detection of weak signal.  The computational remedy for weak signal involves observing and commingling transactions that are unanticipated by the bad actors.

4. Hence the tension.

[End of Testimony] 

I wish I would have also made this point:   

I am concerned.  If organizations don’t start implementing more privacy-enhancing technology I think research and productization in this area will decrease in coming years.  This would be a bad thing. 

RELATED POSTS: 

My Testimony Before The DHS Data Privacy and Integrity Advisory Committee (2005) 

RELATED PAPERS: 

Giannino Bassetti Foundation: Jeff Ubios

Transparency, Privacy and Responsibility: An Interview with Jeff Jonas 

Heritage Foundation: Paul Rosenzweig/Jeff Jonas

Correcting False Positives: Redress and the Watch List Conundrum 

Cato Foundation: Jeff Jonas/Jim Harper

Effective Counterterrorism and the Limited Role of Predictive Data Mining  

Steptoe & Johnson: Stewart Baker

Anonymization, Data-Matching and Privacy: A Case Study  

IEEE Security and Privacy: Jeff Jonas

Threat and Fraud Intelligence: Las Vegas Style 

Santa's Surveillance Operations Center Enjoys Big Gains in 2008(*)

While everyone knows Santa maintains the most invasive, robust, 7x24x365 data surveillance operation in existence against mankind as a whole; nonetheless, far from being called under scrutiny or criticized by privacy advocates, this operations center grew dramatically and saw vast improvements in accuracy throughout the 2008 calendar year.

Santa’s insatiable appetite to know who’s been good and who’s been naughty is the motivatation for doing such a meticulous job collecting and analyzing data on every individual.  As with any surveillance operation there are always a variety of gaps in time for which one cannot account for the behavior and location of an individual.  However this surveillance gap closed to some degree this year.  Jolly lucky for Santa, when certain airlines began refusing cash for in flight sales in lieu of credit cards, Santa’s ability to see that someone had indulged in binge drinking on a flight is now possible.  Other gains in 2008 were seen in many areas including, more phones with GPS and cameras, parking garages only accepting credit cards ... making one’s anonymous presence in that location more difficult. 

Of course, Santa is benevolent.  So, virtually no one has a problem with Santa’s surveillance operation and related data centers.  

Also worthy of mention; the Santa's data governance and overall IT security must be extraordinary as there continue to be zero reports of a breach.   (If there were a breach the operation would have had to make a public disclosure -- as mandated by law -- duh!)

And on an unrelated note, if I were to make a prediction, the notion of giving people who have been naughty a piece of coal in their stocking is possibly a policy that needs to be reconsidered … for obvious reasons.

(*) Forward looking statements contained in this report are opinion only and should not be counted as facts or performance predictions.

RELATED POSTS:

Six Ticks till Midnight: One Plausible Journey from Here to a Total Surveillance Society

USC School of Cinematic Arts, "Imagine the World in 2050"

In April, 2008 five IBM scientists including myself appeared at the USC Film School on a panel discussing what the world would look like in 2050. Picture here.

The general topic was “putting the science back in science fiction.”

We each had five minutes to articulate how technologies might affect us in the year 2050. The rest of the session was dedicated to Q&A. I presented a newly crafted hand-drawn PowerPoint deck entitled “Macro Trends and What They Mean for 2050.” 

In summary, I made these observations about the future:

1. Big advances are generally not the result of any single technology but rather the result of combinations of technologies.

2. Good news: The world continues to become less dangerous. Today, we live longer than any time in the history of man – and this trend will continue. More here: The World is Not a More Dangerous Place.

2050 Prediction: Your doctor is 102 and this is not weird.

3.0 Fewer people can create much more, much faster … and more easily.

3.1. The bad news is faster death: Today, it takes less than 50 people and less than $100,000 to manufacture – and infest humans with – the reanimated 1918 Spanish Influenza virus. Some estimates place the death toll of such an event at 160 million people. That is an awful lot of death for relatively little cost and effort. More here: More Death Cheaper in Future.

3.2. The good news is faster wealth: On the other side of the coin, wealth can now be created faster than ever. For example, Marc Zuckerberg of FaceBook went from zero to over a billion in net worth in under three years. More here: Ludicrous Speed Billionaires.

2050 Prediction: Your 14-year-old neighbor makes $10B from their bedroom.

4.1. Surveillance societies are not only inevitable, irreversible … but more importantly they are irresistible! You love location-based services on your phone and you will love RFID chips in your sunglasses so you can find them if you lose them. More here: Six Ticks till Midnight: One Plausible Journey from Here to a Total Surveillance Society. 

4.2. Sensors become ubiquitous … not due to governments … rather this is caused by commercial enterprises as they compete for consumers who are eager to adopt any and all technologies that help them optimize their lives. More here: Ubiquitous Sensors? You Have Seen Nothing Yet.

4.3 Ubiquitous sensors result in countless piles of data. But to make real sense of this data it must be commingled. As such, these piles of data will have a tendency to converge into one pile of data. More here: More Data is Better, Proceed with Caution.

5. When disparate information is stitched together context emerges – much in the same way individual puzzle pieces are difficult to make sense of in isolation. Point being, the more puzzle pieces that can be associated, the greater the overall picture emerges. This stitched-together data will be stored in a pre-assembled form, in very large databases, ready for use. But unlike today, the enormous power of such “information in context” will not be in the hands of a few privileged (e.g., multi-billion dollar organizations) … in the future, such capabilities will live in the “network cloud” and be available to the masses. With such, the future will not be like Tom Cruise in the movie “Minority Report” where he stands in front large screens and searches for data using his hands to navigate. Nope, that would be old school.

2050 Prediction: Collective intelligence will locate what you need to know … and tell you!

To bring this prediction to life I presented the following example: The collective intelligence system in the network cloud will know the precise geo-coordinates of where you are physically standing right now. As well, imagine if it is also amassing data related to the current activities of migratory birds. While at one moment these two data sets may have no nexus … the moment a sensor presents some wind speed data to the collective intelligence system … this new data point being the puzzle piece that connects your location data and the birds’ migratory data. The cloud then will immediately push real-time relevant insight to you, the consumer. In this case, you are told to “jump to the right one foot.” Seconds later, to the your delight, a bird poop falls from the sky … splats on the ground … exactly where you were standing! 

6. When collective intelligence serves you and your doctor … you are going to love it. But when it serves the police looking at you … you are going to hate it. 

7. And that is the truth about the future … it’s going to be love/hate. 

Here is a related news story:

C|Net – Imagining the Tech World in 2050

Here is a related 7 minute YouTube video:

Imaging the World of 2050

Data Decommissioning – Destruction of Accountability

Having designed a lot of systems over the years – more often than not the customer says they plan on performing periodic purges of historical data. This always seems logical at the time. But, it turns out once you have data it becomes hard to justify its destruction. And if anyone actually destroys data … one is at the same time eliminating any accountability whatsoever (not to mention other adverse consequences).

Data decommissioning is a double-edged sword.

After a number of personal missteps over the years, I have revised my think about data decommissioning. Today I imagine a process where accountability is maximized while the risk of unintended disclosure, misuse, and repurposing are minimized. The goal being to write accountability data into storage de-optimized for information retrieval … therefore rendering retrieval practical only for infrequent, forensic inspection. In simple terms, think paper tape, think hard copy reports, or think microfiche. Alternatively, in more sophisticated settings, I suspect immutable audit logs optimized only for investigative/forensic-specific information retrieval might be useful too. [More detail about this line of thinking available in this paper that Peter Swire and I penned on behalf of the Markle Foundation.] Obviously, at some point in time when there is no longer any reasonable expectation of information accountability, repeatability, etc. wholesale data decommissioned makes sense (burn the microfiche).

How I arrived at this revised thinking in part came about from this series of events.

Many years ago, I deployed a system designed to address a single, very specific threat. Then, several years later I concluded that long after that threat was over, the aggregated data set had probably lived on. I would not have thought twice about the privacy and civil liberties implications of this had I not started to engage in conversations with privacy advocates. Following these conversations, I decided that there are some scenarios in which data decommissioning should be "baked in."

Subsequently, with this in mind, when a pro bono opportunity to assist with a humanitarian disaster relief effort presented itself, I proposed a data destruction caveat for the contract. While the customer didn’t seem to care much one way or another, I was excited to learn the customer agreed to the wholesale destruction of the aggregated data set upon project closure. And delete it all we did.

A small victory for privacy it seemed – that is, until a few years later when I realized that I could no longer prove what was done, right or wrong. In fact, had there been any after-the-fact disputes about incorrect action taken based on the recommendations of the technology, I would have had to say, "We destroyed the evidence!"

In summary, when designing systems which require strong audit, accountability and repeatability processes … very careful consideration must be given to delete processes.

Deeper Technical Points:

1. Much like the challenges that come with processing deletes, record changes can have the same issues. This occurs when a system overwrites changes rather than keeping each incremental record state and its temporal relevance. When overwriting changes – one is deleting previous values; it is this de facto deletion that compromises audit and accountability processes.

2. A further complicating factor is that not all changes are the same. Some changes are corrections, i.e., the earlier value was incorrect, e.g., wrong driver’s license number or a missing apartment number in an address. Another type of change is one where a value supersedes a previous value, e.g., when recording a married name, new email address, or new cell phone number. Further complicating matters, most systems of record do not have a mechanism to capture the difference between corrections and updates – forcing system designers to make some assumptions.

3. When synchronizing data across information sharing environments, propagating deletes through this ecosystem forces each receiving party into this same accountability dilemma.

Related Trivia:

1. When data actually does get purged it is often prompted by a forcing-function. The three purge scenarios I have seen are: (a) all the ancient history is compromising performance; (b) there is no interest in paying for more storage; and (c) "oops - we shouldn’t have been collecting that!"

2. With all the countless copies of data being made, how can one be sure it is ever all deleted anyway?

RELATED POSTS:

Data Tethering

Out-bound Record-level Accountability in Information Sharing Systems

Information Incontinence

Immutable Audit Logs (IAL’s)

How Many Copies of Your Data? Is Somewhat Like Asking: How Many Licks to the Center of the Tootsie Pop?

Information Incontinence

I was on a call the other day working on a family project when the other party asked for my cell phone number. I handed it over on two conditions: (1) she throw it away after the project was completed, and (2) I made her swear to not enter my cell phone number into any computer. Immediately following this conversation my girlfriend overheard me muttering, "Computers are dangerous." Let me explain.

When it comes to preventing information leakage … the best rule is:  "Don’t ever let the data be placed into digital form."

Then for extra protection it is best not to ever speak it.  And, in coming years, it will be best not to ever think it either. (See P300 post below)

RELATED POSTS:

P300 "Brain Fingerprinting": A Very Freaky Future Indeed

How Many Copies of Your Data? Is Somewhat Like Asking: How Many Licks to the Center of the Tootsie Pop?

Van Halen, Risk Management and Breaking the Law (Allegedly)

One of the freedoms we have is the freedom (ability) to knowingly bend or break a law.

While in New York this week, I discovered that Van Halen was playing Madison Square Garden Tuesday, November 13th! Back in the day when I used to play guitar, Eddie Van Halen was like a super hero to me. Unfortunately, the concert was sold out.

Sold out or not – I decided I was going, one way or another. After checking Craigslist without luck and checking with the hotel concierge who found a pair for $1400.00, I decided to take matters into my own hands.

9:02pm - Madison Square Gardens

I arrived at the curbside with a load of cash on hand looking for a scalper. The police were everywhere. I stumble immediately into an interesting character who claims to have one ticket. When I ask him how much, he says $350. I say "deal!" And with great disregard for scalper laws and the countless police all about, I pulled out my wad of $20 bills and counted them off … all in plain sight.

Allegedly, of course.

I inspect the ticket for signs of being a forgery and accept it. He pockets the cash, and then pulls out his wallet while saying "I have something else for you." I briefly wondered if I had lucked into an undercover policeman! Nope, handing me his card he says "Call me anytime you want a ticket here." Then he says, "Heck for the price you just paid, I'll walk you to the front door."

9:15pm – I'm in the concert!

Allegedly.

Found: An Immutable Audit Log

An immutable audit log is a tamper-resistant recording of how a system has been used – everything from when data arrives, changes, departs, to how users interacted with the system. Each event is recorded in an indelible manner - even the database administrator with the highest level of system privileges cannot alter the past … kinda like the paper tape on an adding machine tape, etched in stone … only more high-tech.

I think (and hope) tamper-resistant audits will become common place in settings ranging from health care patient records to government surveillance systems. The primary value being twofold:

a) Accountability. Enable policy folks charged with oversight and accountability to validate that a computer system has been used within policy and law: and,

b) Deterrence. The "chilling effect" caused by the knowledge that a tamper resistant audit log is in place – deterring a corrupt person or two from bad behavior.

Well, good news. I stumbled onto a software company in Spain called Kinamik which has been dedicating its technical resources towards the creation of … a tamper-resistant audit log!

Now what? What if no one wants to pay for one? Will tamper resistant audit logs need to be built-in to commercial off-the-shelf systems to reach the market? If so, will organizations actually pay for the additional disk space and processing requirements to turn such a log on? Or, will they simply turn the feature off?

This is important technology and one that really needs to see the light of day, especially in conjunction with non-transparent government systems.

If any of my readers have thoughts as to what kind of incentives or levers will be needed to make such audit logs a reality, I would love to hear from you. As well, if you discover any other companies selling tamper-resistant logs, please let me know. I would like to compile a list.

RELATED POSTS:

Yesterday’s Technology Review Story: Blinding Big Brother, Sort of

Immutable Audit Logs (IAL’s)