Jeff Jonas

Self-Correcting False Positives/Negatives: Exonerate the Innocent

This blog entry is dedicated to false positives and false negatives, specifically why it is so essential that systems find and fix them.  A false negative occurs when an assertion about something true, is missed e.g., the true perpetrator of a crime is overlooked.  A false positive occurs when an assertion is made about something being true when, in fact, it is not true, e.g., when a court convicts someone for a crime he/she did not commit.

Imagine for a moment a DNA database.  What if a subject’s second DNA profile does not match the subject’s previously collected DNA profile? Would anyone notice this false negative?  Or, what if a DNA profile from a new criminal investigation is submitted to the DNA database and it matches someone who has been in prison for 15 years.  Is this proof of a false positive?

How could such errors happen?  Contamination?  Incompetence?  Criminal negligence?  Regardless, such tragic things really do happen.

Las Vegas police reveal DNA error put wrong man in prison

“It did not catch the mistake in Jackson's case, however, because the mistake was not merely a mislabel -- it was the wrong DNA in the wrong vial.”

DNA EVIDENCE: Officials admit error, dismiss case

“… acknowledging that the police lab accidentally had placed Sotolusson's name on another man's DNA sample.”

Asheville area murder case turns on missteps

“… revelations DNA evidence that might have cleared the men never made it to defense attorneys.”

Now the million dollar question: If a DNA database like the Combined DNA Index System (CODIS) were to get new evidence that brought doubt to an earlier assertion; can the system itself detect it?  Is anyone notified?  Better yet, is the defendant notified?

Systems with the ability to use new observations to reverse earlier assertions are able to self-correct false negatives and self-correct false positives – in essence, changing their minds about the past.  Yes … smart systems flip flop.

Traditional systems are generally unable to use new observations to reverse earlier assertions and thus end up with internal inconsistencies – the evidence exists right before your eyes (in the database) while an errant assertion from the past lives on.  Some call this “database drift.”

One common remedy to database drift involves periodically re-processing all the data.  Which begs the question: How long do you want to wait for a right answer, when the correct answer is in hand, and right before your eyes?  The other big issue is that if your system requires periodic reprocessing to correct for this database drift, how long is that going to take?  Point being, in very large databases this can take a very long time.

I have been extremely interested in how to prevent this type of database drift – detecting previous false negatives and false positives at the split second each new observation arrives – doing this over billions of rows of historical data while sustaining thousands of transactions a second.

In essence, this kind of algorithm works like this: Now that I know this, had I known this first, are there any assertions I have made that would have been made differently – in real-time at high transaction rates over ALL historical data?

From a privacy and civil liberties perspective, systems incapable of correcting previous false negative and false positives are problematic e.g., an innocent person remains harmed (incarcerated) despite the fact that late-arriving evidence clears his/her name.

Fortunately for some wrongly convicted, the Innocence Project is addressing this problem.  This organization uses DNA evidence to exonerate the wrongfully convicted.  They have nearly 300 examples (like Michael Morton) of folks wrongly imprisoned – in some cases these innocent folks have been incarcerated for many decades.  This amazing group does this by hand, it takes a lot of work, and they have an enormous backlog of cases worth reviewing.

I encourage those working on identity-based assertion systems to add self-correcting false negatives and self-correcting false positives to your systems.

[TECHNICAL NOTES]

Technical Note #1: A Mini-Demonstration of a Self-Correcting False Positive

I first stumbled across the need for self-correcting false positives about a decade ago when we discovered a particular data set in which three out of every eight million people, a Patrick and a Patricia, lived at the same address with the same last name.

Imagine what might happen in this scenario.  You already have this one record in your database:

Entity #1

Record 1, Patrick Smith, Male, 123 Main Street, 555-1212

And today you get Record 2 containing:

Pat Smith, 123 Main Street, 555-1212 and date of birth 03/17/1986

Let’s say for the sake of argument that same name, address and phone number are deemed sufficient evidence to assert that an entity is the same (not always a great idea by the way).  The results:

Entity #1

Record 1, Patrick Smith, Male, 123 Main Street, 555-1212

Record 2, Pat Smith, 123 Main Street, 555-1212 and date of birth 03/17/1986

This is fine and dandy and all … until Record 3 appears on the scene:

Patricia Smith, Female, 123 Main Street, 555-1212, and date of birth 03/17/1986

At this moment, one has an opportunity to recognize the earlier error … in this case, the Pat Smith record is now most likely to be Patricia, NOT Patrick.  At this split second, a system capable of self-correcting false positives fixes the past replacing it with these new assertions:

Entity #1

Record 1, Patrick Smith, Male, 123 Main Street, 555-1212

Entity #2

Record 2, Pat Smith, 123 Main Street, 555-1212, and date of birth 03/17/1986

Record 3, Patricia Smith, Female, 123 Main Street, 555-1212, and date of birth 03/17/1986

 

Technical Note #2: A More In-depth Look at CODIS Processes

Over the course of my research, I was fortunate to come across background information about CODIS.  Some may find this interesting or helpful.

CODIS has three tiers:  LDIS (local lab databases), SDIS (statewide databases), and NDIS (USA database).  Selective LDIS records roll-up to SDIS and selective SDIS records roll-up to NDIS.

The criteria to enter the LDIS tier are the least stringent while the criteria to reach the NDIS level are the most stringent.  A profile in the NDIS database will have come from a SDIS database.  And entries in a SDIS database may have come from a LDIS database below it, possibly varying by each state’s lab structure/organization.

Each tier is broken down further into several indices:  Forensic, Convicted Offender, Arrestees (depending on state legislation), Missing Persons, Relatives of Missing Persons, and Unidentified Human Remains (LDIS, SDIS, NDIS).  Some LDIS/SDIS databases also have a Volunteer and or Suspect Index (depending on respective local/state legislation).  

DNA Profile Submission

Each sample’s profile is generally only eligible for submission to a single index.  If a forensic profile matches an individual and is uploaded to the forensic index, that individual’s reference profile can’t simply be added to the convicted offender index later after conviction.  Also, the reference sample collected for comparison to the evidence profile cannot be retested to re-obtain the individual’s profile for convicted offender submission either—an entirely new sample is collected upon conviction for submission to the convicted offender index. 

The only information contained in each of these databases is:

a) the DNA profile

b) the lab submitting the profile

c) the lab employee responsible for submitting the profile

d) some numerical identifier for the submitting lab/employee to track the profile

CODIS Search Procedures

Only certain index combinations are compared against each other.  For example, the Relatives of Missing Persons Index is only compared to the unidentified Human Remains Index while the Convicted Offender Index is compared against all indices except the Relatives of Missing Person’s index.

• If a profile only meets LDIS criteria, it resides in the LDIS database and is only compared within the appropriate indices to other LDIS profiles  (profiles belonging to the individual laboratory)
  • For example, if a profile meets Miami’s LDIS upload criteria, but not Florida’s SDIS criteria, the profile would simply reside in and be compared against the appropriate indices within Miami’s LDIS.
  • If the profile meets SDIS criteria, it is compared within the appropriate indices to other profiles in both the respective LDIS and SDIS databases but not outside LDIS databases within the state (Put differently, a profile that has reached SDIS will not be compared against other states’ LDIS or SDIS profiles.)
    • For example, if a profile meets both Miami’s LDIS and Florida’s SDIS upload criteria, but not NDIS’s, the profile would reside in both the LDIS and SDIS and be compared against the appropriate indices at both levels. It would not, however, be compared against any profiles contained in NDIS, nor would it be compared against the LDIS and SDIS of other states, say New Orleans’s LDIS and Louisiana’s SDIS.
    • If the profile meets NDIS criteria, it is compared to other profiles within the appropriate indices in the respective LDIS, SDIS, and NDIS databases, but not outside SDIS/LDIS databases
      • For example, if a profile meets Miami’s LDIS, Florida’s SDIS, and NDIS upload criteria, it would reside at all three levels and be compared against appropriate indices at each.  It would not, however, be compared against profiles residing only in other states’ LDIS or SDIS systems.

CODIS Matches and DNA Identification Errors

The CODIS software performs comparisons between the appropriate indices/tiers of the databases and then reports back any matching profiles with their associated identifiers to the submitting lab(s).  Then the lab(s) must follow up on their end to retest their matching sample(s) and report back the confirmation results to each other (the exception being forensic profiles are NOT re-tested to avoid consumption of evidence).  If the source (DNA donor) of the matching DNA profiles is known, this info is exchanged between the individual submitting lab(s) and it is at this time a “false-positive” match is discovered.  

Basically, this type DNA identification error can only be caught after a match has occurred – meaning the falsely matching profiles have to wind up in an index/tier where they will be compared to each other … an erroneous LDIS profile residing in the SDIS database will not be detected if the falsely matching profile resides in the NDIS database, a different SDIS database, or even an unsearchable index in the same SDIS!  Note: this can take an unreasonably long time, if it ever happens at all. 

Additionally, since the database contains minimum identifying attributes for each entity or DNA profile, and the attributes mostly contain information that can only be utilized by the submitting lab, the CODIS software doesn’t have any capability to detect “false negative” non-matches.  It’s important to note that this inherent limitation was developed intentionally to protect individuals’ privacy … but as a consequence this may lead to erroneous identification and wrongful incarceration. 

Here is another interesting weakness in CODIS.  When a subject’s DNA becomes crime scene evidence, this DNA profile may possibly appear in the databases up to four times over the entire course of a single investigation:  First as a forensic sample (the evidence profile attributed to him), second as a volunteer (the reference specimen he consented to database entry according to local/state law), third as an arrestee (the reference specimen collected from him when arrested for a qualifying offense according to local/state law), and fourth as a convicted offender (the reference specimen collected from him when convicted of a qualifying offense according to local/state/federal law). 

Matches should occur at each of these stages of submission if there are no DNA identification errors, and multiple matches would definitely validate each individual testing result.  However, if a match does not occur at one or more of these submission stages, then an error has occurred.  After the forensic profile was uploaded, each subsequent related submission provided an opportunity to detect an identification error because each subsequent match reported by CODIS would be returned to the lab for confirmation.  CODIS’s inability to detect a falsely-negative non-match at any and each of the submission stages means opportunities to detect errant results are missed – this faulty evidence means the wrong people are convicted or freed. 

Even if CODIS had the capability to detect/correct a false positive, waiting for that false-positive match to occur is problematic because it takes longer to occur/detect than a false-negative non-match, or even worse, might never occur at all.  To obtain the false positive association, the true DNA donor to the forensic profile must somehow get uploaded to an index/tier that will be compared to the forensic profile that was erroneously attributed to the original subject.  The match might occur immediately if the true donor’s profile is already there, but might never occur if it’s residing in an ineligible index/tier or the true donor’s profile is forever absent altogether.

Basically, detecting/correcting the false-positive match prevents a subject from being wrongfully convicted for the second crime, but might only minimally exonerate him for the first wrongful conviction (he may have already served his sentence, be deceased, etc.).  A false-negative non-match actually precludes a false-positive match unless the falsely matching profile is immediately available for comparison when the erroneous profile is uploaded to the database. 

 

RELATED POSTS:

Smart Systems Flip-Flop

Smart Sensemaking Systems, First and Foremost, Must be Expert Counting Systems

 

Big Data Q&A for the Data Protection Law and Policy Newsletter

I have been given permission to re-publish an interview I did with the Data Protection & Law Policy newsletter.  Also to appear in the e-Finance and Payments Law & Policy newsletter.

May be of interest to some of my readers.

[INTERVIEW]

1. Data protection challenge of the future: what is Big Data?

The three V’s - Volume, Velocity, and Variety – are the essential characteristics of “Big Data”. While data protection and privacy laws are still busy catching up with technologies of yesterday, Big Data is growing at a lightning speed on a daily basis. How can companies deal with the data protection challenges brought about by Big Data in order to truly benefit from the opportunities introduced by Big Data? First, one must truly grasp what is Big Data. We interview Jeff Jonas, Chief Scientist at IBM Entity Analytics, to obtain his perspectives and definition of Big Data, and his experience handling Big Data. 

2. When did data become big?

Big Data did not become big overnight.  What I think happened is data started getting generated faster than organizations could get their hands around it.  Then one day you simply wake up and feel like you are drowning in data.  On that day, data felt big.

3. Please explain and elaborate on the characteristics of Big Data?

Big Data means different things to different people.

Personally, my favorite definition is: “something magical happens when very large corpuses of data come together.”  Some example of this can be seen at Google, for example Google flu trends and Google translate.  In my own work, I witnessed this first in 2006.  In this particular system, the system started getting higher quality predictions and faster as it ingested more data.  This is so counter intuitive.  The easiest way to explain this though is to consider the familiar process of putting a puzzle together at home.  Why is it do you think the last few pieces are as easy as the first few – even though you have more data in front of you then ever before?  Same thing really that is happening in my systems these days.  It’s rather exciting to tell you the truth.

To elaborate briefly on the new physics of Big Data, I pinpointed the three phenomena of Big Data physics in my blog entry - Big Data. New Physics – drawing from my personal experience of 14 years of designing and deploying a number of multi-billion row context accumulating systems:

1. Better Prediction.  Simultaneously lower false positives and lower false negatives

2. Bad data good.  More specifically, natural variability in data including spelling errors, transposition errors, and even professionally fabricated lies – all helpful. 

3. More data faster.  Less compute effort as the database gets bigger. 

Another definition of Big Data is related to the ability for organizations to harness data sets previously believed to be “too large to handle.” Historically, Big Data means too many rows, too much storage and too much cost for organizations who lack the tools and ability to really handle data of such quantity. Today, we are seeing ways to explore and iterate cheaply over Big Data.

4. When did data become big for you? What is your “Big Data” processing experience?

As previously mentioned, for me, Big Data is about the magical things that happen when a critical mass is reached.  To be honest, Big Data does not feel big to me unless it is hard to process and make sense of.  A few billion rows here and a few billion rows there – such volumes once seemed a lot of data to me. Then helping organizations think about dealing volumes of 100 million or more records a day seemed like a lot.  Today, when I think about the volumes at Google and Facebook, I think: “Now that really is Big Data!”

My personal interest and primary focus on Big Data these days is: how to make sense of data in real time, that is fast enough to do something about the transaction while the transaction is still happening. While you swipe that credit card, there is only a few seconds to decide if that is you or maybe someone pretending to be you.  If an unauthorized user is inside your network, and data starts getting pumped out, an organization needs sub-second “sense and respond” capabilities. End of day batch processes producing great answers is simply late!

5. What are the technologies currently adopted to process Big Data?

The availability of Big Data technologies seems to be growing by leaps and bounds and on many fronts.  We are seeing a large corporate investments resulting in commercial products – at IBM two examples would be IBM InfoSphere Streams for Big Data in motion and IBM InfoSphere Big Insights for pattern discovery over data at rest.  There are also many Big Data open source efforts under way for example HADOOP, Cassandra and Lucene.  If one were to divide these into types one would find some well suited for streaming analytics and others for batch analytics.  Some help organizations harness structured data while others are ideal for unstructured data.  One thing is for sure – there are many options, and there will be many more choices to come as Big Data continues to get investment.

6. How can companies benefit from the use of Big Data?

I’d like to think consumers benefit too, just to be clear.  To illustrate my point, I find it very helpful when Google responds to my search with “did you mean ______”.  To pull this very smart stunt, Google must remember the typographical errors of the world, and that I do believe would qualify as Big Data.  Moreover, I think health care is benefiting from Big Data, or let’s hope so.  Organizations like financial institutions and insurance companies are benefitting from Big Data also by using these insights to run more efficient operations and mitigate risks.

We, you and I, are responsible in part for generating so much Big Data.  These social media platforms we use to speak our mind and stay connected are responsible for massive volumes of data.  Companies know this and are paying attention.  For example, my friend’s wife complained on Twitter about a specific company’s service.  Not long thereafter they reached out to her because they too were listening.  They fixed the problem and she was as happy as ever.  How did the company benefit? They kept a customer.

7. What is the trend of processing Big Data?

I think a lot of Big Data systems are running as periodic batch processes, for example, once a week or once a month.  My suspicion is as these systems begin to generate more and more relevant insight, it will not be long before the users say: “Why did I have to wait until the end of the week to learn that?  They already left the web site.”; or, “I already denied their loan when it is now clear I should have granted them that loan.”

8. What are the complications dealing with the privacy implications brought about by Big Data compare to average sized data?

There are lots of privacy complications that come along with Big Data.  Consumers, for example, often want to know what data an organization collects and the purpose of the collection.  Something that further complicates this: I think many consumers would be surprised to know what is computationally possible with Big Data.  For example, where you are going to be next Thursday at 5:35pm or your three best friends, and which two of them are not on Facebook. Big Data is making it harder to have secrets. To illustrate using lines from my blog entry - Using Transparency As A Mask – ‘Unlike two decades ago, humans are now creating huge volumes of extraordinarily useful data as they self-annotate their relationships and yours, their photographs and yours, their thoughts and their thoughts about you … and more. With more data, comes better understanding and prediction.  The convergence of data might reveal your “discreet” rendezvous or the fact you are no longer on speaking terms your best friend.  No longer secret is your visit to the porn store and the subsequent change in your home’s late night energy profile, another telling story about who you are … again out of the bag, and little you can do about it.  Pity … you thought that all of this information was secret.’

9. What are the privacy concerns & threats Big Data might bring about - to companies and to individuals whose data are contained in 'Big Data'?

My number one recommendation to organizations is “Avoid Consumer Surprise.” 

That said, my concern is many consumers don’t seem to give a hoot. When is the last time you actually read the privacy statement or terms of use on your favourite social media site? I think in the future we’ll see Big Data being used to make the services offered even more irresistible.  Your Internet searches will become custom crafted lenses.  As a student of privacy and someone building Privacy by Design (PbD) into my inventions, I think about these things all the time.

10. How are companies currently applying privacy protection principles before/after Big Data has been processed?

I think there are many best practices being adopted.  One of my favorites involves letting consumers opt-in instead of opting them in automatically and then requiring them to opt-out.  One new thing I would like to see become a new best practice is: a place on the web site, for example my bank, where I can see a list of third parties whom my bank has shared my data with.  I think this transparency would be good and certainly would make consumers more aware.

11. What is “Big Data”, according to Jeff Jonas?

Big Data is a pile of data so big - and harnessed so well - that it becomes possible to make substantially better predictions, for example, what web page would be the absolute best web page to place first on your results, just for you.

 

G2 | Sensemaking – One Year Birthday Today. Cognitive Basics Emerging.

Following a quiet two-plus year gestation period, G2 came to life January 28^th, 2011 – one year ago today – when I formally announced its existence here: Sensemaking on Streams – My G2 Skunk Works Project: Privacy by Design (PbD).

“This new technology, something that might be characterized as a “big data analytic sensemaking” engine, is designed to make sense of new observations as they happen, fast enough to do something about it, while the transaction is still happening.”

Oh, the difference a year makes.  Over the last twelve months G2 has evolved in many ways. Three of the more exciting characteristics to emerge are:

Orders of Magnitude, appreciation of

G2 now has the ability to consider proportions when determining the best way to count things … evolving counting methods in real-time, without external influence (humans).  There is a big difference between zero, one, two, three, etc.  But at some point why take the effort to increment a counter from 4,090,223 to 4,090,224?  Why is this important?   Imagine how much attention it would take if you had to keep an exact count of everything you had ever seen.  What a waste.  In the interest of saving its energy for more important tasks, G2 starts using orders of magnitude to optimize its resources.  Of course, G2 can be told to keep exact counts (despite scale); if an exact count is needed from time to time, G2 will count them, one-by-one.

Geospatial and Temporal Reasoning, use of

G2 is showing early promise in the area of geospatial and temporal reasoning.  Using the basic physics principle “Two different things cannot occupy the same space at the same time,” G2 can more accurately assert when things are the same.  Also, G2 can now tell the difference between something happening “here and now” versus “then and there.”  If you were reading a book about the 1906 fire that swept through San Francisco, you would not be freaking out about the danger, filling the bathtub and watering the roof of your house.  No, you understand that this is information about “then and there,” not “here and now.”  Why is this important?  When it comes to making recommendations about a “next best action,” if one doesn’t take into account geospatial and temporal factors, be prepared for a slew of bogus recommendations.

Confusion, awareness of

And, just today, G2 suddenly developed the ability to notice when it becomes confused about something, making “note to self” about this important fact.  This awareness of confusion is triggered when facts appear to disagree.  If someone has three phone numbers – no big deal.  On the other hand, if someone has five different dates of birth, that just doesn’t seem quite right does it?  That would be confusing.  Why is this important?  Well, if you are looking to analytics to make important decisions, wouldn’t you want to know during the decision making process if there was related confusion … before action is taken?

While G2 is showing no signs of evolving any motor function any time soon … the cognitive advances we are seeing and the benefits we hope this will bring society … make these exciting times.

Happy Birthday G2!

 

Sincerely,

 

Jeff Jonas and the entire

G2 Development Team

 

RELATED POSTS:

Sensemaking on Streams – My G2 Skunk Works Project: Privacy by Design (PbD)
Smart Sensemaking Systems, First and Foremost, Must be Expert Counting Systems
On A Smarter Planet … Some Organizations Will Be Smarter-er Than Others
Data Finds Data
Accumulating Context: Now or Never
General Purpose Sensemaking Systems and Information Colocation
Big Data. New Physics.
Master Data Management (MDM) vs. Sensemaking

 

Sensemaking on Streams – My G2 Skunk Works Project: Privacy by Design (PbD)

Over the last twenty-eight months I have been quietly running a skunk works effort that I’ve code named “G2.”  To my delight, on January 28th, 2011 this system became officially viable and will be entering something akin to a “sea trial” phase through 2011.

I believe this system will prove to be my most innovative work to date.  I also believe it is the most responsible technology I have created to date.

This new technology, something that might be characterized as a “big data analytic sensemaking” engine, is designed to make sense of new observations as they happen, fast enough to do something about it, while the transaction is still happening.  This engine brings to life many of the principles I have been openly sharing on my blog, ranging from Sensemaking Systems Must be Expert Counting Systems, Data Finds Data, Context Accumulation, Sequence Neutrality and Information Colocation to new techniques to harness the Big Data/New Physics phenomenon.  That said, as this is version 1.1, there remain many things to do to realize my full vision.  It is a very ambitious effort, but more about that some other day. 

In terms of responsible innovation, I am even more proud to report that my team and I have baked in, from conception, more privacy and civil liberties enhancing technologies than any other product I am aware of to date. 

Friday, January 28^th, 2011 – my official launch date – also happened to be the international Data Privacy Day.  And on this day, internationally recognized privacy commissioner, Ann Cavoukian hosted a few hundred privacy executives and practitioners from around the world in Toronto Canada at her Privacy by Design: Time to Take Control conference.  During my keynote entitled “Confessions of an Architect” I highlighted seven (7) exciting features that have been baked into this new technology (Privacy by Design), specifically:

1. Full Attribution

2. Data Tethering

3. Analytics in the Anonymized Data Space

4. Tamper-Resistant Audit Logs

5. False Negative Favoring Methods

6. Self-Correcting False Positives

7. Information Transfer Accounting

The full presentation is here.

Here is a summary of the above seven PbD features:

1. FULL ATTRIBUTION: Every observation (record) needs to know from where it came and when.  There cannot be merge/purge data survivorship processing whereby some observations or fields are discarded.  Why is this so important?

A. If received data does not contain its data source and transaction pedigree, then system-to-system reconciliation and audit are virtually impossible, especially in large information sharing environments.

B. If the system merges and purges observations, only later to discover the wrong observations were merged or purged, then without full attribution correcting these earlier mistakes can be difficult if not impossible.  The typical alternative being periodic batch re-processing.

C. The Universal Declaration of Human Rights has four articles containing the word “arbitrary” e.g., Article 9 reads “No one shall be subjected to arbitrary arrest, detention or exile.”  If you don’t know where the data came from or when, how can any resulting action be anything but arbitrary?

2. DATA TETHERING: Adds, changes and deletes occurring in systems of record must be accounted for, in real-time, in sub-seconds.  Why is this so important?

A. Data currency in information sharing environments is important, especially if one is using data to make important, difficult to reverse decisions that affect people’s freedoms or privileges.

B. When derogatory data is removed or corrected in a system of record, it is vital to reflect such corrections immediately.  For example, if someone is removed from a watch list, how long should they have to wait before their name is cleared?

3. ANALYTICS ON ANONYMIZED DATA: The ability to perform advanced analytics (including some fuzzy matching) over cryptographically altered data means organizations can anonymize more data before information sharing.  Why is this so important?

A. With every copy of data, there is an increased risk of unintended disclosure.

B. Data anonymized before transfer and anonymized at rest reduces the risk of unintended disclosure.

C. If organizations can now share information in an anonymized form and still get a materially similar result, why would organizations want to share information any other way?

[Technical Note: As every anonymized value maintains full attribution, re-identification is by design to support Data Tethering as well reconciliation and audit.]

4. TAMPER-RESISTANT AUDIT LOGS: Each record of who searches for what should be logged in a tamper-resistant manner – even the database administrator should not be able to alter the evidence contained in this audit log. Why is this so important?

A. Every now and then people with access and privilege take a look at records without a legitimate business purpose, e.g., should an employee at a financial services institution take a peek into their roommate’s file.

B. Tamper-resistant logs make it possible to audit user behavior.

C. And, when the word gets out to the work force that such accountability exists, this can cause a chilling effect on misuse.

5. FALSE NEGATIVE FAVORING METHODS: The ability to more strongly favor false negatives is of critical importance in systems that could be used to affect someone’s civil liberties.  Why is this so important?

A. In many business scenarios, it is better to miss a few things (false negatives) than inadvertently make claims that are not true (false positives).  False positives can feed into decisions that adversely affect people’s lives – e.g., the police find themselves knocking down the wrong door or an innocent passenger is denied the ability to board a plane.

[Technical Note: Sometimes a new observation can lead to multiple conclusions.  Systems that are not false negative favoring may select the strongest conclusion and ignore the remaining conclusions.  But had the strongest candidate not existed, the second strongest conclusion would be asserted.  One false negative favoring method involves remedy such a condition, for example by reversing an earlier conclusion should a future observation bring to light that fact that multiple possible conclusions now exist.]

6. SELF-CORRECTING FALSE POSITIVES: With every new observation presented, prior assertions are re-evaluated to ensure they are still correct, and if no longer correct, these earlier assertions can often be repaired – in real-time, not end of month.  Why is this so important?

A.  False positives occur when an assertion (claim) is made, but is not true.  If relied upon to make a decision, false positives can adversely affect people’s lives e.g., consider someone who cannot board a plane because he or she shares a similar name and date of birth as someone else on a watch list.

B. Without self-correcting false positives, databases start to drift from the truth and become provably wrong (even to the naked eye) – necessitating periodic (batch) reloading to true-up the database.

C. Periodic monthly reloading to correct for false positives means wrong decisions are possible all month until the next reload, even though the system had everything it needed to know beforehand.

[Technical Note: Reversing earlier assertions in real-time at scale, as new observations present themselves, is computationally non-trivial.  Imagine making an assertion that two people are the same because they share exactly the same name, address and home phone number – only later to learn through another series of observations that these are really two different people (a junior and a senior).  Our “self-correcting false positives” feature self-corrects for these rare cases, in real-time.  We consider our ability to perform sequence neutrality at scale one of several breakthrough aspects of our work.]

7. INFORMATION TRANSFER ACCOUNTING: Every secondary transfer of data, whether to human eyeball or tertiary system, can be recorded to allow stakeholders (e.g., data custodians or the consumers themselves) to determine how their data is flowing.  Why is this so important?

A. It is often cumbersome to learn who has seen what records, or what records have been shared with tertiary systems.

B. Much like a US credit report that contains an inquiries section exposing the list of recent inquiring parties, now so can your medical or financial file.

C. Users can now be easily provided with such disclosures, increasing transparency and control e.g. enabling a consumer in some cases to request an information recall.

D. When there is a series of leaks, information transfer accounting makes discovery of who accessed all records in the series quite trivial.  This can narrow an investigation when looking for criminals within.

What has me most excited is that where some features above would typically be an extra priced option in my new system so many are built in (e.g. this tamper-resistant audit logs).  And some of our privacy and civil liberties enhancing features cannot even be turned off!

Yes, there is an official name for my new technology.  And no, I’m not telling you, because this is not a sales pitch.  Rather, I am simply trying to inspire other technologists to consider Privacy by Design as they innovate.

I’ve had two most great days at IBM.  The first great day was in January 2005 when IBM bought my company, SRD.  And the second greatest day came six years later on January 28^th, 2011.

RELATED MATERIAL:

Privacy by Design (PbD)

 

RELATED POSTS:

Smart Sensemaking Systems, First and Foremost, Must be Expert Counting Systems

Data Finds Data

Accumulating Context: Now or Never

Sequence Neutrality

General Purpose Sensemaking Systems and Information Colocation

Big Data. New Physics.

Source Attribution, Don’t Leave Home Without It

Data Tethering: Managing the Echo

To Anonymize or Not Anonymize, That is the Question

Immutable Audit Logs (IAL’s)

Out-bound Record-level Accountability in Information Sharing Systems

Big Data Flows vs. Wicked Leaks

Decommissioning Data: Destruction of Accountability

Big Data Flows vs. Wicked Leaks

I was invited to deliver a short keynote about "big data" at today's OECD roundtable focused on the economics of personal data and privacy.  My presentation here.

Most big data flows by design.  But when big data leaks the consequences can be wicked. 

That said … protecting big data from wicked leaks is not going to be easy.  Defending against external cyber penetrations and insider threats are both hard problems.

Now with the Wikileaks disclosures it is clear the game has changed.  Historically, public disclosure of classified data has been limited and infrequent – let’s even say, to a degree, tolerable.  Contrast that with the scope of the recently leaked cables.  At some point it may become intolerable.  In which case, I will not be surprised if a number of governments around the world attempt to enact new, wide-sweeping, anti-leak legislation directed at not only those engaged in the initial theft of the data, but the distribution points (e.g., Wikileaks) and the publishers (e.g., the media).  The principle being if one knowingly receives and benefits from stolen property, they are accomplices.  This pendulum could swing so far (backwards) the future will have far fewer media leaks than the historical (tolerable) volumes – i.e., this whole fiasco resulting in less transparency and accountability.

BTW: I suppose it could have been worse.  What if the 250,000 classified cables where selectively and quietly passed around to various foreign intelligence services?  What if the US thought these were secrets?  Imagine believing one has a certain security posture … when one does not.  Would that be worse? 

Organizations with big data worth protecting must employ extraordinary controls to reduce the risk of unintended disclosure.  On that note, I closed with a few ideas related to protecting big data from wicked leaks, including:

Central indexes.  There are actually a number of scenarios where a single, central catalog of pointers is better than lots of copies of the same data scattered all over the place – the advantage being fewer copies of the data and uniform access controls and audit logs.

Anonymization.  Despite the imperfections of data anonymization, when it comes to reducing the risk of unintended disclosure, most would agree that data anonymization is still better than clear text.

Immutable audit logs.  Tamper resistant audit logs can be used to help prove the users of the system are complying with law and policy.

Real-time active audits.  It is now going to be essential that user activity be more rigorously analyzed, in real-time, for inappropriate behavior.  Audit logs have actually been part of the problem – just another big pile of data – evidence of misuse hiding in plain sight against the backdrop of millions and millions of benign audit records.

 

RELATED POSTS:

Big Data. New Physics.

More Death Cheaper in Future

How Many Copies of Your Data? Is Somewhat Like Asking: How Many Licks to the Center of the Tootsie Pop?

It’s All About the Librarian! New Paradigms in Enterprise Discovery and Awareness

"Need to Know" vs. "Need to Share" – A Very Fine Line Indeed

To Anonymize or Not Anonymize, That is the Question

Immutable Audit Logs (IAL’s)

Using Transparency As A Mask

Big Data. New Physics.

I spoke at Defrag 2010 earlier today and introduced what I am describing as the new physics of big data.

Having designed and deployed a number of multi-billion row context accumulating systems over the last 14 years I cannot help but notice some very interesting, very exciting phenomenology.  Not research.  Not Theory.  Real.

1. Better Prediction.  Simultaneously lower false positives and lower false negatives.  A bit more about this here: Prediction: Channel Consolidation and Puzzling: How Observations Are Accumulated Into Context. 

2. Bad data good.  More specifically, natural variability in data including spelling errors, transposition errors, and even professionally fabricated lies – all helpful.  A bit more about this here: It Turns Out Both Bad Data and a Teaspoon of Dirt May Be Good For You and There Is No Such Thing As A Single Version of Truth. 

3. More data faster.  Less compute effort as the database gets bigger.  A bit more about this most exciting phenomenon here: The Fast Last Puzzle Piece.

And more good news, as context accumulates, a better sense of when and where to place one’s attention (apply compute effort) develops -- including: (1) very smart observation filters and (2) fully automated ability to determine very specific, very relevant questions – answers to which it may decide to fetch itself.  A system that Googles itself?  I have not blogged about this thinking as of yet, but hopefully will one of these days.

Anyway, imagine that: As the database grows, fewer CPU cycles are needed for better predictions and you never really wanted to clean all that data up in the first place.

I also took this keynote opportunity to share my latest skunk works project – a project my team and I have been working on for almost two years now.  Yes, it’s true, I am building something – a sensemaking engine, designed to fully harness this big data phenomenon.  Among other exciting properties, this system will also have an unprecedented number of privacy-enhancing features baked into it.  Internally I have been calling this little skunk works effort “G2.”  And when this little girl grows up I have big hopes for her.  For example, maybe she will help cancer researchers find a cure.

My Defrag 2010 MS PowerPoint presentation here.

 

RELATED POSTS:

Algorithms At Dead-End: Cannot Squeeze Knowledge Out Of A Pixel

Smart Sensemaking Systems, First and Foremost, Must be Expert Counting Systems

Data Finds Data

Puzzling: How Observations Are Accumulated Into Context

Your Movements Speak for Themselves: Space-Time Travel Data is Analytic Super-Food!

“Macro Trends: The Privacy and Civil Liberties Consequences … and Comments on Responsible Innovation” – My DHS DPIAC Testimony, September 2008

Using Transparency As A Mask

[This is re-post.  I posted this on the Concurring Opinions blog August 2nd, 2010.]

As mankind deploys increasing numbers of sensors, and makes more sense of this data, more of our secrets are revealed.  In a world of greater transparency, will you be able to be you?  Or will you feel obligated to mask who you are, drawn to the safety of the center of the bell curve?

Will a more transparent society make you average?

Imagine for a moment that video feeds from street surveillance cameras are the blue puzzle pieces, your path through life lit up by your cell phone location as the green puzzle pieces and your Facebook social network as the yellow puzzle pieces.  Flicker the brown puzzle pieces and Twitter, orange puzzle pieces.  And maybe one day your energy consuming devices in your home may be spewing out the magenta puzzle pieces. As increasing volume and range of data converges, a colorful, highly revealing picture of our lives will unfold, with or without our knowledge or permission.  Traditional physical sensors like credit card and license plate readers are one thing.  The human is the sensor, thanks to Web 2.0, is altogether a different thing.

Unlike two decades ago, humans are now creating huge volumes of extraordinarily useful data as they self-annotate their relationships and yours, their photographs and yours, their thoughts and their thoughts about you … and more.

With more data, comes better understanding and prediction.  The convergence of data might reveal your “discreet” rendezvous or the fact you are no longer on speaking terms your best friend.  No longer secret is your visit to the porn store and the subsequent change in your home’s late night energy profile, another telling story about who you are … again out of the bag, and little you can do about it.  Pity … you thought that all of this information was secret.

How will mankind respond? Will people feel forced to modify their behavior towards normal only because they fear others may discover their intimate personal affairs?  This is what Julie Cohen and Neil Richards have worried about – the “chilling effect.”

Or, more optimistically, will the world become more tolerant of diversity?  Will we be willing to be ourselves in a more transparent society?

Personally, I shiver at the though of being on the hump … the hump of the bell curve.  I hope for a highly tolerant society in the future.  A place where it is widely known I am four or five standard deviations off center, and despite such deviance: my personal and professional relationships carry on, unaffected.

And oh, by the way, more goodness … diversity is good for resilience.

Miscellaneous: About the title of this post: I just thought it was a funny expression.  Other funny expressions I enjoy include:

    1) Kill all extremists.

    2) When you can fake sincerity, you have it made.

 

RELATED LINKS:

David Brin’s: The Transparent Society

 

RELATED POSTS:

Six Ticks till Midnight: One Plausible Journey from Here to a Total Surveillance Society

Santa’s Surveillance Operations Center Enjoys Big Gains in 2008(*)

Ubiquitous Sensors? You Have Seen Nothing Yet

USC School of Cinematic Arts, “Imagine the World in 2050″

Van Halen, Risk Management and Breaking the Law (Allegedly)

Transparency, Privacy and Responsibility

P300 “Brain Fingerprinting”: A Very Freaky Future Indeed

The Truth is Out There … Way Out There … and Some Times it Should Be Left There

Responsible Innovation: Designing for Human Rights

Podcast: The Future of Privacy

Your Movements Speak for Themselves: Space-Time Travel Data is Analytic Super-Food!

Puzzling: How Observations Are Accumulated Into Context

 

Open Government: The Privacy Imperative

O’Reilly Media published a book this February 2010 called Open Government: Collaboration, Transparency, and Participation in Practice.  In this book Jim Harper and I coauthor a chapter entitled: Open Government: The Privacy Imperative (pp 315-325).

Excerpt

“If participatory government is to have the confidence of citizens, it must be very sensitive to individual privacy and related values. Open government will not truly succeed if it is not welcoming to everyone. The most credible and successful open government systems must have the trust of all potential users, not only that of technophiles and the web generation. Open government systems must garner the trust of people who disagree with administration policies, people who distrust government generally, and people who are leery of the Internet.

As governments enter the 2.0 era, they must examine what information they collect from citizens and the rules governing how these data collections are used and controlled. A variety of techniques can protect privacy and foster the sense among all citizens that interacting with government will not expose them to adverse consequences, retribution, or negative repercussions. New and improved privacy practices can help fulfill the promise of open government.”

We go on to make specific recommendations in the areas of data minimization, anonymous access, data retention and decommissioning, and data tethering (maintaining integrity across the ecosystem as data is shared), and more.

The full chapter is available here (PDF pp 29-39) in this free sampler from O’Reilly. 

RELATED POSTS

To Anonymize or Not Anonymize, That is the Question

How Many Copies of Your Data? Is Somewhat Like Asking: How Many Licks to the Center of the Tootsie Pop?

Data Tethering: Managing the Echo

Full Attribution, Don’t Leave Home Without It

Out-bound Record-level Accountability in Information Sharing Systems

Data Decommissioning – Destruction of Accountability

Immutable Audit Logs (IAL’s)

Responsible Innovation: Evil is as Evil Does

The Fondazione Giannino Bassetti organization recently published “Conversations on Innovation, Power and Responsibility.”  In this document, author Jeff Ubois captured a few of my thoughts in a section entitled: “Applying the UN Declaration of Human Rights” (pp 31-33).   And as related to “is technology good or evil?” I said this:

 

Excerpt: “You probably can’t stop people from doing R&D to see what’s possible. But then how is it going to be deployed, what are its controls, what’s the oversight and accountability? And on and on, a series of checkpoints about how it’s going to be used,” Jonas says. “Until you pair the technology up with the use, you can’t even begin to assess its spectrum of good to evil. You have to go right to a use case and right to a governance model, and then you can start determining where it is going to sit on the spectrum of good versus evil.”

 

The entire publication is available in PDF here.

 

And then, of course, everyone can just change their mind and use something planned for one purpose for an entirely different purpose: A coat hanger to open a car door.

 

RELATED PAPERS: 

 

Giannino Bassetti Foundation: Jeff Ubios

Transparency, Privacy and Responsibility: An Interview with Jeff Jonas 

 

RELATED POSTS: 

Responsible Innovation: Designing for Human Rights

Responsible Innovation: Some Things are Best Left Un-invented

Responsible Innovation: Staying Engaged with the Privacy Community

My Testimony Before The DHS Data Privacy and Integrity Advisory Committee (2005)

Your Movements Speak for Themselves: Space-Time Travel Data is Analytic Super-Food!

It doesn’t matter who you say you are!  Where you are (space), when you’re there (time), and your movements over time (travel) are closer to the truth.

I’ve seen a lot of data in my life, and I’d like to think I have a decent grip on what can be accomplished with data and analytics.  However, I recently stumbled upon some facts that have radically reshaped my understanding of the world we are living in.  What I thought was years away is already here! Our toes are dangling over the edge of a very different future.

Now, before you get all worked up, remember: You have helped create this, most folks love this, and most will continue to eat this up despite the obvious consequences.

Mobile devices in America are generating something like 600 billion geo-spatially tagged transactions per day.  Every call, text message, email and data transfer handled by your mobile device creates a transaction with your space-time coordinate (to roughly 60 meters accuracy if there are three cell towers in range), whether you have GPS or not.  Got a Blackberry?  Every few minutes, it sends a heartbeat, creating a transaction whether you are using the phone or not.  If the device is GPS-enabled and you’re using a location-based service your location is accurate to somewhere between 10 and 30 meters.  Using Wi-Fi?  It is accurate below10 meters.

Fancy.

It should be no surprise that all this data lives in the coffers of the cell providers.  Lots of people know that.  What is new, at least to me, is that this data is being provided to third parties that are leveraging specially designed analytics to make sense of our space-time-travel data. 

With the data out and specialized analytics emerging, this infant industry is already doing some pretty amazing work. Your space-time-travel data makes where you live and where you work self-evident, and it reveals your most frequent, periodic, infrequent and rare destinations. 

The data reveals the number of co-workers that join you Thursdays after work for a beer, and roughly where you all go. It knows where these same co-workers call home, and just exactly what kind of neighborhood they come from (e.g., average income, average home price) … information certainly useful to attentive direct marketing folks.

Large space-time data sets combined with advanced analytics enable a degree of understanding, discovery, and prediction that may be hard for many people to fully appreciate. Better prediction means a more efficient enterprise and nifty consumer services. 

Cellular companies are now receiving essential insight about their customers (e.g., to better understand and predict customer churn).  Major retailers can now better understand changes in consumer behavior (e.g., how far their customers are traveling on average this month compared to previous months).  Consumers are benefiting by getting real-time traffic information so they can avoid congested roads.  (I have a colleague that thinks he is saving two to four hours a week in commute time due to this service!)

Tip o’ the iceberg.

I can barely get my mind around the ramifications. My concept about what comes next shifts almost daily now.  A government not so keen on free speech could use such data to see a crowd converging towards a protest site and respond before the swarm takes form –  detected and preempted, this protest never happens.  Or worse, it could be used to understand and then undermine any political opponent.

A stalker might be questioned just days after he starts and before his victim is personally aware of it – detection previously beyond human capacity.  Maybe it’s not a crime in this case, and it turns out to be just a private investigator with poor tradecraft hired by a suspicious husband.

Such a surveillance intensive future is inevitable, irreversible and as I have said before here … irresistible.

Why?  Companies must be competitive to survive and consumers have quite the appetite for almost anything that optimizes their life, especially if it’s cheap or free.  For example:

Tuesday afternoon your [free] Gmail account advises you that your buddy Ken is going to be 15 minutes late to the pool hall this coming Thursday, unless he leaves work 15 minutes early … which he has only done twice in seven years.  Brilliant!

Your Starbucks drink of choice (a grande vanilla soy latte in my case) is handed to you the instant you pull up, and you did not call ahead nor did they ask.  Priceless!

When powerful analytics commingle space-time-travel data with tertiary data, the world we live in will fundamentally change.  Organizations and citizens alike will operate with substantially more efficiency.  There will be less carbon emissions, increased longevity, and fewer deaths.

I think people should know about this imminent new age we are marching into.

[Theatrical pause.  Breathe.]

Now I’m going to step back and address some questions you may have, using the good news/bad news format.

Good news: The space-time-travel collected by the cellular network carriers is de-identified when provided to these third parties for privacy reasons in that it does not include your name, address, phone number, etc.; rather, unique identifiers are assigned to transactions from the same device so that trends can be measured.

Bad news: If you were to provide your home, work and one other address (e.g., gym, school) in most cases, with just these data points, you are re-identified.  With just a few days of space-time-travel activity, your top three or four more frequently visited destinations become self-evident, and without a whole hell of a lot of effort you could be re-identified through a tertiary data set like a credit header.

Good news: There is so much data being produced, a lot of transactions are tossed aside, are sampled and summarized to make the computational effort feasible.  Historical data also falls off the back of the wagon (ages off the system) rather quickly.

Bad news: The competitive nature of this emerging business model will likely require these organizations to make more sense of more data faster.  Cloud computing and new classes of algorithms will make it possible to keep more transaction detail, keep it longer, and commingle it with other large and very interesting secondary data sets (e.g., phone books and property records).

Good news: So far there are only a handful of companies already entrusted with this data.

Bad news.  It may not be good news that only a few companies do this.  If only one company can monitor the consumer foot traffic of all Nordstrom stores in near real time,  this would be an unfair advantage in terms of predetermining its financial condition before anyone else.  As I learned from countless conversations with my friends at the ACLU, very powerful tools in the hands of a few is not often a good idea without one hell of a lot of oversight and accountability. And even then, this is no panacea.

Good news: Some of the organizations holding space-time-travel data are fully aware of the privacy consequences and are offering consumers the ability to opt-out – meaning, if they get a transaction about you it will be permanently removed from the system and all future correlation.

Bad news: If by chance a snapshot of sufficient detail had been sold off to another party before the opt-out request, then the toothpaste is out of the tube.  Data tends to replicate, more about this here.

Good news: Not any old mom and pop operation can get into this business.

Bad news: That won’t be true for long.  Suppose an aspiring entrepreneur makes a compelling proposition to a number of parties holding space-time-travel data.  Anticipating free analytics and a cut of the future action, the parties work a deal. For computing power this entrepreneur simply hops onto Amazon’s EC2 cloud and partners with a data aggregator to get some tertiary data and what do they have?  An ultra-sexy prediction engine.

Good news: People tend to appreciate location-based services, which is why they are opting in.

Bad news: Sensitive information about people is no longer under their own control.  As well, a number of well held secrets (e.g., your hideout) evaporate overnight.

Good news: If you want to escape the consequences of having your space-time-travel being graphed by others, here are some options that come to mind:

(a) Stop using mobile devices;

(b) Use multiple devices e.g., use one device only at work, and only a land line at home – all mobile devices being off at all other times (never moving around with a device on) – being sure these mobile devices are registered to someone other than you – and if you need to use some kind of device while on the move or at other locations. see (c) below;

(c) Unregistered, cash-purchased, disposable devices – used once then discarded (or recycled!) – although in some cases you can use the device a few times, but you better let some fancy software (which I may have to invent) advise you what is safe usage and what is not.

(d) If you can figure out locations on earth where only one cell tower exists (and you are not moving between towers and never using GPS or Wi-Fi) you will probably live safely under the radar – unless you are a way bad mofo and others know it, in which case, you are ‘going down’ anyway because there are more tricks (expensive) which will be levied against you.

Bad news: Few are willing to be this inconvenienced.  And if only a handful of innocent, clean living folks go to this same effort that the bad guys MUST employ … well crap, that in itself may be considered by some to be signal.

Net Net: My guess is most consumers don’t fully realize how their space-time-travel data is accumulating and congealing.  I hope consumers come to appreciate how all of these nice conveniences of life are delivered. And I hope they will continue to enjoy these while they make better informed decisions, especially with respect to their privacy.

However, without a feedback loop consumers may never fully appreciate what can be gleaned from their space-time-travel trail. Therefore, one way to enlighten the consumer would involve holders of space-time-travel data to permit an owner of a mobile device the ability to also see what they can see:

(a) The top 10 places you spend the most time (e.g., 1. a home address, 2. a work address, 3. a secondary work facility address, 4. your kids school address, 5. your gym address, and so on);

(b) The top three most predictable places you will be at a specific time when on the move (e.g., Vegas on the 215 freeway passing the Rainbow exit on Thursdays 6:07 - 6:21pm -- 57% of the time);

(c) The first name and first letter of the last name of the top 20 people that you regularly meet-up with (turns out to be wife, kids, best friends, and co-workers – and hopefully in that order!)

(d) The best three predictions of where you will be for more than one hour (in one place) over the next month, not counting home or work.

I think Google’s Android and Latitude products might be able to move on something like this first.  It would then be cool if other holders of space-time-travel data followed.

On the subject of privacy and civil liberties consequences, privacy by design is essential.  And for those of you with ideas in the area of policy or technology, I would be most appreciative if you would share these thoughts with me … sooner rather than later.

I will continue sharing perspectives about these ideas and the apparent consequences with my many friends in the privacy community, the defense/intelligence community, and media.  (Surprisingly, their feedback so far has been quite similar.)  I am also speaking with the organizations amassing and analyzing this space-time-travel data to learn more about what is possible.  From the perspective of the analytic engines I create, this space-time-travel data looks like “super food.”

USEFUL REFERENCE:

Internet Society: Wireless Location Privacy: Las and Policy in the US, EU, and Japan

 

RELATED POSTS:

Six Ticks till Midnight: One Plausible Journey from Here to a Total Surveillance Society

Prediction: Channel Consolidation

More Data is Better, Proceed with Caution

How to Use a "Glue Gun" to Catch a Liar