Tag Archives: bigdata

Bill Gates is naive, data is not objective

Posted on by • Tagged , , • Leave a comment

Let’s see some more example of data collection and model design not being objective:

  1. We see that cars are safer for men than women because the crash-test dummies are men.
  2. We see that cars are safer for thin people because the crash-test dummies are thin.
  3. We see drugs are safer and more effective for white people because blacks are underrepresented in clinical trials (which is a whole other story about power and data collection in itself).
  4. We see that Polaroid film used to only pick up white skin because it was optimized for white people.
  5. We see that poor people are uninformed by definition of how we take opinion polls (read the fine print).
via mathbabe.org

 

Big Data, Complex Systems and Quantum Mechanics

Posted on by • Tagged , , , , • Leave a comment

But, the worlds of the very small, as well as the very large, are not the only ones that exhibit counter-intuitive, seemingly magical behaviors.  So is the world of highly complex systems, especially those systems whose components and interrelationships are themselves quite complex, as is the case with systems biology and evolution. 

Such is also the case with organizational and sociotechnical systems whose main components are people.  Even though these chaotic systems are in principle deterministic, their dynamic, non-linear nature renders them increasingly unpredictable and accounts for their emergent behavior.  New terms, like long tails, Freakonomics and black swan theory, – every bit as fanciful as quarks, charm and strangeness, – have begun to enter our lexicon. 

via blog.irvingwb.com

It’s always been beyond our ken, that’s what bounded rationality is about. But hope springs eternal that something will solve the understanding problem for us.

Google???s looming hegemony

Posted on by • Tagged , , , , , • Leave a comment

Large utility companies worry about Google. Why? Unlike those who mock Google for being a “one-trick pony”, with 99% of its revenue coming from Adwords, they connect the dots. Right before our eyes, the search giant is weaving a web of services and applications aimed at collecting more and more data about everyone and every activity. This accumulation of exabytes (and the ability to process such almost unconceivable volumes) is bound to impact sectors ranging from power generation, transportation, and telecommunications.

Consider the following trends. At every level, Western countries are crumbling under their debt load. Nations, states, counties, municipalities become unable to support the investment necessary to modernize — sometimes even to maintain — critical infrastructures. Globally, tax-raising capabilities are diminishing.

via mondaynote.com

The reason for the concern, and the solution to the decay of State actors is, Google’s information collection and analysis capability which may become essential to assessing the PPP risk (despite always being underwritten by the tax payer).