Posts Tagged Big Data
Let’s start from the very beginning. The Internet that we know and love today is an open channel that uses free standards for communication. It is an equal-opportunity channel when it comes to traffic/content creation and delivery. The bottleneck is primarily the bandwidth that each individual user has available.
Now let’s say there are two companies providing media sharing capabilities using the SaaS model. Today traffic to and from Company A (mega establishment) and Company B (fledgling start-up) are treated equally by the broadband service provider. Today it is illegal for the broadband provider to treat the traffic from the two companies differently (assuming similar and legal content from both companies).
Now imagine if the broadband company could tell these companies- hey if you pay me an additional fee- your traffic will get higher priority on my network! It’s easy to see how Company A could easily edge out any competition from a startup like Company B based purely on its financial advantage. Bigger and better funded companies can bury competition by simply buying out the bandwidth from the Internet providers.
This essentially can kill technological innovation.
The Openness of the Internet is not a privilege we enjoy- it is and must be a fundamental right. We have a chance now to go and tell the FCC that we want the Internet to remain a free, fair and equal-opportunity channel. The FCC is taking comments from the public on this proceeding (also referred to as preserving Net Neutrality).
The site will remain open for 120 days (starting May 15 2014).. so the clock is ticking!
File your comment here. Proceeding number is 14-28
Net Neutrality is important- or else all the content, access and service we get over the Internet will be based on the highest bidder for bandwidth usage.
Take action. Do it now! #NetNeutrality
The best talks I’ve ever heard always had one thing in common- they gave me goose-bumps and made me teary-eyed! The topics have ranged from technology (yes nerd-alert!) to politics- but most recently- it was education. Yesterday I attended the SxSWEdu Education Expo and sat in on a series of presentations. When Sarah Mabry walked on stage, the first thing I noticed was the bold-white frame of the eye-glasses she was sporting and the second thing was her magician-robe styled outfit. When the third sentence in her speech was “Sarah Mabry is not a wizard”, I knew this was going to be interesting! I wasn’t prepared for inspirational. Ms.Mabry was kind enough to share a written copy of her speech with me- but there were lines she uttered that aren’t contained in the text I now possess. Here are some of my favorites from her presentation:
“A nation is ONLY as strong as its educational foundation”
These are powerful words. She did reiterate this line three times. The message they contain is beyond powerful- it carries an urgency that cannot and should not be ignored. From 2008-2013, the United States fell from being #1 amongst 48 countries in its Global Competitiveness Index ranking to #5 (per the World Economic Forum). Let’s not forget that economic strength is valuable political capital.
“We are all stakeholders- so we must all be investors”
Yes this gave me those goosebumps. There is no doubt that our children are our future. Providing them with the resources to do better than we could is a fundamental responsibility that we all shoulder- as adults, parents, teachers and citizens. Unfortunately, the availability of these resources and opportunities appears to be in sync with the socio-economic divide. We know that the profile of the student population is projected to be over 50% non-Anglo American in the next 20 years and we also know that a majority of the disadvantaged youth population belongs to that same demographic bracket. Talk about predictive analytics- this is a pattern staring us in the face- and one we cannot afford to ignore.
“From those who have been given much, much is expected”
Slightly reminiscent of the Spiderman dialogue “With great power comes great responsibility”!! I think Ms.Mabry’s version is clearly a call to action. The challenges that our educational system faces today are not (and never have been) limited to the socially, economically, linguistically deprived sections of society. It’s a national emergency that requires everyone to listen, participate and contribute.
I got roped into another “the time has come” debate about the Chief Data Officer. I maintain that IMHO the CDataO is pure hype. Here’s how the roles of the existing C-suite play out:
CEO: What we need to do
CMO: How can we use it or apply it to better our product/image/business
CIO/CTO: What tech strategy do we need to adopt to get there and subsequently maintain/scale
COO: How can we implement these changes while maintaining normal business operations?
There’s really no room or need for a data officer- because it is close to impossible for one individual to determine all the combinations in which data can be useful to the people in the company. Will creating a new CDO position magically fix data governance, reliability and accessibility issues let alone make it more meaningful all of a sudden? No way! It’s all about understanding that these issues exist and need to be addressed & fixed.
The reality though is that I get angry and sad to see hype like this gaining momentum- it almost makes the real promise of big data seem illegitimate. I was pleasantly encouraged when Jim Stikeleather from (CIO- Dell/Perot) concurred with this remark:
“Sharda is spot on. There should be a lead data scientist under the CInfoO, and a lead enterprise architect, and a lead security analyst, etc. The issue IMHO is that CInfraOs are neither strategic or knowledgeable enough to lead those other roles. The CMO should have the analysts (mathematical and marketing) who are supported by the data scientist. Business units / COO / etc should have their own domain experts supported by the lead enterprise architect (including enabling the orchestration and choreography among enterprise and SaaS applications). A good CInfoO understanda they are an enabler and facilitator, not an owner (or a prohibitor – which is what CInfraO tend to be).”
Prepping for the holidays? Some others are dreaming about how big data can help them power through the insanity of holiday shopping!
And check out a very entertaining blog of the #bigdatachat Tweet Chat that I participated in- it was Santa centric and a lot of fun!
Ho ho ho and a lovely holiday season to everyone!
The author prophesies that the importance and definition of a data scientist will change in the next few years- as big data technology is adopted in a more mainstream manner. One of the comments was that data scientist is a misnomer because data science is not really a science. Analysis of data does not make it a science. I cannot agree more. It isn’t data science- it is decision science. Here’s my reply to the comment
“Science has always been about setting a hypothesis and proving it right or wrong. We’re just getting started with BigData. The proper and most beneficial use of big data will be when everyone becomes a decision scientist. The data, the analytics, dashboarding and interfaces will simply be tools that will help a business/ enterprise/ organizational decision maker verify the viability of their hypothesis. That’s the real promise of bigdata. If it isn’t, it should be about being the toolkit for Decision Science.”
“Intuition is a very powerful thing, more powerful than intellect, in my opinion.” – Steve Jobs
When setting new direction or exploring new ideas, business decision makers often rely on recognizing a pattern or developing a notion that they cannot quite articulate. What they do have to rely upon is years of experience, a deep knowledge and expertise in their domain. Massimo Pigluicci, a philosophy professor at City University of New York says “Intuition is the result of your subconscious brain picking up clues and hints and calculating the situation for you, and that’s based solely on experience”. This is the definition of the best kind of intuition in the business world.
Mr. Jobs had a highly refined gut instinct. Most others benefit from the ability to capture, store and measure a flood of data (big data). This deluge of data has empowered organizations to create analytics and metrics to track behavior and predict the future with greater accuracy than ever before. We can now exploit the advantages that big data to better understand competition, customer base, factors that affect productivity, sales & revenue and to determine the courses of innovation. MIT Professor and director of the MIT Center of Digital Business Erik Brynjolfsson’s studies have shown that data driven decision making does indeed improve the decision making performance by 4-6% in productivity and profitability (ref. HBR Oct 2012).
The more data we can gather, the more sophisticated analytical methods we can adopt, the more decisions we can make with little to no human intervention. Does this mean then that intuitive decisions are not the path to success when working with mountains of structured and unstructured data?
We believe that data driven business analysis doesn’t have to be at odds with intuitive decisions making. Intuition and analysis work best when teamed together to enhance the value offered by the other. This is accomplished by utilizing tools such as Genie the intuitive analyst tool from Quicklogix. Business leaders and analysts can use its simple Google-like querying interface to build an evidence story to corroborate their business instinct.
Big data is making us smarter like never before. Complement this with the intangible advantages of knowledge, experience and yes- intuition to derive the best benefits.
“A good decision is based on knowledge and not on numbers” – Plato
What are the guiding objectives for making a business decision? We want to satisfy our customers. We want to be able to demonstrate that the service or product we provided them has quantifiably improved their loyalty or customer satisfaction. We want to increase revenue and profitability. We want to reduce costs and prevent disasters.
Decision science is defined as the design of efficient procedures that either aid a decision maker’s effort or evaluate courses of action according to chosen criteria or policies so that a decision logically follows from the computations (ref: Principia Cybernetica Web). The theories that support these efforts take into consideration the uncertainty of the situation, the information available about consequences, the risks involved, costs & benefits of each action and the time, resources and preferences at the decision maker’s disposal.
Good decision making must take into account logical, environmental and emotional perspectives. Today we have the ability to capture, store and measure a flood of data (big data). This deluge of data has empowered organizations to create analytics and metrics to track behavior and predict the future with greater accuracy than ever before. MIT Professor Erik Brynjolfsson’s studies have shown that data driven decision making does indeed improve the decision making performance by 4-6% in productivity and profitability (ref. HBR Oct 2012).
Companies need to rise to the challenge posed by the availability of this improved predictability. There is no longer the excuse of not having enough data to prove that a proposed solution was bound to succeed or fail. The value of the executive is still his/her ability to create a vision, spot new horizons to explore and chart a map to get there without being afraid to embrace the opportunities offered by the new data paradigm. The value of the domain expert is in his/her ability to ask the right question. The value of the IT team is in their ability to recognize that data proliferation and accessible analytics are essentially the wheels that will propel their organization forward and that they themselves are responsible for keeping the wheels well-oiled.
2500 years after Plato- numbers matter but knowledge still holds its own in the process of decision making.