Martin Lindstrom has spent time with 2,000 families in more than 77 countries to get clues to how they live — resulting in the acquisition of what he likes to call Small Data. In his new book, Small Data: The Tiny Clues That Uncover Huge Trends, he argues that the Small Data explains the “why” behind what Big Data reveals.
Big Data is all about finding correlations, but Small Data is all about finding the causation, the reason why. –Martin Lindstorm
In his book, he talks about how the corporate world has become completely blinded by Big Data. He also talks about how companies which are completely reliant on Big Data actually have started to have a problem. He cites the example of Walmart, irrespective of having the largest data-mining and big data operations in the word, they had to issue a second profit warning recently. He gave another example of Lowes Foods, which is my home town favorite in NC and now one of the fastest growing in the region. Lowes Foods is actually living with the consumer in the community to understand the Small Data. That means listening closely to the community needs by embedding themselves into the community. They’re talking with everyday consumers and integrating themselves deeper. As a result, they now have completely turned around and is now one of the fastest growing in the region because they’re listening to the consumer and the Small Data.
Need for Small Data
In a traditional sense, I agree it is very hard to describe emotions using just data and numbers. I also agree that it is very important to listen to the consumer via. the Small Data. It would be ideal to elicit Small Data by living in the communities or by deeply embedding into the communities, but we can’t ignore the practical challenges and resource limitations to replicate this nationally.
Small Data lives on the Cloud
Let’s also look at how we are communicating in today’s world. Technology’s rampant popularization over the past decade in terms of social media has meant that texting, WhatsApp, Next Door, Facebook, Instagram, and Twitter have inevitably taken over as the most efficient ways of communicating with each other. One can make an argument that in today’s world Small Data actually lives in the form of tweets, images, videos, product reviews, surveys, community blogs, etc. somewhere on the cloud.
Understand Small Data with Cognitive Computing
Understanding Small Data in today’s world means understanding the verbal and non verbal communications that are taking place on the cloud. I believe cognitive analytics can bridge the gap quite nicely.
Cognitive Analytics is creating a new partnership between people and computers that enhances, scales and accelerates human expertise.
Like humans, cognitive analytics understands individual behaviors and can engage with humans. It also learns from us and improves over a period of time. We use our cognitive capabilities to understand emotions and other human behaviors. Likewise, cognitive analytics can sort through millions of tweets, images, speeches, product reviews and other customer generated data points to understand human behavior, emotions, personality traits,etc. Image below outlines cognitive capabilities available on the IBM Watson Developer Cloud. Google and Microsoft are offering similar capabilities as well.
Develop Big 5 Personality Traits from user generated content
IBM’s Watson Developer Cloud includes a Personality Insights API/tool, which leverages linguistic analytics to extract a spectrum of cognitive and social characteristics in the form of Big 5 personality traits from the text data someone generated through blogs, tweets, forum posts, and more. This kind of personality analysis helps to understand, connect to, and communicate with customers on a more personalized level.
Evolution of Cognitive technologies
Cognitive technologies have improved dramatically in recent years. From machine learning, natural language processing (NLP), speech recognition and tone analysis, it now can extract visual insights from a collection of customer images from Instagram or Facebook collections. Corporations can now leverage these visual cues (Small Data) to understand customer behavior and needs.
Big Data meets Small Data
When companies start to weave-in Small Data from cognitive analytics into insights from Big Data and predictive analytics, they can deliver an enriching and personalized experience for their customers.
In summary….
There is a huge desire for tactile interaction with people, but unfortunately the physical community as we know is dying. As a result, the human interactions are moving into the cloud. We also need to fly/dive deep into the cloud to understand deep consumer psychology (Small Data) by analyzing human interactions on the cloud. This type of analysis is going to be the biggest asset for every company going forward.
Srini Kasthoori 