I must confess that big data analysis wasn’t exactly on my radar when I started engineering at IIT Roorkee. In fact, I started out as a paper and pulp technology major. I soon lost interest in my regular classes and started to work on other projects instead. My first...
Industry
Recent Articles
Is Spark better than Hadoop Map Reduce?
For anyone who gets into the Big Data world, the terms Big Data and Hadoop become synonyms. As they learn the ecosystem along with the tools and their workings, people become more aware about what big data actually means, and what role Hadoop has in the big data...
Big data analytics – Not just a matter of scale
Follow effective action with quiet reflection. From the quiet reflection will come even more effective action. Peter Drucker It is only when you reflect on your experiences do they translate into learning. And so, having been in the technology space for close to three...
The search for Long Johns in cold New York
It is minus 13 °C in New York City (with the wind chill), and I feel woefully unprepared for the cold. I immediately head to the Uniqlo store on 5th Avenue - to check out their Heattech vests, long johns and socks. They are out of long johns. A sales associate tells...
Big data meets mobile: simpler choices on smaller screens
As we begin our journey in 2015, here are a few thoughts on big data trends for mobile devices. In the age of more, we may soon see the twilight of search 15 years ago, the defining problem of the Internet age was how do I find the information I need? This gave birth...
8 important things to remember while mining data
1. Having a clearly-defined and measurable objective for a project in mind Data mining is often a difficult and time consuming task. Hence, not having a clear idea how to mine the data will severely affect the project’s focus. So having a clear idea of what, how and...
How Big Data helps us in the fight against crime: Beyond 'Minority Report'
In today's complex world, criminals are everywhere. There is a greater responsibility on law enforcement authorities than ever before to apprehend criminals even before they commit crimes. This idea of arresting people before they commit a crime was first featured in...
What does a Data Scientist do?
Data Scientists...! We hear about them everywhere around us and there is so much hype around data science, data scientists and the slew of accompanying technologies like Hadoop, Hive, Mahout, Hbase and with keywords like big data, machine learning, information...
Re-Imagining the Store of the Future – Part II
You are in a department store. After waiting in line, you get to the fitting room - and try out the outfit you've picked out. You love it, except…..it’s the wrong size. You now have to change back into your clothes - go out pick the right size, and get back to the...
Re-Imagining the Store of the Future
A couple of months ago, on a trip to the US, I decided to replace my aging MacBook with a MacBook Air. I placed the order online, since I also wanted to soup up the memory and hard disk. While I was at it, I decided to order a “COVER” for the machine. Four days later,...
Big Data Wants to be Your Shopping Buddy
Imagine a nice Friday afternoon walking around your local mall. You walk into your favorite browsing for a new outfit to wear to your friend's party later in the evening. Although you usually shop in-store, sometimes you shop online, as many people all over the world...
Relational Vs Non-Relational databases – Part 1
In this post, we have seen some of the Big data technologies that are used to store and analyse data. For the past few years NoSQL or Non-relational database tools have gained much popular in terms of storing huge amount of data and scaling them easily. There are...
Crayon's Business Interest Graph: How does it work?
Crayon's SimplerChoices Business Interest Graph (BIG) is a B2B application that identifies a company’s business interests, needs and most relevant market opportunities with external data and events. It combines deep insights on companies, markets and key events to...
How does a Choice Engine overcome the choice problems?
Consider a typical decision – you have just arrived in a new city after a 6 hour flight, it’s 6.30 PM and you want to get a decent vegetarian meal. What do you do?You fire up Google, search for good restaurants in that city, and filter them by type, price, location...
Sales bonus and big data
Leaving behind the old way of sales, companies now have become extremely sophisticated when it comes to helping sales and marketing teams achieve their numbers, and hence the green bucks that inadvertently flow after the quota is achieved, by being creative on Big...
Search vs Choice – A new type of engine for a new kind of age!
15 years ago, the defining problem of the Internet age was how I find the information I need? This gave birth to search engines and Google’s great vision of “organizing the world’s information”. A decade later, as Google continues to improve its search with...
How big data is transforming the world of hospitality
Over the past few years, Big Data has transformed the way the hospitality, travel and tourism sector (HTT) functions. It has lodged a new dimension of customer care in HTT by opening up mind-boggling possibilities -- spurring a 'paradigm shift' from simply viewing...
Design creates culture. Culture shapes values. Values determine the future.
"Design creates culture. Culture shapes values. Values determine the future." This quote from Robert L Peters best defines what Crayon is setting out to do. We are having a design discussion, as we shape our products, our culture and our future. This exercise is also...
Government – A Big Data Analytics playground
From citizenship applications, employment status, filing taxes to census recording – all provide massive amounts of data to the Government. A torrent of data in the form of medical files, satellite images, financial records and social media information. To a Big Data...
Big Data Made Visual
“Computers in the future may weigh no more than 1.5 tons” - Popular Mechanics, forecasting the relentless march of science, 1949. Predictions. Rock solid, one day. Busted as myth, the next. But the relentless march of our imagination – and creation - must go on....
The Metaphysics of Big Data: Problem of Induction
How do we, human beings, acquire knowledge? Think for a sec! And don't tell me that it is from text books, TV, internet or newspapers etc!!! Scholars say that there are mainly (debatably) five valid ways (Perception, Inference, Comparison, Verbal Testimony and...
Big Data and the Paradox of Choice
"Autonomy and freedom of choice are critical to our well-being, and choice is critical to freedom and autonomy. Nonetheless, though modern Americans have more choice than any group of people ever has before, and thus, presumably, more freedom and autonomy, we don't...
'Big Data vs Intuition' paradox explained
Every business leader will have a dad's story, I believe, -- a memory or nostalgia how he/she was inspired by the attitudes/thoughts/decisions (sometimes seemingly tough, painful and even stupid) of his/her dad in paving a unique way to success and beyond. Here is one...
Big Data – Hype, Reality, Bill Gates and the Buddha
Today's NYT carried a very good blog post on the six myths surrounding big data. Contrast this with another equally good article on the premise of Big Data - that "...hidden in all that information lies a hyper-resolution map of the world's behavior in space and...
Big Data can set your mind free…
In my last post, I talked about the paradox of big data, or how data can be misery... In today’s post, I will talk about how that paradox can be resolved... and how data can set you free. Consider these statements by some of the leading writers, academics and...
The Paradox of Big Data… or Why Data is Misery
We've heard it shouted from every rooftop recently– big data is the next big thing, we’re a data-driven company, data underlies everything we do.... So here’s a paradoxical, even heretical thought: Data is misery. Why? Consider this. We are all becoming adherents of...