Crayons. Chronicles. Captains
Meet our talented tangrams from across the Box, and find out where we’ve been making headlines
Will Big Data End Your Career Prematurely?
In 2001, I wrote a book called Making It Personal that predicted a growing tension between privacy and personalization. To illustrate potential developments, I included a number of fictionalized but plausible scenarios, including this one. Here's my question: could...
Big Data Explains Why Umpires Make Bad Calls
Holliday’s body language speaks clearly, and his reaction is understandable. The pitch was wide, even wider than the first two pitches, both of which the umpire miscalled as strikes. Here’s the data: The PITCHf/x technology that makes this graphic possible, whatever...
Hadoop and big data: Where Apache Slider slots in and why it matters
Code submitted this week for inclusion in the Hadoop stack will help speed the spread of the distributed big-data platform, according to Hortonworks co-founder Arun Murthy. The submission of the Slider framework to the Apache Software Foundation Incubator will result...
Get set, grow
Excerpts from the report… Take the case of I Vijaya Kumar, a long timer with Wipro, where, as CTO, he was popularly known as IVK. Eventually IVK decided to move on and pursue his own calling. “I have always been interested in new technologies and innovation...
Surveillance fears are causing people to be more cautious online — and that’s good
Almost half of Americans are increasingly cautious about their use of the internet following the NSA surveillance leaks, according to a Harris poll commissioned by security outfit ESET. This is pretty much what I and many others predicted when the agency’s data...
By turning security into a data problem, we can turn the tables on the bad guys
Big data and predictive technologies – sometimes called artificial intelligence (AI) – are changing the world. Everyone from Google to your supermarket and hospital are leveraging the power of data to transform the way they operate fundamentally. While the traditional...
What Will Happen to ‘Big Data’ In Education?
Yesterday, a $100 million startup lost its last customer. According to a Politico article, the state of New York, inBloom‘s last remaining client, will delete all student data on the repository due to privacy concerns. InBloom’s company spokesperson told Politico the...
Big Data: Big Hype, Utopian Promise, or Impending Hackocalypse?
Whether it’s big government or big corporations you fear may be encroaching on our right to privacy, you should have no trouble these days finding news to fuel your suspicions. If, on the other hand, you happen to run a business, you probably find clear-eyed...
Big data showdown: Cassandra vs. HBase
In this brave new world of big data, a database technology called "Bigtable" would seem to be worth considering -- particularly if that technology is the creation of engineers at Google, a company that should know a thing or two about managing large quantities of...
The frightening truth about the security of our healthcare data
Is your healthcare data safe? That’s not something most people think about on a regular basis. We take for granted that our medical records, family histories, insurance coverage and the rest of the data associated with our health is protected carefully by those who...
Would you borrow money from your boss? The privacy impact of ‘Big Data’ loans
Would you take out a loan from the company you work for? Plenty of new lenders are betting that you might. But even if you wouldn’t, a new crop of lending products is bound to impact your privacy. The National Consumer Law Center released a report recently called “Big...
Big Data Renders College Diplomas Worthless; Billionaires Nonplussed
It used to be accepted without question that a college degree was necessary to get a good job, and over the course of a lifetime, you would make more money with a college degree than without one. But not so fast. Despite the propaganda put out by colleges who hope to...
Can Big Data Make You a Better Publisher?
What came first, the data or the publisher? How could a person publish without having at least some idea who was going to read or see it? Whatever the case, publishers published and the data started to roll in. In the last few years, the data drip has become a...
Police think ‘Big’ in fight against petty crime
Local police are turning to information technology to help them combat a rising tide of petty theft in the city. At the heart of the latest crime-fighting campaign is the "Big Data" system set up by the Shanghai Public Security Bureau. Providing officers with...
Big Data Box and the Black Crayon
SINGAPORE: Ask him what made the founders christen their venture as Crayon, and I Vijay Kumar, CTO, Crayon Data and ex-CTO ofWipro intrigues with a simple and short answer – Because a crayon box is all about imagination and yet without any manuals! Imagination is...
Master the art of big data job scheduling
Almost every department and business function in the enterprise has a big data application. Consequently, the need to centralize and harness big data assets has led more organizations to move big data responsibilities and assets to the corporate data center. This is a...
10 Ways To Fight Digital Theft & Fraud
The best way to thwart digital theft and fraud is to use a holistic, connected approach that takes advantage of the latest technologies and applies advanced analytics to vast data sets. It's an approach IBM touted at a mid-March event in New York where it brought...
Is big data dating the key to long-lasting romance?
If you want to know if a prospective date is relationship material, just ask them three questions, says Christian Rudder, one of the founders of US internet dating site OKCupid. Do you like horror movies?" "Have you ever travelled around another country alone?"...
How predictive scores are changing big data
When it comes to big data, marketing departments are frequently at a loss of how to make the most of it. With all the information about data science, predictive analytics, and using big data to make better marketing decisions, it can be overwhelming. Big data is a new...
Getting Marketers Up to Speed with Big Data
That alarm marketers are hearing is a wake up call from big data. It's telling marketers that they need to be more metrics driven, more technically savvy and more process oriented. At the top of the food chain, CMOs are taking on some of the responsibilities that...
Listening to the Public’s Concerns About Big Data
In the next two years, the future of truly personalized learning and student achievement outcomes will largely be determined on how effectively data is used. Success depends on addressing fast growing issues of how data is collected and maintaining student privacy....
Big data retailing offers tailor-made shopping for all
Imagine everything being known about you the minute you walk in to a department store - your name, measurements, purchase history in-store and online, even your views on life, the universe and everything. Would that make you feel like a celebrity or a victim of...
How Your Sensor Data and The Internet of Things Can Save You A Lot Of Money
The Internet of Things, Industrial Internet, Internet of Everything, no matter how you name it, the upcoming connected world will change everything and create massive amounts of sensor data. According to Cisco, it is going to be a $ 19 trillion market within the...
The Mystery of the Exploding Tongue: How reliable is Google Flu Trends?
“When it went off the rails, it really went off the rails,” said David Lazer, a computer scientist at Northeastern University. Lazer is talking about Google Flu Trends, the internal Google analytical tool that tracks data such as keyword searches—things like “cough,”...
Retailers weigh in on role, impact of big data
A study from New York-based information firm 1010data Inc. shows that while retailers are facing challenges in delivering the reporting functionality that business users need to enable data-driven decision-making, they are optimistic about big data’s ability to...
Smart Cars and Big Data: Automotive Analytical Concerns
At the launch of the German auto industry event CeBit, Volkswagen’s chairman Martin Winterkorn stated, “The car must not become a data monster. I clearly say yes to Big Data, yes to greater security and convenience, but no to paternalism and Big Brother.” The English...
Making Sense of Big Data in Search Marketing
To make marketing data work, you need to be able to measure and understand performance against a standard set of metrics. The challenge lies in marrying data from web analytics with search and social signals, and then being able to tie that to business value (for...
Big data used to catch fraudulent tax returns
Identity thieves are stealing billions of dollars a year through fraudulent tax refunds—and the IRS isn't the only target. The 43 states that collect an income tax are also being flooded with these bogus returns. How serious is the problem? A report from the Treasury...
Major companies not making full use of big data to spot fraud
Most companies are spurning the chance to improve their anti-fraud and anti-bribery efforts by not taking full advantage of big data analysis, according to research from business consulting firm EY. EY found that 63 percent of senior executives surveyed at leading...
Graph Processing Using Big Data Technologies
Processing extremely large graphs has been and remains a challenge, but recent advances in Big Data technologies have made this task more practical. Tapad, a startup based in NYC focused on cross-device content delivery, has made graph processing the heart of their...
Big data – an opportunity or a threat in the fight against fraud?
It’s shocking to think that despite the UK losing an estimated £52billion to fraud annually, some businesses still bury their heads in the sand when it comes to confronting the problem, whether it comes from an internal or external threat. In times when people are...
Crayon's Business Interest Graph: How does it work?
Crayon's SimplerChoices Business Interest Graph (BIG) is a B2B application that identifies a company’s business interests, needs and most relevant market opportunities with external data and events. It combines deep insights on companies, markets and key events to...
Use Data to Tell the Future: Understanding Machine Learning
When Amazon recommends a book you would like, Google predicts that you should leave now to get to your meeting on time, and Pandora magically creates your ideal playlist, these are examples of machine learning over a Big Data stream. With Big Data projected to drive...
Crime fighting with big data weapons
For Lombroso, a sloping forehead, large ears, very long arms and any one of a host of other physical characteristics pointed toward their owner's in-born propensity for a life lived on the wrong side of the law. If only it were that easy to spot the bad guys. Now, it...
In marketing, embracing a world where data rules
Big data this, big data that. The popular Silicon Valley buzzword has become so ubiquitous -- the term was even added to the Oxford English Dictionary last year -- that it seems to mean everything and nothing at the same time. Big data will transform industries!...
How Big Data Analytics is Aiding Search for Flight 370
As the hours and days go by following the sudden and mysterious disappearance of Malaysia Airlines Flight 370 somewhere in Southeast Asia, more people and organizations are joining the search party. And they are using every tool at their disposal -- not the least of...
How To Track People And Collect Their Data Without Spooking Them
It’s one of the toughest problems facing any information-driven business today: How do you collect “big data” about anonymous groups of users or customers, while at the same time guaranteeing that you’re not also gathering “little data” about their private lives? Put...
Big data is watching you
Three years ago the satirical news service The Onion announced that the CIA was shutting down many of its surveillance operations due to the runaway success of the state-controlled social media service, Facebook. After years of running costly spying operations, the...
Data analytics and sport: a match made in heaven?
Big data has seeped into all aspects of life and the sporting world is no exception. Data analytics has been applied to sport as varied as cricket, football, American football, basketball, rugby, Gaelic games, motor racing, tennis and sailing. Analytics isn’t solely...
The Death of Data Science has been Greatly Exaggerated
I’ve been finding it fascinating to watch on as data scientists discuss the death of data science. It all started with a rather sensationalized post over at Slashdot: “Data Science is Dead” by Miko Matsumura, Vice President at Hazelcast. It wasn’t even a question for...
The Death of Data Science has been Greatly Exaggerated
I’ve been finding it fascinating to watch on as data scientists discuss the death of data science. It all started with a rather sensationalized post over at Slashdot: “Data Science is Dead” by Miko Matsumura, Vice President at Hazelcast. It wasn’t even a question for...
Crayon Data In The Final List For Big Data Award
HANOVER: Suresh Shankar is surrounded by big data, literally. The co-founder of Singapore-based Crayon Data, which has a development centre in Chennai, is greeting startup enthusiasts standing before a giant panorama graphic that manifests the largesse of big data...
Leveraging real-time scoring through BigData to detect insurance fraud
Fraud risk exposure from claims is a major concern for the insurance industry, and it’s growing. But timely detection of attempted fraud is challenging. Effective fraud detection must sift through huge volumes of data coming from many different sources, both inside...
What’s better for your big data application, SQL or NoSQL?
One of the critical decisions facing companies embarking on big data projects is which database to use, and often that decision swings between SQL and NoSQL. SQL has the impressive track record, the large installed base, but NoSQL is making impressive gains and has...
Improving Customer Service With Big Data
As more and more businesses compete with one another for customers, companies are realizing that their level of customer service is almost as important as the quality and affordability of their products and services. One way companies are assessing and improving their...
8 Ways Big Data and Analytics Will Change Sports
Analytics and big data have potential in many industries, but they are on the cusp of scoring major points in sports. From coaches and players to front offices and businesses, analytics can make a difference in scoring touchdowns, signing contracts or preventing an...
5 things that will remake big data in the next 5 years
Big data has evolved a lot of the past few years; from a happy buzzword to a hated buzzword, and from a focus on volume to a focus on variety and velocity. The term “big data” and the technologies that encompass it have been pored over, picked over and bastardized...
Big Data And Privacy: An Uneasy Face-Off For Government To Face
Thrust into controversy by Edward Snowden’s first revelations last year, President Obama belatedly welcomed a “conversation” about privacy. As cynical as you may feel about US spying, that conversation with the federal government has now begun. In particular, the...
Machine Learning Meets Economics
Economists often build "structural models," where they specify a specific model of individual behavior and then use data to estimate the parameters of the model. Although such models require strong assumptions, they have the advantage that they can make principled...
How big data will haunt you forever: your high school transcript
Arizona State University, like many colleges across the United States, has a problem with students who enter their freshman year ill prepared in math. Though the school offers remedial classes, one-third of students earn less than a C, a key predictor that they will...
Hadoop Market is Expected to Reach $50.2 Billion, Globally, by 2020
Hadoop is a distributed processing technology used for Big Data analysis. Hadoop market is expanding at a significant rate, as Hadoop technology provides cost effective and quick solutions compared to traditional data analysis tools such as RDBMS. The Hadoop Market...
Each Social Network Is Using A Very Different Data Lens To Understand And Target Users
The average global Internet user spends two and a half hours daily on social media, and information on their activity - gathered under the catch-all 'big data' - reveals a great deal about what makes them tick. Now, social networks are making significant investments...
Using Big Data to Audit Access to Patient Data and Beyond
Middlesex Hospital, located in Middletown, Conn., is in the anticipation business. And thanks to analytics software, business is starting to get good. Auditing who is accessing patient data in an electronic health record (EHR) is a requirement for hospitals attesting...
A Guide to Checkpointing in Hadoop
Checkpointing is an essential part of maintaining and persisting filesystem metadata in HDFS. It’s crucial for efficient NameNode recovery and restart, and is an important indicator of overall cluster health. However, checkpointing can also be a source of confusion...
Big Data Goes To School
Public schools nationwide are taking a cue from business, harnessing big data to improve student outcomes, help school districts make better hiring decisions and help governments use their education dollars more effectively. The results may be more successful...
Top 12 ways to disaster-proof your critical business data
There is no silver bullet to disaster recovery, says Jack Bailey, engineering manager at cloud computing provider iland. "When outage horror stories take over headlines, executives tend to have kneejerk reactions and look to adopt whatever disaster recovery offering...
Open source data grows up: Choosing MySQL, NoSQL, or both
Open source data has a split personality. There’s the NoSQL zealot who likes to fire off tirades against the restrictive world of relational databases, then there’s the MySQL devotee who’s a staunch defender of everything structured – with all that data living neatly...
Big Data’s Fading Bloom
Nobody would deny that Big Data was one of the most talked about areas in tech last year. And while Big Data was once viewed as the golden child of tech, its bloom is fading in terms of the value that it is able to deliver all on its own. There was a time not that...
The three most common NoSQL mistakes you don’t want to be making
Like many emerging technologies, NoSQL has gone through a hype cycle that saw widespread implementation followed by decidedly mixed results. The cargo-cult mentality that had everyone jumping on board the non-relational database train has certainly led to some high...
How Netflix uses your data to work out what you want it to commission
Netflix, the internet streaming service, can ask producers of original programming to find new actors if the suggested ones don't have a history of impressing the online audience. But Netflix's director of corporate communications Joris Evers insists that was never...
Sports is big business, which means it’s fertile ground for big data
While there’s no real gauge for how strong an idea can be, a decision by Booz Allen Hamilton to build a practice around that idea might be a good indicator of longevity. That is good news for the sports world, or at least the companies that want to cash in on it by...
How To Think About Big Data And Privacy
There was a great passage in Alexis Madrigal’s recent interview with Gibu Thomas, who runs innovation at Walmart: “Our philosophy is pretty simple: When we use data, be transparent to the customers so that they can know what’s going on. There’s a clear opt-out...
How To Reconcile Big Data and Privacy
In many ways “big data” and “encryption” are antithetical. The former involves harvesting, storing and analyzing information to reveal patterns that researchers, law enforcement and industry can use to their benefit. The goal of the latter is to obscure that data from...
Is the marriage of Big Data and advertising heading for an ‘uncanny valley’?
One of the major ways in which big data is currently used by marketers is to drive the targeting of online advertising to ensure it is delivering relevant messages to the right audiences at the right time. The scale of investment in online advertising is consistently...
Big Data Moonshots and Predictive Interaction
“Automation is not the solution to data transformation and data cleaning.” “Predictive Interaction is a new model for high-level human-data interaction that radically improves the productivity and accessibility of the most time-consuming work in the analytics...
The Mobile Revolution and Opportunities for Big Data
Factual founder and CEO Gil Elbaz spoke at Startup Grind in Mountain View, CA this past February. Listen to his talk about the great possibilities that arise from today’s influx of new data from mobile.
Introducing the Wolfram Language – Knowledge-based Functional Programming
In this video, Stephen Wolfram introduces the Wolfram Language, a “knowledge based” symbolic programming language that enables powerful functional programming, querying of large databases, flexible interactivity, and easy deployment.
How Investors See the Big Data Market and Its Investment Opportunities
The Big Data market is booming as we showed earlier in the Big Data Investment Infographic and also in 2014 this will continue. Jake Flomenberg, a big data industry veteran turned Venture Capitalist, talks about how he views the larger big data market. Jake works with...
Big Data and Law Enforcement: Was ‘Minority Report’ Right?
Big Data has always been around in some form or another. At some point, businesses decided to use that information to learn what makes their clients tick with phrases like “sales funnel analysis,” “actionable insights,” and “positive business impact.” So it stands to...
Top 3 ways marketing can increase revenue from Big Data
How can marketing departments leverage big data to help their organizations increase revenue? Quite simply, by preparing sales to lead with buyer's concerns rather than waiting for the buyers to ask questions. Big data systems use new mathematical models to quickly...
Marketers Take Note: Big Data Is About to Get Personal
Big Data has been a hot topic for the past couple years, and now it's about to get personal. Companies recognize the massive amount of customer data at their disposal, but it's becoming more important than ever to more effectively collect and analyze this data to...
Why the internet of things is big data’s latest killer app — if you do it right
Every year, nearly 2 million people contract infections while staying at the hospital, and inadequate handwashing is a big cause. It turns out we might be able to help fix the handwashing problem with help from intelligent sensors, and a Huntsville, Ala.-based...
Why We’ve Learned So Little From Big Data
How do we make Pre-K teachers stakeholders in their students' college and career readiness 12 years into the future? The question, posed by Howard County School Supt. Renee Foose at this SxSWedu panel, perfectly describes the gap between practice and theory when it...
Big Data makes for big opportunities in car manufacturing
Cars are becoming more intelligent, that is certainly clear.The average family saloon now has more computing power than Apollo 11, the shuttle that first took man to the moon. One major aspect of this ‘intelligence’ is how the information a car gathers is transported...
Book Publishing’s Big Data Future
The publishing industry is not one of the overachievers in terms of its use of big data. And since my book on big data—Big Data @ Work—is out, I thought it might be fun to speculate on what big data will do to the business of publishing books. The goal of any...
Browse Less, Find More
Excerpts from the report… Most recommendation engines are based on collaborative filtering, which looks at the affinity between two products in the same category or between two categories. It has limitations because it only looks at past purchase history of a...
Can we protect ourselves from online privacy threats?
Are hackers, long considered a threat, now the best defence we’ve got? For years, since the emergence of Wikileaks as a media sensation, we’ve been aware of a group of digital activists who have warned us, often in apocalyptic language, about the scope and scale of...
The One Hidden Skill You Need to Unlock the Value of Your Data
Data scientists are a hot commodity in today’s data-abundant world. Business leaders are relying on data scientists to improve how they acquire data, determine its value, analyze it and build algorithms for the ultimate purpose of improving how they do business. While...
The One Hidden Skill You Need to Unlock the Value of Your Data
Data scientists are a hot commodity in today’s data-abundant world. Business leaders are relying on data scientists to improve how they acquire data, determine its value, analyze it and build algorithms for the ultimate purpose of improving how they do business. While...
The perils of data-driven cricket
Cricket is an art, not a science. It's a fact that needs restating after the disintegration of Andy Flower's reign as England coach. Slavery to data had gone too far. The triumphs of the more jocund Darren Lehmann, Flower's coaching antithesis, are a salutary reminder...
Why civil rights groups are warning against ‘big data’
The backlash against the government's use of bulk phone records for intelligence purposes has been led mostly by technologists used to speaking the language of privacy. But a new push by civil rights organizations to challenge "big data" — both in the public and...
Hadoop 2 puts big data environment on friendlier analytics turf
In a recent conversation with project team members from a client, one shared an internal slide deck used to promote the benefits of big data (in general) and Hadoop (in particular) among both key management decision makers and the development and implementation groups...
How one app helped 50,000 women get pregnant
Almost everyone over the age of 12 knows how babies are made. But big data is changing even that in a big way. At the very least, it’s making the process of getting pregnant a lot more predictable. Today, Ovuline announced that more than 50,000 women have used its...
Twitter ‘big data’ can be used to monitor HIV and drug-related behavior
Real-time social media like Twitter could be used to track HIV incidence and drug-related behaviors with the aim of detecting and potentially preventing outbreaks, a new UCLA-led study shows. The study, published in the peer-reviewed journal Preventive Medicine,...
How to Develop a Big Data Strategy for Better Email Personalization
If you aren’t incorporating Big Data into your email strategy this year, it’s high time to get started. According to ExactTarget’s 2014 State of Marketing survey, collecting, measuring and using behavior-based data is one of the top three most common priorities in the...
Start-Up Founded By 2 IIM Graduates Gets Rs 7.3 Cr Angel Funding
Mumbai, Feb 26: Jungle Ventures, a Singapore-based venture capital firm, has led a Rs 7.3-crore investment in Crayon Data, a Singapore-based start-up, along with support from Springs Seeds Capital, a Singaporean State-owned investor. Founded in 2012 by two Indian...
From Talent Wars to Big Data: 5 Trends Affecting Incentives
These major shifts are having an impact on corporate incentives, according to the Incentive Research Foundation. 1. The Battle for Talent Despite the unemployment figures, hiring events abound, companies are poaching A-players from their competitors, and the Baby...
The Use and Abuse of Big Data and Hadoop
Big data analytics is having a huge impact on us all. The ability to collect, store and analyze massive amounts of disparate data---using analytics platforms such as Hadoop in the cloud to uncover hidden connections, correlations and insights---is playing a bigger...
Big-Data Hiring: 5 Facts From The Field
IT recruiting always has to have a bright, young, new thing. The data science role has been hogging the spotlight for months now, with recruiters citing strong demand and CIOs griping about where to find big data pros. One key question: Should you be hiring from the...
What’s So Bad About Big Data In Little Classrooms?
When I was a kid we used to have ‘book fairs’ at school. Once or twice a year a room in the school would be transformed into a bookstore. Folding tables were covered in paperbacks. I remember glossy biographies of celebrities, movie novelizations, and TV show themed...
How much is big data compressible? An interesting theorem.
I am doing some research to compress data available as tables (rows and columns, or cubes) more efficiently. This is the reverse data science approach: instead of receiving compressed data and applying statistical techniques to extract insights, here, we are looking...
Wearable technology in business: big data benefits bring legal and privacy risks
The Consumer Electronics Show (CES), held in Las Vegas each January, reveals tech vendors' latest visions of the future. This year many of them were marketing a vision replete with wearable devices. Market analysts also envision a bright future for wearable tech....
How We’ve Been Visualising Big Data Over the Centuries
“Big data” might be a modern buzzword, but the concept goes back as long as people have been collecting records (albeit with our perception of what counts as “big” growing over the years). And while interactive data visualisations and infographics might be new to the...
As Big Data Evolves, B2B Marketers Focus On Data Management
The growth of Big Data has not gone unnoticed by B2B marketers. In an annual study from Infogroup Targeting Solutions, 54% of marketers said they have already invested in Big Data. Up to 30% of marketers said they plan to invest in Big Data for the first time in the...
Jungle Ventures Invests Rs 7.3 Crore In Crayon Data
Data analytics firm Crayon Data has received funding from Singapore-based Jungle Ventures as it prepares to expand operations in overseas markets. Singapore-government-owned Spring Seeds Capital also participated in the deal. Crayon, founded by two graduates of the...
Big Data Platform Comparisons: 3 Key Points
Our recent 16 Top Big Data Analytics Platforms collection has generated lots of interest and plenty of comments and questions. To respond to the latter, we jumped at the chance to do a Google+ Hangout with the editors of sister UBM website AllAnalytics so we could go...
Why I won’t be sharing my medical data with anyone – yet
If ever there was an open goal for big data, healthcare should be it. By gathering information from doctors, patients, drug companies, insurers, and charities, and putting the big data machinery to work on analysing it, we should be able to get better insights into a...
5 examples of big companies managing big data on Hadoop
As one of the world’s most popular free Java-based programming networks, Apache Hadoop is being used by an increasing number of companies who can no longer manage their data using traditional methods. The open-source platform can deal with a wide variety of data,...
How Big Data Could Facilitate Selective Perception Marketing
This scenario should be familiar to the many of us. Let’s say you have just purchased a brand new iPhone. No sooner had you bought the phone, than you see brand new iPhones all over the place! Another classic example is that when women get pregnant, they suddenly...
How is big data changing the film industry? Here is how!
The film industry is huge, generating in excess of $520 billion revenue in the USA in 2013. Imagine if this behemoth of the entertainment world could be further refined by taking account of consumers’ preferences, viewing habits and cultural interests? Big data makes...
5 Reasons Why Hadoop Is Ready for Enterprise Prime Time
As 2014 gets into full swing, Hadoop is increasingly being used for applications that are integral to daily business operations. No longer is Hadoop viewed by some organizations as just a platform for big data proof-of-concept applications. IT leaders should be...
Big data: Keys to the strategic positioning of PR?
The Public Relations industry has long argued for a place at the top table. We’ve been held back in this ambition by the perception that we become useful only at the tail end of the corporate value chain and by our lack of rigour with regards to measurement. Social...
Faster, more capable: What Apache Spark brings to Hadoop
Apache Spark is an execution engine that broadens the type of computing workloads Hadoop can handle, while also tuning the performance of the big data framework. Hadoop specialist Cloudera recently announced that it will offer commercial support for Apache Spark,...
Good data scientist hunting – the sexiest job of the 21st century
It may be the “sexiest job of the 21st century”, but beyond that there isn’t a great deal of consensus on how to define a data scientist. Part of the reason it’s so hard to pin down a meaningful definition of a data scientist, is because the scope of what a data...
Should big data analytics decide whether you get promoted?
Although some organisations are more political than others, a fully apolitical organisation seems unthinkable, at least for anybody who's ever had a job. However, in the age of big data, where supercomputers and machine-learning algorithms track, predict and influence...
Interview with David Cox, the most famous statistician still alive
Sir David Cox is arguably one of the world’s leading living statisticians. He has made pioneering and important contributions to numerous areas of statistics and applied probability over the years, of which perhaps the best known is the proportional hazards model,...
The dirty truth about big data and NoSQL
If I asked you for the defining characteristic of a big data customer, you'd probably say they're sitting on large amounts of data. If I asked for the defining characteristic of a NoSQL customer, you might answer they require high levels of concurrency. Well, if...
Can Twitter’s Big Data Influence The Music Business?
Twitter is a vast repository of all sorts of data on trends and users, but that data isn't always easy to delineate, even for Twitter. Now a new venture with former Warner Bros top exec Lyor Cohen's new 300 label intends to take advantage of the social network to...
Big data infrastructure goes far beyond Hadoop
Wikibon Principal Research Contributor Jeff Kelly provides an inclusive basic tutorial of the big data environment, including technologies, skill sets, and use cases, in “Big Data: Hadoop, Business Analytics and Beyond”, and while the environment starts with Hadoop...
10 Emerging Analytics Start-ups in India to Watch in 2014
Each year we publish a list of analytics startups in India that are making all the right noises and showcases the most potential among newcomers. Below is the list for this year. Read 10 Emerging Analytics Start-ups in India to Watch in 2013 Crayon Data Founded in...
Value of big data in health care is measured not just in dollars, but in lives
While wait times, access to family physicians and funding formulas make up the bulk of the debate about Canadian healthcare reform, one key to boosting productivity is better use of big data analysis in all aspects of healthcare. Indeed, the potential for big data to...
Value of big data in health care is measured not just in dollars, but in lives
While wait times, access to family physicians and funding formulas make up the bulk of the debate about Canadian healthcare reform, one key to boosting productivity is better use of big data analysis in all aspects of healthcare. Indeed, the potential for big data to...
4 best ways you can use Big Data in your job hunt
Job hunters beware: Social recruiting is the next big wave to influence the hiring process. Shon Burton, founder of HiringSolved, says his company has analyzed and created profiles of up to 100 million people by scouring their unblocked online activities on Facebook,...
Top Big Data Apps for Security
Companies are continually searching for new ways to get more value out of their data. Several months ago we published an article on how Big Data can be used to improve security, looking primarily at what the company KEYW was doing in this arena. KEYW is far from the...
Health data: some questions about profits, privacy and the public interest
It’s time for an informed public debate about the many questions raised by the commodification of “big data” in health, according to Canberra Law School academics, Bruce Baer Arnold and Wendy Bonython. Who is taking care of your health data? Bruce Baer Arnold and...
When Big Data Met Porn
In his 1987 book The Secret Museum, Walter Kendrick explored the many ways that technology transforms pornography. Technological innovations—the advent of the printing press, the rise of the home video camera, the widespread adoption of the VCR—changed the way, he...
8 Features of a True Enterprise-Grade Platform for Hadoop and NoSQL
Businesses have several options when looking for a Hadoop and NoSQL solution. The advantage of using the right enterprise-grade solution is that it can provide the dependability, ease-of-use, and speed required for real production use. Without these, you can’t deploy...
How Football is Being Transformed by Big Data
The 2013 Oakland A’s, says Allen Barra in the Atlantic, not the 2002 team depicted in the film, are the real Moneyball team. Given that football is more popular than baseball, is there a football Moneyball equivalent to the data-driven strategy exemplified in the film...
Leveraging Big Data and Cloud for Better Security
Organizations are exploring a variety of technologies and techniques to bolster their security, including big data analytics and cloud-based security services. As the sophistication of information security breaches continues to grow, organizations realize that they...
Cognitive computing can take the semantic Web to the next level
The IT industry is approaching a point where we'll soon need to distinguish between "cognitive" and "semantic" computing. The terms are blurring into each other. Actually, the distinctions among them have never been clear. Sometimes it's even easier to refer to them...
Big Data in Education: Big Potential or Big Mistake?
When learners interact with content in your course, they leave behind ‘digital breadcrumbs,’ so to speak, which offer clues about the learning process. We’re now able to collect and track this data through learning management systems (LMSs), social networks, and other...
How-to: Create a Simple Hadoop Cluster with VirtualBox
I wanted to get familiar with the big data world, and decided to test Hadoop. Initially, I used Cloudera’s pre-built virtual machine with its full Apache Hadoop suite pre-configured (called Cloudera QuickStart VM), and gave it a try. It was a really interesting and...
Interest Graphs to Choice Engine: 5 critical components to succeed: The Taste Graph [Part 4]
Complex algorithms, ontologies, and machine-learning based classifiers are needed to separate noise patterns and spam-like comments, from genuine signals. This is where the complexity arises, and the signal strength calibration algorithm has to then be customized...
5 Things Managers Should Know About The Big Data Economy
At the turn of the 20th century, most people lived as they had in the middle ages. Almost half of the US population was employed in agriculture. Life expectancy was less than 50 years. Indoor plumbing was rare, as was telephone use. There were very few cars and no...
Three Data Opportunities B2B Marketers Are Missing Today
For all of the discussion of data in marketing, and Big Data in 2014, most B2B marketers are showing they are inept when it comes to actually using data to improve the majority of their marketing. Yes, B2B marketers are doing a few things well when it comes to using...
Crayon Data shortlisted for CODE_n Innovation awards
The CODE_n contest is held as part of world’s leading business IT event CeBIT 2014 BANGALORE,INDIA: Crayon Data, an India-Singapore based big data and analytics start-up has been shortlisted as one of the finalists at the third edition of annual CODE_nInnovation...
Data Modeling In The Age Of NoSQL And Big Data
Hadoop Hbase. MongoDB. Cassandra. Couchbase. Neo4J. Riak. Those are just a few of the sprawling community of NoSQL databases, a category that originally sprang up in response to the internal needs of companies such as Google, Amazon, Facebook, LinkedIn, Yahoo and more...
Big Data Crushes Anti-Vaccination Movement
For years it’s been relatively easy to ignite medical controversy with emotional (but often anecdotal) evidence. TV is a popular format for doing just that. It’s quick, colorful and dramatic (and increasingly in high-def and big-screen). Add a well known celebrity (or...
Interest Graphs to Choice Engine: 5 critical components to succeed: Signals Vs. Noise [Part 3]
It all starts with the data…There are 2 approaches to collecting data on one’s interests, influences, and tastes: Implicit & Explicit. Companies such as Hunch and Nara have all attempted to address this by asking their users to share interests explicitly, with the...
“Mongo DB is the perfect fit… NoSQL in general is a better fit”
MongoDB, the open-source NoSQL database, was recently named "Database Management System of the Year" by DB-Engines – with good reason. DeveloperTech (DT) spoke with Matt Asay (MA), VP of Business Development and Corporate Strategy, about why customers should choose...
Sharing Your Healthcare Data Is The Ethical Thing To Do
In March, the NHS Health and Social Care Information Centre (HSCIC) will start collecting anonymised patient information from GPs in England for the very first time. It is hoped that the ‘Care.Data’ scheme will enable healthcare professionals to monitor the spread of...
Interest Graphs to Choice Engine: 5 critical components to succeed: Data [Part 2]
When you build a Choice engine, I believe there are 5 key factors that have to be considered: 1. The basic ingredient: data. Is it collected implicitly or explicitly, and what is the quality of said data in its role of inferring interests? 2. Distilling ‘signals’ as...
Big data can give athletes the winning edge
Sport at the elite level has always adopted new technologies to capture data from players during play to better understand their performance and their team’s result. Closely aligned with this is the practice of data analytics, and developments here tend to fall into...
Data visualized: The world’s refugees over four decades
To spend just a few minutes experiencing the newly created online data visualization The Refugee Project is to start to comprehend just how enormous, complex, and problematic the ebb and flow of the world’s millions of refugees has been over the past almost half...
Asian startups: Big Data on tap
In September last year, Crayon Data, a Singapore-based big data startup, hit the headlines when it raised US$1.4 million in second round funding from angel investors. Serial entrepreneurs, Suresh Shankar and his partner Srikant Sastri founded this...
Big Data Finds What Football, Basketball and Hockey Have in Common
What do basketball, football and hockey have in common? On the surface, not very much. After all, one sport is played on ice and the other two are played on either a grassy field or a wooden court. But according to University of Colorado computer science professor...
The Future of Business Intelligence: An Impending Realization
Nearly a year ago, an article written for DATAVERISTY™ on Business Intelligence (BI) stated “at some point in the near future BI analysis will simply be another routine aspect of the job for business professionals looking to maximize the use of what will be viewed as...
Big data’s next frontier: Crowd-testing fiction
Creative writing is dying, if not dead,” moaned one commenter, and he wasn’t alone. The story that provoked this particular episode of hand-wringing appeared in the New York Times’ Bits blog, and had to do with the ability of new e-book subscription services like...
Top 3 reasons Hadoop is heading to the Cloud
Cloud computing and big data have been vying for the attention of business owners for several years now. Both initiatives are compelling, as big data analytics promises the potential of powerful new business insights, and cloud computing offers greater flexibility,...
Is there a sequel to NoSQL?
The recent news that MongoDB, Inc. secured $150 million of investment capital underscored the fact that open source data is no longer in its infancy. In fact, judging from the marquee names of the investors—including T. Rowe Price and Fidelity Investments—it’s fair to...
Interest Graphs to Choice Engine: 5 critical components to succeed [Part 1]
What do Quora, Hunch, Nara, and Ness all have in common? They are all companies that generate value for their users by leveraging the power of Interest Graphs. 1. The controversy around Google’s personalised search? Based on an interest graph. 2. Facebook’s Open...
Is 2014 the Year of Personalization?
Big data was the big thing in 2013. According to execs in marketing, technology and research, 2014 will be all about personalization. Here are five of the most talked about marketing predictions from across the web. For marketers planning their 2014 priorities, it's a...
Why Most Marketers Will Fail In The Era Of Big Data
Galileo once said that “All truths are easy to understand once they are discovered; the point is to discover them.” Great marketers uncover those obvious, but unexpected truths to win consumers hearts and sell products. Marketing is, after all, the art and science of...
How new tech can help lawyers rethink their jobs in the big data age
The legal profession is inherently conservative when it comes to adopting new technologies and practices, but firms and lawyers that want to stand out in an evolving field might want to jump on the big data bandwagon sooner rather than later. Law firms took a beating...
3 tools companies can start using to harness the power of Big Data
To the individual user, Big Data might simply mean a new 3-terabyte hard drive, which can be acquired for a hundred bucks or so. But real Big Data projects require clusters of servers, vast amounts of storage, and specialized software to analyze, say, every change...
5 lessons we learned about data science in 2013
Most people know what marketing executives do every day. They try to catch people’s attention through email, ads, tweets, and press releases. As for data scientists, well, their work is not nearly as well understood. That’s been slowly changing this year as companies...
5 Steps to Improving the Customer Experience with Big Data
Technology advancements are making it easier for people to constantly, effortlessly and sometimes unknowingly generate massive amounts of data every second. Lurking in the sprawling data pipelines of global corporations are complex tangles of information that have the...
The big data paradox
oday, people are faced with too many choices and too less time to navigate information on the web. Often, this leads to a counter-intuitive situation where they are not confident about the choices they make. “So, we tried to identify various methods of organising...
How To Build A Successful Data Science Team
Is there really a data scientist shortage, or are organizations simply trying too hard to recruit a unicorn, a jack-of-all-trades who possesses both advanced technical and business acumen? If the unicorn hypothesis is true, it would explain why the scarcity of data...
Interview – Suresh Shankar, Founder at Crayon Data
Analytics India Magazine: What are some of the main tenets (philosophies, goals, attributes) of analytics approach and policies at your organization? Suresh Shankar: Crayon Data aims to simplify the world of big data. Our mission is to make data simpler, more...
Monitoring behaviour and catching cheats: Improving education with big data
Big data has got a lot of attention in the business world, but why shouldn't it be applied to other sectors as well? The ability to integrate and analyse data sets can create useful insights in all areas of life, and several startups have taken that vision and applied...
Store and ETL Big Data in the Cloud with Apache Hive
Big Data and cloud storage paired with the processing capabilities of Apache Hadoop and Hive as a service can be an excellent complement to expensive data warehouses. The ever-increasing storage demands of big data applications put pressure on in-house storage...
Which to trust: the algorithm or intuition?
The McAfee piece to which he is referring is a Harvard Business Review piece entitled: Big Data’s Biggest Challenge? Convincing People NOT to Trust Their Judgment. Patel’s post and the ensuing debate has brought my background in accounting into fresh focus while...
The Rise of the Choice Engines will Simplify Making Smart Choices
“The Rise of the Choice Engines will Simplify Making Smart Choices”, written by Suresh Shankar, appeared in the December edition of PC Quest magazine. The coverage placed Crayon Data amongst key players in the Tech space.
Big data: 5 major advantages of Hadoop
By now, you have probably heard of Apache Hadoop - the name is derived from a cute toy elephant but Hadoop is all but a soft toy. Hadoop is an open source project that offers a new way to store and process big data. The software framework is written in Java for...
Companies Offer Big Data Tools to Predict Teacher Candidates’ Impact
Can big data and predictive analytics help school leaders hire better teachers? That's the new pitch being made by two companies: TeacherMatch, from Chicago, and Washington-based Hanover Research. Both claim that their new algorithm-driven teacher-selection tools can...
The Pros and Cons of Big Data Democratization
We live in very complex times. As much as we talk about integrated insights, it only seems to be getting worse. If I sound a little bleak it’s because this statement has been shared ad nauseam: digital marketers are literally drowning in data. Data sources range from...
Five Examples of Secrets Hidden in Your Data
As big data volumes continue to explode, businesses are challenged to quickly extract rich insight from the mountain of machine-generated data streaming in from devices, sensors, smart meters, operational equipment, social-media platforms and other sources. According...
Avoid 5 Deadly Big Data Mistakes
Today, many companies are already using Big Data to turn their data process into actionable insights. However, in reality, companies often find that Big Data doesn’t necessarily translate into easy success. In an exclusive interaction with CXOtoday, Adrian DeLuca,...
Data Meets Retail
Crayon co-founder Srikant Sastri’s article “Data Meets Retail”, which explains the role of Big Data Solutions in retail in India, was published in the December edition of Images Retail magazine. It appeared in a special section called Perspective.
The three most common ways data junkies are using Hadoop
Analytic applications come in all shapes and sizes–and most importantly, are oriented around addressing a particular vertical need. At first glance, they can seem to have little relation to each other across industries and verticals. But in reality, when observed at...
Privacy and technology: How simple photos are leading to full profiles of ordinary people
We’re no longer just strangers in a crowd. Imagine any street corner in any Bay Area town where, let’s say, four people — Alexandria, Larry, Cory and Cameron — are lost in private thought. Without a single conversation, without even knowing their names, we could learn...
Big-Data Visualization Is Worth A Thousand Spreadsheets
Big data is a big plus: You can collect, organize, and store vastly more information than before. Data about your customers, products, suppliers, sales, and finances—everything that makes your company run—can be quantified, stored in one place, and analyzed. But...
How does a Choice Engine overcome the choice problems?
Consider a typical decision – you have just arrived in a new city after a 6 hour flight, it’s 6.30 PM and you want to get a decent vegetarian meal. What do you do?You fire up Google, search for good restaurants in that city, and filter them by type, price, location...
Sales bonus and big data
Leaving behind the old way of sales, companies now have become extremely sophisticated when it comes to helping sales and marketing teams achieve their numbers, and hence the green bucks that inadvertently flow after the quota is achieved, by being creative on Big...
Big data tool harnesses 125 years of infectious disease info to boost collaboration
The increasing availability of big data from the likes of CMS to get a better handle on population health patterns is a growing trend in healthcare. Health IT startups such as Social Health Insights and Konnectology are just a couple of examples of that in action. But...
Big Retail Is Watching You: Exposing Walmart’s Massive Data Collection Schemes
Outside of its growing reputation for poverty wages, worker intimidation and an overall culture of employee repression, a new report released Wednesday reveals that retail giant Walmart is also throwing its weight behind a massive consumer tracking effort with...
Big Data tools and sports becoming winning combination
Data have long fueled the relationship between sports and their fans. Now, with Big Data tools enabling the collection and analysis of seemingly limitless types of sports-related information, the flow of sports data is poised to accelerate. "If there's data out there,...
Not all big data is created equal, warns marketing scientist
The new frontier of data is full of possibilities, but marketers need to be aware that not all data is created equal, a leading marketing scientist warns. Associate professor and director of the not-for-profit Ehrenberg-Bass Institute, Dr Rachel Kennedy, told...
5 Ways Big Data Can Make Our Cars Smarter
Cars are probably not the first thing that comes to mind when you hear someone talk about data sources, but today’s cars generate approximately 15 gigabytes of data per hour. The reason most of us are unaware of this fact is because most of this data is only stored...
Big Data Requires Focus; Key Ecommerce Metrics
Online consumers generate an avalanche of data. Product purchases; Facebook Likes and comments; Twitter follows; Pinterest pins; on-site searches; pages viewed; recommendations; clicks — consumers do it all, from different devices and locations. And it generates lots...
Online Tools Give Consumers the Power of Big Data
Often, the best way to avoid a rip-off is to know what others are paying for things. What did the guy sitting next to me on the plane pay for his ticket? How much did the neighbor pay for his kid's wedding? That emergency dental procedure? The water pump replacement...
Behind all great big data strategies is a great editor
What do Oscar-winning movies, the perfect Christmas card, and a great analytic strategy all have in common? They're all byproducts of great editing. Editors are often unsung heroes of masterpiece productions, and your big data analytic strategy is no different. In my...
Modeling Human Behavior and The Rise of Big Data
In the wake of the 2012 Presidential election, hundreds of Internet users asked a very important question: “Is Nate Silver a witch?” The accusations of witchcraft came after Silver, statistician and blogger for the New York Times, predicted the winner of all 50 states...
How Big Data is Replacing the Job Interview
Recruiting has always been seen as an art. Employers and recruiters spend hours writing job descriptions, scouring over resumes and conducting interviews to try to judge which candidate will be the best fit for a position. Many of the factors these businesses use to...
How is Big Data Transforming Your 80/20 Analytics?
Even today, most organizations technically struggle to answer even the simplest 80/20 analytics questions: Which 20% of customers generate 80% of the profits? Which 20% of suppliers are responsible for 80% of customer UX complaints? What 20% of customers facilitate...
How Formula 1 turns 3D printing, big data and crappy bandwidth into sport
What draws people to engineering and technology is generally the effort to create the best possible product within a series of technical or physical constraints. That same effort to solve a problem underlies the technology behind a Formula 1 race team, although like a...
Search vs Choice – A new type of engine for a new kind of age!
15 years ago, the defining problem of the Internet age was how I find the information I need? This gave birth to search engines and Google’s great vision of “organizing the world’s information”. A decade later, as Google continues to improve its search with...
The benefits of Big Data solutions for enterprises
Big Data seems to be quite the buzzword these days. So, what is Big Data exactly? Basically, it refers to a collection of data sets or information too large and complex to be processed by standard tools. It is about the art and science of combining enterprise data,...
How big data is transforming the world of hospitality
Over the past few years, Big Data has transformed the way the hospitality, travel and tourism sector (HTT) functions. It has lodged a new dimension of customer care in HTT by opening up mind-boggling possibilities -- spurring a 'paradigm shift' from simply viewing...
Is NoSQL less disruptive than we thought and just, well, useful?
Remember a few years ago when all the talk around emerging NoSQL databases was how they’d make history of relational databases any time that scale was an issue? Well, that hasn’t exactly happened yet, and Hummer Winblad Managing Director Mitchell Kertzman has a theory...
Design creates culture. Culture shapes values. Values determine the future.
"Design creates culture. Culture shapes values. Values determine the future." This quote from Robert L Peters best defines what Crayon is setting out to do. We are having a design discussion, as we shape our products, our culture and our future. This exercise is also...
Big Data: The Key To Better Health Care?
The discussion about how to fix health care in America shouldn't focus on websites or servers or insurance mandates or policies. It should focus on better data analytics. That's according to a panel of experts who spoke Tuesday at the TechAmerica Foundation's Big Data...
Army finds its own way to build a big data system
The Army is harnessing big data to help commanders answer questions that were previously unsolvable by pulling relevant information from thousands of databases into a holistic common operating picture, specifically focused on readiness and resources. The Army...
Why Real-time Data Trumps Big Data for Triggered Emails
Big data has been getting a lot of press lately. There’s a whole treasure trove of information you can use for your email marketing campaigns. You can look at everything from customer purchase history, to what similar customers bought, to a customer’s behavior on your...
Despite Privacy Concerns, Identified Rolls Out Tool for Smarter Recruiting
LinkedIn is an essential tool for recruiters, a network where individuals’ professional qualifications are emphasized and easily searchable. But according to Adeyemi Ajao, the co-founder of Identified, a big data and analytics company focused on professional...
The Marriage of the Giants: Big Data and Big Box
What does Big Data have in common with Big Box Retailing... besides the bigness? How much does Walmart have in common with Amazon... besides the obvious? Where do you think retail is going... besides mobile? Let’s start back a week. Twitter and Royal Mail go public in...
A Unified Customer View is the Key to Leveraging Big Data in Retail
At no point in human history have organisations and individuals had more access to information than they do today. For retailers, technology is capable of revealing truly beneficial insight regarding potential and existing customers, but for many this information...
Google’s flu fail shows the problem with big data
When people talk about ‘big data’, there is an oft-quoted example: a proposed public health tool called Google Flu Trends. It has become something of a pin-up for the big data movement, but it might not be as effective as many claim. The idea behind big data is that...
Cloud Predictive Analytics Most Used To Gain Customer Insight
Using analytics to better understand customer satisfaction, profitability, retention and churn while increasing cross-sell and up-sell are the most dominant uses of cloud-based analytics today. Jim Ericson and James Taylor presented the results of Decision Management...
How retailers can keep up with consumers
The North American retail landscape looks quite different today than it did even ten years ago. The way that consumers make purchasing decisions has dramatically altered: they stand in stores, using their smartphones to compare prices and product reviews; family and...
Visualizing the Big Data of Breast Cancer
A colorful wheel developed by Rice University bioengineers to visualize protein interactions has won an international competition for novel strategies to study the roots of breast cancer. The winning BioWheel by the Rice lab of bioengineer Amina Qutub was chosen this...
Indian-Singapore start-up to launch consumer 'Taste Graph'
An Indian-Singapore start-up is set to launch next year a “Taste Graph” data that could revolutionise consumers relationships with service-based business houses through information technology. The Taste Graph would provide simple data on specific requirements of...
How to Ensure Your Big Data Initiatives Pay Off
Big data has gone mainstream. Everywhere you look, people are talking about how to extract value from the massive and ever-growing troves of information residing within and beyond traditional data repositories. Information that, when fully leveraged, has the potential...
Start-ups raising VC funds face irony
RAISING money has become more challenging now given the current uncertain climate in global financial markets, said Suresh Shankar, founder and director of Singapore-based data analytics start-up Crayon Data, in an interview with The Business Times. But at the same...
Where’s My Magic 8 Ball? Customer-Focused Analytics
Your customers know that they have handed you a lot of their personal information through a variety of interactions and channels. And your customers fully expect that your company has done its job and processed all that information to know what they want, how to give...
Government – A Big Data Analytics playground
From citizenship applications, employment status, filing taxes to census recording – all provide massive amounts of data to the Government. A torrent of data in the form of medical files, satellite images, financial records and social media information. To a Big Data...
The single biggest reason Red Bull dominates Formula 1
Formula 1 fans were treated to a very entertaining race in Korea on Sunday. Did you catch the race? There was plenty of incident,. but once again Sebastian Vettel lined up in pole position and was in front from start to finish. It was another masterful drive from the...
Top 10 successful big data sandbox strategies
Being able to experiment with big data and queries in a safe and secure “sandbox” test environment is important to both IT and end business users as companies get going with big data. Nevertheless, setting up a big data sandbox test environment is different from...
Big data has made privacy obsolete
Watching the legal system deal with the Internet is like watching somebody trying to drive a car by looking only in the rear-view mirror. The results are amusing and predictable but not really interesting. On the other hand, watching the efforts of regulators —...
Crayon offers 'Simpler Choices' for Big Data analytics
Crayon Data, an emerging player in the big data market, is looking to simplify the big data concept in the enterprise. After raising Rs 8.75 Crore in its second round of funding from investors, it is eyeing a Series-A round funding in early 2014. Suresh Shankar,...
Big data – a marketer’s dream or dilemma?
Everyone is talking about "big data", especially in marketing where it promises to enable a new era of perfectly targeted messaging for the modern, connected consumer – or is that just hype, soon to evaporate among privacy fears, confusion, and the technical...
Big Data Success Means Investing in Talent and Tech
With all signs pointing to greater Big Data spending by businesses, the question is: Do you want to get the most out of your investment? To do so, experts say businesses shouldn't just invest on Big Data technologies, but on Big Data talents as well. Gartner, an...
Big Data in Human Resources: A World of Haves And Have-Nots
I've written several times about the Datafication of HR and BigData in Human Resources, explaining the tremendous business opportunity companies have to leverage their employee data to improve operational performance. This week we introduced research conducted over...
Why big data has so far failed medicine
Can big data improve human decision-making? That was the question that MIT's Irving Wladawsky-Berger put to a panel of IBM clients at the company's research colloquium on cognitive computing Wednesday. The meat of his discussion focused on Douglas Johnston, a surgeon...
Big Data, Small World: Kirk Borne at TED
Dr. Kirk Borne is a Multidisciplinary Data Scientist and an Astrophysicist. He is Professor of Astrophysics and Computational Science in the George Mason University School of Physics, Astronomy, and Computational Sciences (SPACS). He received his B.S. degree in...
Barry Schwartz: The paradox of choice
Psychologist Barry Schwartz takes aim at a central tenet of western societies: freedom of choice. In Schwartz's estimation, choice has made us not freer but more paralyzed, not happier but more dissatisfied.
How to Close the Big Data Skills Gap by Training Your IT Staff
It's difficult to talk about big data without also discussing the big data skills gap in nearly the same breath. But is it as bad as it seems? According to a recent CompTIA survey of 500 U.S. business and IT executives, 50 percent of firms that are ahead of the curve...
Sheena Iyengar: The art of choosing
Sheena Iyengar studies how we make choices -- and how we feel about the choices we make. At TEDGlobal, she talks about both trivial choices (Coke v. Pepsi) and profound ones, and shares her groundbreaking research that has uncovered some surprising attitudes about our...
Big Data Startup Crayon Data Wants To Chase Non Linear Revenue Growth, The Holy Grail of Indian IT
Non linear revenue growth has been the holy grail Indian IT companies have been chasing for many years. That is, how do you grow revenues without a corresponding growth in the number of people and costs? A platform play is an obvious choice. The growth in big data...
How to Escape the Dark Valley of Your Hadoop Journey
It happens to the best of us. You know your business is bursting with useful data, and you’ve only begun to scratch the surface. So you strike out to build an analytical platform, using all the great open-source tools you’ve been hearing so much about. First, you have...
Navigating the Big Data Nuances of Financial Crimes
Catch me if you can. That seems to be the mantra of the modern day cyber thief, fraudster or money launderer. Cloaked behind anonymizers, spoofing and botnets, criminals employ extensive means to go undetected. While advances in technology can provide organizations...
How to use Hadoop to mine business value from new types of data
Apache Hadoop has emerged as the leading technology for helping companies mine big data. And while every organization is different, their big data demands are often similar. Hadoop enables companies to collect and process massive amounts of data that was once thought...
Big Data Made Visual
“Computers in the future may weigh no more than 1.5 tons” - Popular Mechanics, forecasting the relentless march of science, 1949. Predictions. Rock solid, one day. Busted as myth, the next. But the relentless march of our imagination – and creation - must go on....
Crayon Data raises more than 1.75 mil SGD in second round of funding
BANGALORE, INDIA: Crayon Data, a Singapore-based big data start-up with a key development centre in Chennai, announced that it has raised in excess of 1.75 million Singapore Dollars (SGD) in its second round of fund raising since the initial round of funding at the...
What happens when media & big data collide?
If there was a PR utopia, it would look something like this: Businesses would send journalists timely, interesting stories. Journalists would receive an ongoing supply of quality information. Readers, in turn, would grow enthusiastic about brands, turning media...
70% of marketers feel ill-equipped to handle “big data” challenges, says GfK report
Marketers feel underqualified to handle the challenges presented by the rise of “big data”, with 70 per cent admitting they don’t have the necessary skills, according to a GfK and Guardian Media Network report. The report, which polled the views of 1,011 UK consumers...
7 Rules of Big Data in Banking
As we move from an era of automating transactions to the era of data, the information challenges banks face are increasingly around understanding and creating intelligence from information. Areas like regulatory compliance, customer insight, or real-time offer...
Tableau Simplifies Big Data Banking
Tableau Consumer Conference 2013 features many of the company’s big data customers who have found ways to use Tableau’s technology to make their businesses better. One such company is Barclays, a UK-based multinational banking and financial services company. At...
Gamification: The Killer App for Achieving the Promise of Big Data
Each day 2.5 quintillion bytes of new data are generated around the world-from tweets and Facebook posts to comments and blogs to data being collected constantly on mobile phones and connected devices, this nearly incomprehensible mountain of information is...
Hadoop Big Data Java – Manufacturing scenario
Purpose: The purpose of this document is to explain how to apply the power of Hadoop Big Data platform in Manufacturing scenario. Challenge: Big data is the term for a collection of data sets so large and complex that it becomes difficult to process using on-hand...
Is Hadoop a New Storage Paradigm?
According to historians of economic theory, there are two ways to define creative destruction. One is negative—that capitalist systems unleash forces that lead to their own demise. The other is positive—that those forces result in better products and services that...
How Big Data Is Changing Football on and off the Field
While most sports fans probably think baseball—and more precisely, Moneyball—when considering sports and data, there is a quiet data revolution that has been percolating in the world of football for some time. Data is driving player training, play calling and...
Big data analytics changing the rules of road travel
San Francisco-based radio cab service provider Uber started operations in India recently. While one can book Uber’s luxury taxi service using a smartphone app, most Indian radio cab players, too, are using similar technology, making use of big data to understand the...
Why gamification and big data go hand-in-hand
Gamification is winning over the corporate sector as a toolkit for motivating the next generation of consumers and users. But it’s the potential of gamification and big data together that is getting industry pundits increasingly excited. Gamification’s ascent from...
Workday tools up HR and finance with big data analytics
Enterprise cloud apps company Workday is adding big-data tools in its next release, allowing firms to mix large volumes of diverse outside data with internal information for HR and financial reporting. In Workday 20, which becomes generally available this week with...
Hadoop Tutorials: Using Hive with HBase
Here is another interesting use case that came up when I was working with one of our clients in the insurance industry. The client had enormous amount of claim data residing in multiple databases in SQL Server which were to be consolidated into one. Some of the...
Legal Issues Raised By The Use Of Web Crawling And Scraping Tools For Analytics Purposes
In 2010, Pete Warden, a software engineer living in Colorado, developed a software program to “crawl” publicly accessible Facebook pages and “scrape” (i.e., collect) information relating to Facebook’s members. Within hours of deploying his software, the application...
How to Tell the Fashion Future?
The fashion industry always wants to know about the next big thing, so Julia Fowler thinks it should take a closer look at the world’s financial markets. Just as old-style traders have been all but replaced by analysts crunching vast amounts of data, Ms. Fowler said...
Big Data Shakes Up Security (Which Isn’t Always Good)
Cyber-security and cyber-criminals are engaged in a constant arms race. The minute a software vulnerability gets patched or a security tool is able to block a class of attacks, malware writers shift gears and look for something new to exploit. For years, security...
Big Data and Payments Drive Loyalty in Consumer Banking
With less than 3 weeks left to Sibos I am eagerly awaiting the start of the conference. One of my favorite events within the event is Innotribe - a forum for forward thinking debate, discussion on innovative technologies, learning about new solutions, and...
Durkheim Project Leverages Big Data to Prevent Veteran Suicides
Suicide has grown to epidemic proportions among U.S. veterans of Iraq and Afghanistan, and the Pentagon and U.S. Department of Veterans Affairs is hoping that social media and big data can help them identify at-risk veterans and get them the care they need. Last year,...
How Big Data Analytics Can Boost Network Security
Conventional wisdom has it that a surefire way of getting your credit card black-flagged is to fill up two or more vehicles with gas and purchase expensive sneakers within a short space of time. That's because this type of spending behavior is rare, and it's also...
Data Storage: REST vs. POSIX for Archives and HSM
Here is my working hypothesis: POSIX (Portable Operating System Interface) can’t scale to meet the demand of clouds and big data, but REST (Representational State Transfer) can’t manage and tier data the way POSIX can, but will likely get those features in the next...
Researching ALS through new eyes — Why data science will foster life science breakthroughs
Amyotrophic lateral sclerosis, commonly known as ALS, is an enduring mystery for life scientists--even those who have dedicated their careers to understanding it and finding a cure. Terminal, essentially untreatable, and frustratingly hard to predict, the disease has...
Clinical data analytics next big thing
The clinical data analytics market is about to get red hot. With the shift toward new payment models and the sheer amount of clinical data contained in electronic health records, more and more healthcare groups are looking to analytics solutions for population health...
This man thinks big data and privacy can co-exist, and here’s his plan
If you’re in the tech industry, particularly the U.S. tech industry, then you’re probably not a big fan of European privacy officials. And German data protection regulators may well feature in your nightmares. After all, Germany is literally the birthplace of data...
To Go from Big Data to Big Insight, Start with a Visual
Although data visualization has produced some of the most captivating artistic displays in recent memory, some of which have found their way into exhibits at the New York Museum of Modern Art and countless art installations around the world, business leaders are...
How small businesses can mine big data
As buzzwords go, big data is currently one of the most powerful—and one of the most perplexing. It sounds like something only multinational conglomerates can afford. But the concept of analyzing very large amounts of data and looking within it for patterns, trends,...
Marketing and IT jump into bed for ‘Big Data’
There are two things emerging marketing channels have in common: they’re based on technology and they’re social. While traditional media are either remaining steady or making small gains in terms of marketing spend, and while print media suffers sharp declines,...
Data mining personalizes direct mail
Big data is turbocharging the old-fashioned business of direct-mail marketing, giving dealers the ability to pinpoint prospective customers and tailor their appeals right down to the desired monthly payment. It's a departure from the days of "spraying and praying,"...
Interview with Dr. Saed Syad, Chief Data Scientist at AdTheorent
Saed Syad: With big data, even without having any predictive modeling, we can answer some of those questions by looking into the database. What we call targeting or retargeting is some sort of a database query that makes it faster than you do it in memory. Just having...
Software AG Launches Real-Time Analytics Platform for Intelligent Business Operations and Announces Acquisition of JackBe Corporation
Software AG (TecDAX: SOW AG) today announced the launch of its new webMethods Intelligent Business Operations Platform (IBO) and the acquisition of JackBe Corporation. JackBe is a privately-held company with headquarters in Chevy Chase, Maryland (USA) and a provider...
Media: The impact of high definition and managing Big Data
The Media and Entertainment (M&E) industry not only manages big data every day, it has been dealing with it longer than most other industries. In the last decade, M&E has made a mass migration to digital media formats for audio and video capture, production, delivery,...
Crayon Data democratizing the use of Big Data
Currently funded by a select group of private individuals and angel investors, Crayon Data is defining the Big Data landscape by not just creating products and applications but by nurturing and developing ideas and thought leadership. With a development centre in...
Top 3 trends in data visualization
Data visualization is core to business intelligence (BI), as it’s the means by which data is communicated to the user. Results, alerts suggestions, or even more controlled guidance—all of these elements are passed through the data visualization function of a BI...
Key steps to big data security in higher education
Higher education is a sector that lends itself to big data analytics. Organisations can draw on multiple sources of education-related information – such as courses and research – and may use Web 2.0 and social media channels as part of this. At the same time, higher...
Design with Big Data in Mind : The HP Angle on Analytics
Dozens of Big Data movers and shakers stopped by theCube at the recently concluded HP Vertica Big Data Conference to share their insights into the hottest trend in IT. Among the guests was Yammer analytics director Pete Fishman, who discussed how his team uses Vertica...
MemSQL Sets the Bar for Real-Time, Big Data Analytics
SAN FRANCISCO, CA--(Marketwired - Aug 20, 2013) - MemSQL today announced a benchmark report that establishes the company's distributed in-memory database as the standard for real-time, Big Data analytics. Conducted by leading IT market research firm Enterprise...
Busted! A MythBusters Approach To Big Data Analytics
During a Big Data analytics project you uncover a ton of information about your business. All of it expands your knowledge, and some of it leads you to new and interesting hypotheses. The trick is to quickly and iteratively test those hypotheses to get the most value...
Data warehousing, BI guru returns to SA
Big data, the newest addition to the strategic direction of data warehousing, will be under the spotlight when data warehousing and business intelligence (BI) guru, Ralph Kimball, presents in SA. ITWeb will host Kimball at the Building a Data Warehouse for the Decade...
Big Opportunities From Big Data, But Barriers Remain
A new survey of telecom operators shows that they are in a good position to generate new opportunities with "big data." Fifty-eight percent of those interviewed for the Informa Telecoms & Media research agreed that the main long-term driver for big data will be...
Why ‘big data’ is essential for retailers exploring the omnichannel
Many retailers have struggled with the pace at which the multichannel world is moving, with consumers increasingly expecting the same levels of service across a variety of different channels. Just when most retailers thought that they had a handle on delivering on...
Why it’s way too early to dismiss big data’s economic impact
The New York Times published an article on Friday asking in its headline whether big data is a economic dud. Numerous quoted professors gave reasons why the analogy of data to oil doesn’t work, nor does the one of data to the electrical grid (and here I thought that...
Building big data? Are you building a security headache too?
The world and its dog has been shocked by the Prism news story. Early in June, we found out that the US National Security Agency (NSA) had developed a secret data-gathering mechanism to steal all our data and store it in a large data warehouse. We are outraged that it...
Gate 6 marries marketing creativity with Big Data analytics
hese days, it seems as if I can't get through a conversation with someone from the high-tech industry without that person referring -- usually in super-reverent tones -- to the vast potential of social media engagement combined with analytics to transform customer...
Tap And Track: How Big Data, Games, And Auctions Are Maximizing Mobile Commerce
Better, faster, more customer friendly. This week, a slew of apps and e-commerce boosters were announced, each relying on big data to home in on the sweet spot that converts browsing to buying. You sipped a crisp, white burgundy with your entree at a local bistro last...
Five Big Data Trends Revolutionizing Retail
It’s Moore’s law of marketing: The aggregate amount of data available to retailers doubles every two years. Acquiring the insights that retailers need to find, target and retain their ideal customers used to be a problem. Now, marketers are tasked with sifting through...
Data visualization: Beneficial but perilous
There's something about data visualizations that make people trust them more than they should. Just ask engineer and data scientist Pete Warden. He created and published a data visualization that used public Facebook profiles to draw conclusions about regionalism in...
How Mobile Marketing Is Being Shaped By The Explosive Growth In Big Data
Here's an overview of the relationship between big data and mobile: First, big data needs to be defined: Big data is most commonly defined as data sets that meet three attributes, known as the three "Vs": volume, variety, and velocity. But there is something more to...
LinkedIn connects big data, human resources
Every second, more than two more people join LinkedIn's network of 238 million members. They are head hunters in search of talent. They are the talent in search of a job. And sometimes, the career site for the professional class is just a hangout for the...
Increase big data business value with a strong BI team
While big data has entered the mainstream and gets airtime on network news shows, the people who manage data and deliver insights are enjoying no new celebrity or fame. In many organizations, they've become nearly invisible. I'm talking about business intelligence and...
Can big data technology be used to replace creative marketing?
Using big data is a careful balance of staying relevant and being creepy, according to marketers at the SAS Forum 2013 in Sydney on Thursday, but it's also left many arguing how much creativity can be replaced with technology. That's the message debated between...
The Hardest Part of Data Visualization : Simplicity + Consumability
With visualization, there is an “ongoing challenge with the size of data and disparity of it,” according to Lilford. From a customer challenge perspective, “it’s really about getting to the size of data, all the sources of data, all the scale of things.” With Tableau...
Srikant Sastri to become co-founder of Crayon Data
Singapore-headquartered Crayon Data, with a development centre in India, has announced appointment of Serial entrepreneur and digital evangelist, Srikant Sastri, as its Co-Founder. Srikant said, I am excited about the way Crayon brings together the world of enterprise...
Indian Services Entrepreneurs Venturing Into Products: Suresh Shankar and Srikant Sastri, CoFounders of Crayon Data
uresh Shankar is the Founder Director at Crayon Data. Prior to founding Crayon Data he held various positions at IBM, RedPill Solutions, ANN AMRO Bank, Young & Rubicon and JWT Fulcrum. He was the CEO at RedPill which was sold to IBM in 2009. Srikant Sastri is...
Big Data, Analytics And The Future Of Marketing And Sales
Big Data is the biggest game-changing opportunity for marketing and sales since the Internet went mainstream almost 20 years ago. That statement often prompts vigorous head nodding from executives, but is quickly followed by head scratching. “How can we make this...
Big pharma mobilising patients in battle over drugs trials data
The pharmaceutical industry has "mobilised" an army of patient groups to lobby against plans to force companies to publish secret documents on drugs trials. Drugs companies publish only a fraction of their results and keep much of the information to themselves, but...
The Metaphysics of Big Data: Problem of Induction
How do we, human beings, acquire knowledge? Think for a sec! And don't tell me that it is from text books, TV, internet or newspapers etc!!! Scholars say that there are mainly (debatably) five valid ways (Perception, Inference, Comparison, Verbal Testimony and...
Amplifying security intelligence with big data
Leading security intelligence solutions today rely upon a set of structured and semi-structured data sources, including logs, network traffic and others, to provide the Security Operations Center with an on-going real-time view of their organization’s security...
Top 10 categories for Big Data sources and mining technologies
Most discussions on organizing Big Data center on repository frameworks – specifically Hadoop clusters and MapReduce frameworks. This technology-focused view often overlooks the most important question, “What are you planning to do with the data you’re collecting?”...
Top 10 categories for Big Data sources and mining technologies
Most discussions on organizing Big Data center on repository frameworks – specifically Hadoop clusters and MapReduce frameworks. This technology-focused view often overlooks the most important question, “What are you planning to do with the data you’re collecting?”...
Are we headed for a platform change for Big Data?
Organizations are just digging in with big data, and already portents of change are forming. This time, it appears to be some re-thinking by HPC (high performance computing) vendors about which hardware delivers the best processing for big data payloads that need to...
Crayon Data Founder, Suresh Shankar analyses Big Data over the last 13 years
With ever increasing volumes of data, companies have started stressing a lot on how to make sense accurately from this and hence the filed of Big Data has emerged. From retail to telco and pharma to digital marketing, data and analytics are becoming core to...
Spoilt for Choice – How to choose the right Big Data / Hadoop Platform?
Big data becomes a relevant topic in many companies this year. Although there is no standard definition of the term „big data“, Hadoop is the de facto standard for processing big data. Almost all big software vendors such as IBM, Oracle, SAP, or even Microsoft use it....
Spoilt for Choice – How to choose the right Big Data / Hadoop Platform?
Big data becomes a relevant topic in many companies this year. Although there is no standard definition of the term „big data“, Hadoop is the de facto standard for processing big data. Almost all big software vendors such as IBM, Oracle, SAP, or even Microsoft use it....
Thamzi and Big Data
Have you ever read an autobiography and felt that the bugger is almost living your dream? One such book I read was "Don't ask any old bloke for directions" by Palden Gyatso Tenzing. Some 200 odd pages full of what I have always dreamt of doing. This book is by an IAS...
Installation of HBase in the cluster
HBase is an open-source, distributed, versioned, column-oriented store modeled after Google 'Bigtable’. This tutorial will describe how to setup and run Hbase cluster, with not too much explanation about hbase. There are a number of articles where the Hbase are...
Big Data Start-up Crayon Data Shares Plans for India Market
The big data-analytics start-up, Crayon Data, has announced its plans for India market. Crayon already has a few customers in the US and UK markets, and its founder and CEO, Suresh V Shankar is confident that they can replicate the success in other emerging markets in...
Big Data and the Paradox of Choice
"Autonomy and freedom of choice are critical to our well-being, and choice is critical to freedom and autonomy. Nonetheless, though modern Americans have more choice than any group of people ever has before, and thus, presumably, more freedom and autonomy, we don't...
Big Data – how airlines should use it more effectively to boost ancillary revenue
As we all know, the airline industry today is extremely interested in two major trends – Big Data and ancillary revenue. Everyone is seemingly talking Big Data, but what is it exactly and how does it help an airline to make money? And then, of course, we have the...
5 Ways Big Data Can Help Retail Supply Chains
The supply chain is a critical component of a retailer’s business. This article identifies fives ways in which big data can enhance a retailer’s supply chain processes. 1. Real-time Delivery Management Tracking delivery of shipped goods has improved over the years....
'Big Data vs Intuition' paradox explained
Every business leader will have a dad's story, I believe, -- a memory or nostalgia how he/she was inspired by the attitudes/thoughts/decisions (sometimes seemingly tough, painful and even stupid) of his/her dad in paving a unique way to success and beyond. Here is one...
Big Data – Hype, Reality, Bill Gates and the Buddha
Today's NYT carried a very good blog post on the six myths surrounding big data. Contrast this with another equally good article on the premise of Big Data - that "...hidden in all that information lies a hyper-resolution map of the world's behavior in space and...
Aviation Needs to Tap Big Data for Engine Design, Maintenance
The aerospace industry, and specifically jet engine manufacturers and the airlines that are their customers, need to catch up with the Big Data revolution, writes Wikibon Big Data Analyst & Principal Research Contributor Jeff Kelly in his latest Wikibon Alert, “Big...
Big Data can set your mind free…
In my last post, I talked about the paradox of big data, or how data can be misery... In today’s post, I will talk about how that paradox can be resolved... and how data can set you free. Consider these statements by some of the leading writers, academics and...
Why You Should Not Build a Recommendation Engine
Recommendation engines are arguably one of the trendiest uses of data science in startups today. How many new apps have you heard of that claim to "learn your tastes"? However, recommendations engines are widely misunderstood both in terms of what is involved in...
The Paradox of Big Data… or Why Data is Misery
We've heard it shouted from every rooftop recently– big data is the next big thing, we’re a data-driven company, data underlies everything we do.... So here’s a paradoxical, even heretical thought: Data is misery. Why? Consider this. We are all becoming adherents of...
Big Data Is Going To Save The Film Industry, Too
As we approach the critical summer box office sales period, there's no doubt the media industry has experienced a fundamental shift. We've evolved from business models fueled by analog content and scarce distribution to a digital world of empowered consumers who are...
How big data can revolutionize pharmaceutical R&D
The big-data opportunity is especially compelling in complex business environments experiencing an explosion in the types and volumes of available data. In the health-care and pharmaceutical industries, data growth is generated from several sources, including the R&D...
A view from the Box
This is a space for us to crayon our thoughts, our points of view. On data. On analytics. On whats the latest. On whats coming your way. On how its going to change your life in maybe small, in maybe significant ways. In all possible aspects. The enterprise view. The...
Why consumer packaged goods companies love big data
If I could sum up “success” in the consumer packaged goods (CPG) industry in one word, that word would be SPEED —the speed at which the best companies take advantage of the latest consumer trends and defend their market share against the competition. Quickly changing...
6 Uses of Big Data for Online Retailers
Most small merchants think that Big Data analysis is for larger companies. In fact, it is important for small businesses, too, as they attempt to compete with the larger ones. This becomes even more important as online retailers interact with their customers in real...
How The Cloud And Big Data Are Changing Entertainment
It's a classic case of the disruptive entrants and a strategic inflexion point: New technology threatens the status quo, as streaming video propelled through the cloud by big data takes on the DVD and Blu-ray. Discs are a venerable golden goose egg for studios, which...