By Andy McCue, 4 May 2006 11:10
NEWS
Poor quality and duplicated data is leading to bad decision-making and undermining costly investment in business intelligence projects, according to industry experts.
Business intelligence software allows organisations to pull together information from various corporate systems and produce reports that aid management decision-making.
But industry players, analysts and customers warn that those decisions are often being made using poor quality, out-of-date or incorrect information because of a failure to undertake data-cleansing exercises first.
Speaking at business intelligence vendor Business Objects' customer conference in Cannes this week, Gartner analyst Andy Bitterer said most companies have about 200 data sources and much of it is poor quality and inconsistent.
He said: "There's not one company that doesn't have a data quality problem."
Mike Pratt, data integrity manager at Business Link London (BLL), said many companies simply think business intelligence tools will solve all their problems without thinking about the quality of the data that the tools will draw upon.
He told silicon.com: "Many companies don't do the groundwork. They need to ask what is it they are trying to measure; do they capture that data; and do they have the quality of data to make it meaningful."
Pratt said that by cleansing, purging and standardising its customer database as part of a business intelligence project, BLL has reduced duplication of addresses and contact details from 50 per cent to 2.5 per cent and saved £100,000 by not sending out irrelevant marketing to people who don't even exist.
Business Objects chairman and chief strategy officer Bernard Liautaud told silicon.com that data quality is one of the biggest headaches for CIOs and senior executives today because of the amount of information enterprises hold.
He said: "There's too much data and it's duplicated hundreds of times. The mistake companies make is that they start from the data they have. They need to ask what data do their users need and what are the questions they are asking. Understand the questions, how they can be answered and what kind of data is needed."
Comments
There are 5 comments. Join the discussion
1. Iain Benger-Stevenson
The same old story. Will businesses never learn that short term gain will always produce long-term headaches? You get nothing for nothing!
2. Nick Cole
Organisations are too ready to assume that information in a computer myst be right and infallible.
Executives must use the information they have to help them make an informed decision. Using their experience and common-sense for which they presumably get paid vast salaries.
Sometimes poor data is the result of inadequate specification or short circuiting the data design process based on a restricted view of the holistic environment. This is also a result of executive short sightedness and rush to make something live before it is really ready.
3. David C Pitt
Businesses generally, and IT in particular have to get away from the presumption that data has any significance of and in itself. Data is only useful when part of a larger Process, it has no real meaning. The Data Warehousing industry makes the critically wrong assumption that data for its own sake is a Good Thing, without considering context.
Data is Dead, until a Process gives it Life!
4. Roger Huffadine
Crap in = Crap out. I thought this was the first module of 'Computers Studies 101'. A very interesting bit of research in the 1980's demonstrated the faith that people, mistakenly, put in computer data. The study showed that people were something like 80% more likely to believe information if the computer listing still had the tractor feed holes attached.
5. Paul Cooper
Poor data quality is not a new problem. Companies just under-estimate the need for a proper focus on data, focusing on the technology implementation instead. At least until a project can’t ‘go-live’ due to data quality problems that is.
Throwing technology at data quality issues doesn’t solve the problem. Companies first need to optimise the data they hold – often within disparate databases and internal systems. This involves cross-checking, converting and combining information into a consistent whole. This is difficult because data is normally peppered with errors such as: executives that are no longer there; companies that have been acquired or merged with another; incorrect address or telephone information; and companies that have closed or gone bankrupt. And that’s the simple stuff, there are also anomalies in the way data is entered, which makes de-duplication difficult – William Gates or Bill Gates for example.
The fact remains, using out-of-date information affects business productivity and profitability. Investment in content optimisation processes now will reap rewards in improved operations and decision making later on.