Information overload

Businesses are drowning in data - but how much good is it doing them? Stewart Baines finds out how to retrieve and understand essential corporate information.

IT professionals are regularly asked to solve the 'information overload' problem but the term may be a misnomer. Almost three in five managers of the 1,000 surveyed by Accenture said they miss information valuable to their jobs every day. On average, respondents to the survey of UK and US professionals spend up to two hours per day scrabbling to find data. If we're so replete with information, why can't we find what we need?

As information volumes increase, so do two other things: the number of data sources and the places to put it. Greg Todd, senior executive for Accenture Information Management Services, says: "It's a consistent theme - there's so much information being thrown up by different technology platforms."

RSS feeds, email, search engines and corporate intranets are competing with internal reporting systems that may not always have access to all of the data streams that employees need. An increasing choice of potential storage locations (and therefore places for people to look for it) make the mess complete.

Employees also store information on their own hard drives, which makes it difficult or impossible for others to find. IT departments compound the problem by duplicating data in different systems or storing too much of it as they frantically try to ward off future compliance problems.

Richard Kellett, head of technology strategy for SAS UK, says: "It's a bit of a comfort blanket. If you keep everything then you can answer any question from the business, theoretically. But what you end up with is something that costs a lot and delivers little."

Before tackling the problem, companies need to understand whether they're dealing with structured or unstructured data. Data from a company's logistics, finance or marketing activities can be pulled into a database that can then be fed to executives using a reporting system or dashboard. A collection of dials and gauges providing an at-a-glance analysis of business performance can be a crucial way to keep everyone in the business informed.

For a dashboard to work, though, a reporting system has to integrate data from separate systems, which must themselves be sophisticated enough to export it. The alternative is to move to a single platform with built-in reporting capabilities.

Netsuite, an Oracle spin-off that provides a hosted business management for SMEs, is carving out a niche for itself here. The company provides online CRM, shipping, inventory management and financial services, among others.

The idea is that by putting everything into the same underlying databases, managers can get a joined-up view of the business. This means no rushing around to see if you have the inventory to hand when someone calls up with a large order, for example.

Mini Peiris, VP of product management at Netsuite, says: "There's a benefit to see all of your key metrics for the business in one place."

But moving from piecemeal back-office systems to a unified platform carries its own investment overhead - and once you've moved, you have to persuade the employees to use the new system. Gavin Whatrup, group IT director at Creston, a collection of PR, advertising and direct marketing businesses, knows getting grassroots buy-in for new working practices can be a challenge.

He says: "We suffer as an industry with unstructured data, particularly email. People use it as primary, sometimes their only, data store. It's completely wrong.

"But in marketing there isn't much top down control. People work independently and use their own processes." He takes a hands-off approach, leaving employees to manage their own information flow on the basis that as successful businesses they must be doing it right.

Surely there are other ways to handle unstructured data, without negotiating tensions between centralised and individual ways of working? Mapping desktops to network drives and using enterprise search appliances is one way to mitigate the problem.

Another is to bring order to unstructured information in automated ways. Adrian Butcher, strategic ECM consultant at document management vendor Open Text, says: "An example would be extracting an order number or customer name from an email and using that to directly access information in a CRM or ERP system."

That still leaves companies with the challenge of collecting the most valuable asset of all: the tacit information that exists in people's heads. How do you get 15 years of unspoken rules and experiences out of the senior support engineer's head and into a digestible format for other people? Depending on the size of the company, wikis, blogs and corporate social networking software could be a way forward.

For now, though, many companies probably have enough of a challenge managing what they already have. And while they continue to grapple with that, more of it keeps piling up.