Customer Data Intergration Techniques

Techniques for managing complexity

Attributes and their values can become extremely complex and dynamic due to the many changes individuals go through. Multiply all these fields by the millions of records a business or organization may have in its data sources, then factor in how quickly and how often this information changes. The Data Warehousing Institute (TDWI) says: “The problem with data is that its quality quickly degenerates over time. Experts say 2% of records in a customer file become obsolete in one month because customers die, divorce, marry and move.”^[1]
To put this statistic into perspective, assume that a company or charity has 500,000 customers, donors or prospects in its databases. Cumulatively, if 2% of these records become obsolete in one month, 10,000 records go stale per month; or 120,000 records every year. Within two years about half of all the records may become obsolete if left unchecked.
Peppers and Rogers^[who?] call the problem, "an ocean of data"^{[this quote needs a citation]}. Jill Dyche and Evan Levy, gurus in this field^{[citation needed]}, have boiled the challenges down to five primary categories:

1. completeness – organizations lack all the data required to make sound business or organizational decisions
2. latency – it takes too long to make the data valuable: by the time of use, too much has become obsolete or outdated (slowed by operational systems or extraction methods)
3. accuracy
4. management – data integration, governance, stewardship, operations and distribution all combine to make-or-break data-value
5. ownership – the more disparate the owners of the data-source owners, the more silos of data exist, and the more difficult it becomes to solve problems