Guide to Disparate Data

A recent research report shows that less than half (49%) of business leaders report they can reliably generate data insights. That’s a significant miss of a growth opportunity.

One of the issues many organizations struggle with is disparate data. Even when the underlying data is correct, disparate data can prevent you from getting a unified view of your company’s operations or customers, and prevent you from getting the right data analytics or AI insights.

Disparate data is data that exists in various systems, databases, or locations that don’t connect or communicate with each other. For example, transactional data may be part of an ERP system while customer data resides in a Customer Data Platform (CDP). Another example is your vendors’ data resides in their systems while your company’s data is contained within your company’s “walls.” Data scattered across systems that are incompatible, often have different structures, schemas, and quality standards, all of which make unifying and analyzing the data challenging.

Data fragmentation can create challenges for organizations that want to get a unified view of their operations, customers, patients, and stakeholders. Here’s a look at the business functions and capabilities that can be impacted by disparate data.

• Time and effort: Have you ever measured the amount of time your organization spends trying to find the right data or cleaning and reconciling duplicate data sitting in siloed systems? This time could be spent in more productive and innovative ways, improving operations and streamlining processes.
• Cost: Data spread in disparate systems costs more to maintain. You are incurring ongoing maintenance costs on top of the initial capital costs.
• Errors: Data reconciliation is often manual. This is an error-prone process that can ultimately reduce your confidence in your data.

• Quality: With data spread in various systems, it’s often inevitable that it will be duplicated. When this happens, you won’t know which system’s data to trust. Your decisions may be based on inaccurate or old data.
• Speed: De-duping and cleaning duplicate data are time-consuming and slow down your strategic decisions.
• Missed market opportunities: Agile organizations can make data-driven decisions and move with speed. If you can’t match them in agility and speed, you may be missing out on business growth.

• Training and errors: Research has shown that 56% of technical leaders have wasted resources training AI on bad data. AI effectiveness depends on large volumes of clean, consistently formatted, integrated data in order to make reliable predictions or take the right actions. Inconsistent data formats lead to training errors and biased predictions, while duplicates and contradictions create noise that degrades AI model performance.
• AI insights and agentic AI actions: When data is scattered across incompatible systems with different structures, schemas, and standards, AI models won’t access the complete picture needed to generate comprehensive insights. And agentic AI action won’t be as expected.
• Time and cost: The time-consuming manual work needed to harmonize disparate data can significantly delay your AI implementation and increase costs.

• Customer needs and behaviors: When your customer data resides in disparate data, you will have a difficult time generating a 360-degree view of your customers’ needs. For example, sales history in the ERP platform isn’t merged with service data from the customer’s service logs or with their web activity. Effectively, the picture you have of this customer is limited to the single system you are looking at. Your organization isn’t able to discern the customer’s preferences and behaviors and can’t tailor products or services to meet their needs.
• Customer experience: Customers today expect their favorite brands to understand their preferences and to offer them personalized experiences. For example, a customer who calls a car dealer expects the dealer to have the records of their purchase and service history. But fragmented customer data spread across systems will give the dealer a limited view of past interactions, and may disappoint the customer.

Disparate data can create integration complexities, increase your technical debt, and create data consistency issues.

Disparate systems are often legacy systems, built decades ago with outdated technologies and proprietary formats. These systems weren’t built to communicate with each other and often lack APIs for data exchanges. They may also store data in obsolete databases or flat files, and often use proprietary coding languages that require technical experts.

Maintaining these siloed systems isn’t just technically complex – it is also expensive. It is difficult to find IT professionals who are familiar with the old programming languages, such as COBOL, and they are expensive to hire. You also incur the ongoing licensing and support costs from the software vendors.

Different systems storing the same data may bring consistency issues. For example, one system might define "active customer" as anyone who purchased in the last 90 days, while another uses a 365-day threshold. Conflicting definitions and validation rules create inconsistencies when data is combined, and confusion about which version of the truth is correct.

If the disparate systems that store your data are legacy systems, they will most likely have scalability issues. These systems can’t easily handle increasingly high data volumes or the processing demands of today without performance issues or complete failures. Their rigid architecture won’t allow you to add more servers, for example, forcing you to either accept the slow performance or invest in expensive hardware upgrades.

Maintaining consistent data governance policies across multiple systems is complex. Each system may have different security controls, data retention rules, and access permissions you’ll have to synchronize manually if you want to implement organization-wide governance policies.

Data redundancy also increases security risks. Multiple systems create more potential attack surfaces and increase the likelihood of breaches because of potentially outdated security protocols.

Once your organization decides it is time to unify disparate data, you have several technology options to consider. What you choose will depend on your future desired architecture, costs, length of implementation, and resource availability.

Integration platforms can connect to disparate data sources, ingest data, transform it into a consistent format, and deliver it to target destinations such as business applications or advertising platforms. Some integration platforms come with prebuilt connectors, allowing you to create unified views of your data without custom APIs or code.

Zero copy is a technique that allows you to access data in its original location rather than moving or duplicating it. The key benefits of this approach are that it eliminates the cost of moving data and can dramatically improve efficiency.

Extract, Transform, Load (ETL) and Extract, Load, Transform (ELT) processes take data from source systems and deliver it to a destination system. Their difference lies in when the preparation and validation step happens. In the ETL process, it happens before loading, while ELT de-dupes data once it’s loaded in the target system.

Master data management (MDM) creates and maintains a single source of truth for data entities such as customers, products, and employees. MDM eliminates duplicates and resolves data conflicts, giving everyone in your organization access to the latest, cleanest data.

Cloud solutions can reduce disparate data issues, but not eliminate them entirely. Simply moving disparate data to the cloud won’t automatically resolve data format differences or give you a unified data architecture – this is where your data strategy comes into play.

Here’s how the cloud helps:

• Cloud platforms can consolidate data from multiple sources into data lakes, warehouses, or data platforms. Modern data cloud platforms come with APIs or connectors that facilitate data sharing.
• Cloud platforms can scale as your data volume grows without the limitations of disparate legacy systems.
• Many cloud platforms come with security and compliance layers, which means you won’t have to source these solutions separately.

Before you start eliminating disparate data systems, take stock of the resources you have available and their skillsets. Below’s a short list of the common skills you’ll need.

• Data strategy and IT architecture: Designing your to-be state requires big-picture data architecture skills to start with. Data architect(s) can design your future state, identify gaps in your technologies, and make infrastructure recommendations in collaboration with IT leadership.
• Technical development: Depending on which integration platform(s) and technologies you choose, you may need ELT or ETL development skills, MDM skills, cloud platform knowledge, and API skills.
• Project management: Reducing siloed data systems can be complex. Your project manager(s) will need to communicate with all your stakeholders, keep the project on track, and manage expectations.
• Change management: Begin change management efforts early, educating stakeholders and employees about what to expect and how data integration will change their workflows. Consider extending training well past the implementation until you hit your targets for organizational adoption.

Enterprise data holds immense potential, but only when it’s unified.

Data 360 connects directly to platforms such as Snowflake and Databricks to unify your disparate data and ground Agentforce. Learn more about Data 360, the world’s most trusted data platform.