Optimize Data 360 Ingestion for Cost Efficiency

Data 360 is a powerful platform for ingesting, processing, unifying, and activating customer data from various sources. However, architecture choices can significantly impact credit consumption. Organizations can incur unnecessary costs through inefficient ingestion, processing, and consumption strategies.

In Data 360, many usage types calculate credit consumption based on records processed or accessed, rather than just the number of rows returned. This means inefficient pipelines that scan millions of rows but return only a handful of rows can consume credits without delivering proportionate business value.

By aligning ingestion choices with row-processing mechanics, architects can significantly lower credit consumption while maintaining system performance. This post explores high-impact strategies for reducing rows processed across the ingestion pipeline to optimize data volume and manage costs.

Choose the right ingestion path

The choice of ingestion method influences the number of rows processed by Data 360, which serves as the primary driver of  credit consumption. Any data filtered or kept outside the platform incurs zero processing cost. Standard ingestion routes duplicate and register every row within the data lakehouse, whereas zero-copy architectures allow querying without replicating it. 

Use internal CRM data pipelines to save costs

Since August 2025, internal CRM data pipelines consume zero credits. This applies to core CRM data from Sales Cloud, Service Cloud, Marketing Cloud Engagement, Marketing Cloud Personalization, and B2C Commerce Cloud via the Salesforce CRM connector. Note that specialized Industry Cloud connectors are not all zero-cost and may still incur standard ingestion credit usage.

Consider zero-copy options

Before choosing how to ingest external data, ask yourself whether you need to ingest it at all. Zero-copy lets you query data where it already resides, without moving it into Data 360.

Scenario	Zero-Copy	Standard Data Ingestion
Query frequency	For infrequent or ad-hoc queries, paying per query is cheaper than continuous ingestion	For frequent queries, ingestion and query cost is lower than repeated zero-copy query cost
Data retention / localization	Data must remain in the external data lake due to strict retention or localization requirements	Multiple data copies are acceptable; copying to Data 360 is allowed
Credit cost	~70 credits per 1 million rows processed (but incurs compute on both Data 360 and external source)	500–2,000 credits per 1 million rows depending on the source

If your data is queried frequently and transformations are applied, standard ingestion is often more cost-effective. If queries are infrequent or ad-hoc, or if the data cannot leave its current location, zero-copy is the way to go.

Select standard ingestion replication modes for batch versus streaming

If you have chosen data ingestion over a zero-copy approach, the next step is to determine your data replication strategy.

From a cost perspective, streaming ingestion incurs higher credit consumption than batch processing. Streaming pipelines can require continuous micro-batching and constant index updates to maintain live state reconciliation, continuously counting toward rows processed. In contrast, batch ingestion processes records in scheduled chunks, which can reduce overhead and continuous compute usage.

Pro tip: If a use case does require near-real-time data, you can sometimes adapt batch-oriented sources for streaming by enabling database change events. For example, when working with AWS, you can configure the AWS Database Migration Service (DMS) with Change Data Capture (CDC). By reading transaction logs to capture real-time inserts, updates, and deletes, DMS can continuously push data to streaming targets like Amazon Kinesis or Apache Kafka for immediate ingestion.

While streaming ingestion offers real-time capabilities, it is more expensive than batch processing. Architects should evaluate whether datasets truly require real-time availability before selecting a streaming approach.

In many analytics and reporting scenarios, batch processing delivers the necessary data freshness while offering a more cost-effective approach.

Evaluate query federation versus file federation for zero-copy

When employing zero-copy strategies, the underlying federation model can influence credit usage. Query federation executes compute on both Data 360 and the external database layer, leading to dual-compute billing and driver bottlenecks on large datasets. File federation (Apache Iceberg) connects directly at the storage level, which can eliminate secondary compute overhead and lower the total credits required to query records.

Factor	Query Federation (JDBC)	File Federation (Iceberg)
Best for	Datasets under 100M rows	Datasets over 1B rows
Compute cost	High (dual compute: paying both Data 360 and external source)	Low (Data 360 compute only: accesses Iceberg files directly via storage)
Speed	Slower (driver and query layer bottlenecks)	Faster (bypasses driver, direct file access)
Change capture and automation	CDC/Flow automation available only via acceleration	CDC/Flow automation natively available

JDBC (Java Database Connectivity) is a standard Java API that enables Java applications to interact with relational databases.

The rule of thumb: If your data is already in a supported cloud storage format like Iceberg and you do not want to copy the data over, you can use zero-copy/federation ingestion. In zero-copy, file federation can be a better choice, offering faster performance while spending fewer credits on insights.

Optimizing zero-copy costs: For workloads using zero-copy and live queries, enable acceleration on frequently accessed data. With zero-copy acceleration, it can minimize repeatedly paying for external compute. 

Optimizing Data 360 costs does not require sacrificing capability. By making intentional choices at the ingestion stage, you can reduce credit consumption.