La corruption silencieuse : pourquoi vérifier la taille d'un fichier ne suffit pas

A file arrives. It weighs 412 MB. The expected file weighs 412 MB. The size check passes. The workflow continues. The load triggers. Data integrates into your Datahub or Data Cloud.

And a few hours later, your marketing teams report that contacts have disappeared. Segments are wrong. Campaigns are sent on truncated databases.

The corruption was there from the start. Silent.

What actually happened

During an SFTP transfer to Azure Blob Storage, the connection dropped mid-transfer. The file was created on the destination side. Its size matched what had been transferred up to the interruption, not what should have been transferred.

The source file was intact. The destination file was truncated. And both weighed exactly the same number of bytes according to the verification system, because that system wasn't comparing source size with destination size. It was only checking that the destination size was greater than zero.

Why size checking alone is not enough

There are several forms of silent corruption in a file pipeline:

• Truncation: the transfer drops, the file is created with what was received. The size is less than expected, but if you don't know the expected size, you won't see it.
• Block corruption: the file arrives at the correct size but certain blocks are altered during transit. Size is correct. Content is not.
• Partial encoding: an encoding error during compression or conversion creates a readable but incomplete file across certain line ranges.
• Empty file validated: a 0-byte file with the right name passes an existence check. Without a minimum threshold, it enters the pipeline.

What Adobe Campaign Datahub does natively: almost nothing

Adobe Campaign Classic and Campaign v8 offer file transfer activities with existence verification options. But file integrity verification, MD5 checksum, source/destination size comparison, row counting, is not native. It must be implemented in JavaScript activities or Shell scripts called from the workflow.

On a Datahub connected to Azure Blob, integrity verification responsibility falls entirely on the team designing the import workflow. The platform loads what it receives, without validating what it should have received.

What Salesforce Data Cloud does natively: not better

Salesforce Data Cloud offers ingestion connectors (Ingestion API, Cloud Storage Connector) with validation jobs. These jobs check format, schema, field types. They do not check whether the file received matches the file sent.

A correctly-formatted truncated file, with the right columns and types, integrates into Data Cloud without error. The validation job turns green. The loaded data is incomplete.

The robust workflow: three layers of protection

What this changes on Adobe Campaign and Salesforce Data Cloud

The principle to remember

A file pipeline into a CRM is reliable only when three conditions are met: received size is verified against a reference, content integrity is validated by checksum or row count, and the pipeline knows what to do on failure without manual intervention.

Without these three layers, you have a pipeline that works until the day it doesn't, and you only find out by looking at your campaigns.

This topic is directly linked to the integration architecture I described in my article on Salesforce and Snowflake synchronisation: the robustness of a data pipeline is not measured when everything works, but when a connection drops at the wrong moment.