Which file format is used for raw data stored in OneLake as part of the data pipeline?

Which file format is used for raw data stored in OneLake as part of the data pipeline?

• A. CSV
• B. JSON
• C. Parquet
• D. TXT

Answer: (C)

Parquet is the format designed for efficient analytics in data pipelines. It stores data by columns, which lets analytic queries read only the necessary data and perform fast scans on large datasets. It also supports complex, nested data and includes metadata and a schema, enabling schema evolution and reliable interpretation as data flows through the pipeline. Parquet’s built-in compression reduces storage and speeds up input/output, which is crucial for big data workloads typical in OneLake pipelines. In contrast, CSV and TXT are plain text formats that are row-oriented and lack intrinsic schemas, making them bulky and slower to process at scale. JSON is flexible for semi-structured data but is text-based and typically more verbose, leading to higher parsing costs and less efficient columnar analytics. For raw data in a data pipeline where performance, scalability, and downstream processing matter, Parquet is the best fit, especially within OneLake in Fabric.

Parquet is the format designed for efficient analytics in data pipelines. It stores data by columns, which lets analytic queries read only the necessary data and perform fast scans on large datasets. It also supports complex, nested data and includes metadata and a schema, enabling schema evolution and reliable interpretation as data flows through the pipeline. Parquet’s built-in compression reduces storage and speeds up input/output, which is crucial for big data workloads typical in OneLake pipelines.

In contrast, CSV and TXT are plain text formats that are row-oriented and lack intrinsic schemas, making them bulky and slower to process at scale. JSON is flexible for semi-structured data but is text-based and typically more verbose, leading to higher parsing costs and less efficient columnar analytics. For raw data in a data pipeline where performance, scalability, and downstream processing matter, Parquet is the best fit, especially within OneLake in Fabric.