What is FastBCP?
FastBCP is a high-performance command-line utility developed by arpe.io that exports data from relational databases to local file systems and cloud object storage platforms. Available for Windows, Linux, and as a hardened Docker image, the tool installs in under a minute using a single executable that bundles every required database driver without needing separate MSI or RPM installers. It supports over 10 main certified database sources and more than 80 compatible database engines, outputting data into analytics and interchange file formats at speeds up to 33 times faster than native database export tools.
Designed for enterprise environments, the software uses parallel data transfer and streaming techniques to maximize network bandwidth while maintaining a low memory footprint, even when handling datasets spanning hundreds of gigabytes. It ensures high security through CA-certified digital signatures, obfuscated passwords in logs, and a built-in parser that blocks injection attacks. Additionally, it supports comprehensive compliance and error handling by providing full audit trails and enterprise-grade logging to consoles, JSON files, and external database sinks.
Features
- Ultra-Fast Parallel Export: Utilizes parallelism and data distribution techniques with just 2 to 3 parameters to export data up to 33x faster than native tools.
- Single Executable Deployment: Installs in under a minute with a single executable that bundles all required database drivers, requiring no MSI or RPM packages.
- ADBC Wire Protocol Drivers: Provides direct wire protocol connections to Oracle, PostgreSQL, SQL Server, and Netezza for ultra-fast Parquet exports without extra dependencies.
- Multi-Format Output Support: Exports database records into modern analytics and interchange formats including CSV, Parquet, JSON, XLSX, and BSON.
- Broad Database Compatibility: Connects natively to 10+ main databases including PostgreSQL, Oracle, SQL Server, MySQL, and Teradata, plus 80+ compatible database engines.
- Cloud and Local Storage Targets: Writes exported files directly to local filesystems, AWS S3, S3-compatible storage, Azure Blob Storage, Azure Data Lake Storage Gen2, GCS, and OneLake.
- Low Memory Streaming: Uses streaming data transfer to keep memory usage low and maintain stable performance across massive datasets of hundreds of gigabytes.
- Enterprise Security and Logging: Features CA-certified digital signatures, password obfuscation in logs, injection attack prevention, and audit logging to console, JSON, PostgreSQL, and MSSQL sinks.
Use Cases
- Exporting large relational database tables to local CSV, Parquet, JSON, XLSX, or BSON files.
- Migrating on-premises or cloud database workloads directly to cloud object storage platforms such as AWS S3, Azure Data Lake, GCS, and Microsoft Fabric OneLake.
- Accelerating enterprise data pipelines and reducing daily batch execution times.
- Extracting and comparing large volumes of data in analytics platforms like Databricks during database migration projects.
- Replacing slow custom export scripts and traditional ETL tools with automated command-line data extraction.
How It Works
Select a Database Source
Connect to your database using native or ADBC drivers across supported platforms such as PostgreSQL, Oracle, SQL Server, Netezza, MySQL, MariaDB, ClickHouse, SAP HANA, Teradata, or ODBC.
Choose an Output Format
Configure the FastBCP engine via command-line parameters to process and format your query or table data into CSV, Parquet, JSON, XLSX, or BSON.
Export to Target Destination
Execute the parallel export to stream the output files directly to your local filesystem or cloud object storage such as AWS S3, Azure Blob Storage, ADLS Gen2, GCS, or OneLake.
FAQs
-
How is FastBCP licensed and priced?
FastBCP is licensed per data-owning company with flat pricing rather than pricing based on CPU or server count. -
Which operating systems and environments does FastBCP support?
FastBCP runs on Windows, Linux, and is also available as a hardened Docker image. -
How does FastBCP outperform traditional native export tools?
It achieves up to 32x peak and 16x average faster export speeds by using parallelism and specialized techniques to evenly distribute data across multiple channels. -
Does FastBCP require installing external database drivers?
No, it installs in under a minute using a single executable that bundles every required database driver without needing MSI or RPM installations.
