Replace Fivetran + Snowflake: MySQL to S3 Parquet Analytics, Self-Hosted

说明与用途

团队通过将 MySQL 表导出为 S3 中的 Parquet 并使用 SynapCores 查询，替代 Fivetran 和 Snowflake，降低数据移动和计算成本。

问题

Teams often pay for two separate vendor services—an ETL tool and a cloud warehouse—to move and query operational data, incurring growing costs for data movement and compute time.

结果

Exporting MySQL tables to Parquet in S3 and querying them with SynapCores reduces costs by eliminating ETL row charges and warehouse compute minutes, while providing millisecond query performance comparable to DuckDB.

工作流

• Configure SynapCores connection to MySQL
• Set destination to S3-compatible bucket
• Create export job with cron schedule (snapshot or incremental)
• Run job and verify partitions
• Register Parquet as external SQL table
• Query data with plain SQL

证据

The ETL tool charges for moving rows. Fivetran, for example, bills on monthly active rows — every row inserted, updated, or deleted in a synced table counts.

来源

SynapCores v2.0 added a data-lake export built into the database: A connection to your MySQL holds the credentials once… A job exports tables to Parquet on a cron schedule you set… Each run reports rows, bytes, peak memory, and exit code… The Parquet is then registered as an external SQL table and queried in place.

来源

We benchmarked SynapCores against DuckDB on the same Parquet objects in the same bucket: Scan 189,361 rows of a lake table – 10.1 ms vs 28.6 ms; Count a join across two lake tables – 5.5 ms vs 10.9 ms.

来源

With SynapCores CE on your own infrastructure: Software: $0 — the Community Edition is free. Server: one VM sized for your query load… S3 storage: about $23 per TB-month on S3 Standard.

来源