Summary
- For no-code scheduled ingestion with reverse ETL in the same product, pick Skyvia.
- For managed connectors with the least upkeep at enterprise scale, pick Fivetran.
- For real-time CDC from databases into several destinations, pick Estuary.
- For event streaming in milliseconds, pick Apache Kafka.
Everybody wants one source of truth. Hardly anybody enjoys the step before it: pulling data out of twenty different systems so someone can finally query it.
That’s the work data ingestion tools do. They take SaaS apps, databases, files and event streams and drop all of it into a warehouse or a lake. Here’s the snag. “Ingestion” is a label stuck on quite different machines (some load on a timer, some read a database’s change log, some push events the second they happen). Choose the wrong type and you’ll either pay for speed nobody uses or sit around for hours waiting on data you needed in minutes.
Full disclosure: we’re the team behind Skyvia, which is one of the ten tools below, so yes, we’re biased. We’d rather show our work than pretend otherwise. Earlier this year we pushed the same dataset through five of these tools and noted what broke. This update brings in five more that turn up on almost every list in this category, and each was checked against its own pricing pages and docs.
Every tool gets the same fields. You’ll see how it ingests (batch, CDC or streaming), the fastest schedule on each plan, what the bill counts, the entry price with its billing basis, and two real limits.
How we chose and checked these tools
Ingestion mode comes first, since it settles most of the other questions. Does the tool load on a schedule, read a database’s change log (change data capture, or CDC), or stream events as they happen? How often can it run on the plan you’d actually buy? People stretch “real-time” to mean anything quicker than overnight, so each entry lists the real minutes by plan.
Each entry covers the same ground:
- Best for: the buyer and the job.
- Ingestion mode: batch, CDC or streaming, and the fastest schedule by plan.
- Pricing: the unit you’re billed on, the entry price and how it’s billed, plus a free plan or trial if there is one.
- Deployment: cloud, self-hosted, or a mix.
We tested five of them ourselves: Skyvia, Fivetran, Airbyte, Qlik Talend Cloud and Apache Kafka. The setup was a mock e-commerce dataset. Think 10K rows of raw CSVs mixed with nested JSON, pulled from a PostgreSQL database and a REST API into Snowflake. Where it made sense we pushed harder: we built a connector from scratch, ran local infrastructure, and fed tools bad data on purpose. Each entry says what happened. We judged the other five (Hevo Data, Estuary, Apache NiFi, AWS Glue and Azure Data Factory) from their documentation and pricing pages, with no trial run.
Prices, plans and limits all come from the vendors’ own pages, as seen from the US. They’re accurate as of November 4, 2026.
One more thing: we make Skyvia, and every tool here, Skyvia included, was held to the same fields, limitations and all.
Who didn’t make the cut? Matillion, Stitch and Integrate.io are ELT and ETL tools at heart, so they sit in our ELT tools list. Amazon Kinesis (Data Streams and Data Firehose) and Azure Event Hubs are cloud streaming services. In this article Kafka speaks for event streaming, and AWS Glue and Azure Data Factory speak for the cloud-native route.
Debezium is a CDC engine that runs on Kafka Connect, and Confluent Cloud is managed Kafka, so both live inside the Kafka entry. dlt and Meltano are Python and config-driven libraries that engineers run on their own; Airbyte does that job here. Snowpipe and Databricks Auto Loader only load into a single platform. Informatica is an enterprise data-management suite sold by contract, and Qlik Talend Cloud fills that slot on this list.
Which data ingestion tools load on a schedule?
Streaming isn’t what most reporting and BI work needs. A refresh every hour or every 15 minutes is plenty, and scheduled batch ingestion costs less to run and is easier to repair. The four tools below load full tables first, then only new and changed records on a schedule. All four run as a managed service (Airbyte can also be self-hosted).
Best for no-code ingestion plus reverse ETL in one tool: Skyvia
Skyvia is a no-code cloud data platform, and an ingestion pipeline is usually the first thing people build in it. Replication creates the warehouse tables for you. After one full load, it only picks up new and changed records, on whatever schedule you set. The same connection then feeds Import (reverse ETL back into apps included), Synchronization and Data Flow, so sending data back out doesn’t mean buying a second tool. Analysts, RevOps teams and small data teams get the most from it, when they want SaaS and database data in a warehouse and no code.
- Best for: teams without data engineers that need 200+ sources in a warehouse and want reverse ETL from the same platform.
- Ingestion mode: scheduled batch with incremental updates. Runs go every minute on Professional and Enterprise, every hour on Standard, and once a day on Free and Basic. SQL Server gets native change capture (Change Tracking); every other source relies on timestamp-based incremental updates.
- Pricing: records written per month, by plan, with unlimited users. For 5M records, Basic costs $79 a month billed annually ($99 billed monthly) with daily runs, and Standard costs $159 billed annually ($199 billed monthly) with hourly runs. Professional starts at $499 monthly ($399 annually) for 10M records and runs every minute. There’s also a Free plan with 10K records a month, and you don’t need a credit card for it.
- Deployment: cloud (Microsoft Azure, western US); an On-premise Agent reaches databases behind a firewall over outbound HTTPS; Enterprise can run on a dedicated virtual machine.
Setup in our test was a single trip through the wizard. We connected PostgreSQL, connected Snowflake, picked the objects and chose a schedule. Skyvia built the tables itself. After that, run history listed row counts and errors for every run, and email notifications warn you when an integration fails.

Outside our test, cybersecurity company TitanHQ automated SugarCRM-to-Snowflake ingestion for Tableau reports with Skyvia. Its engineers also evaluated Fivetran and picked Skyvia to stop maintaining their own Python extraction scripts.
Strengths
- Overage is off by default. Once a plan’s monthly records are used up, new runs pause until the next period instead of billing extra, and failed records aren’t billed.
- You can delete source-deleted records in the target, flag them in an
_is_deletedcolumn or keep them, and history mode holds every version of a record.
Limitations
- Source-side metadata changes aren’t picked up automatically, so a new or changed column means you update the integration yourself.
- There’s no log-based CDC or streaming. A one-minute schedule is as fast as it gets, and that takes Professional.
Want to see the setup? This SQL Server-to-Snowflake tutorial walks through replication and syncing data back to the source.
Best for hands-off connectors at enterprise scale: Fivetran
With Fivetran, ingestion is something you buy as a fully managed service. Once you pick a connector and point it at a warehouse, Fivetran takes over keeping it alive. Database sources get log-based CDC. And since the dbt Labs merger closed in June 2026, the transformation side comes from the same company too. The natural buyer is an enterprise data team that prefers paying for connectors to hiring people who babysit them.
- Best for: enterprise teams moving hundreds of sources into Snowflake, BigQuery or Redshift who want as little upkeep as possible.
- Ingestion mode: scheduled batch, plus log-based CDC through logical replication on PostgreSQL. Standard syncs every 15 minutes at best, Enterprise and Business Critical every minute, and out of the box a connection syncs every 6 hours.
- Pricing: Fivetran charges by Monthly Active Rows (MAR), counting each distinct primary key once per month, per connection and table. Even if a row changes three times, it still counts as one MAR. Standard doesn’t have a published per-MAR rate. Since January 1, 2026, connections with 1 to 1 million MAR also carry a $5 monthly base charge each. The Free plan covers up to 500,000 MAR a month.
- Deployment: Fivetran runs as a managed cloud service. Enterprise and Business Critical customers can also use Hybrid Deployment to run pipelines within their own network.
The PostgreSQL connection in our test read the write-ahead log for incremental updates instead of repeatedly querying live tables, so the source database had less work to do. We added a column upstream and it landed in Snowflake with no redeploy. Fivetran’s own docs add a wrinkle, though: on PostgreSQL, a new column appears in the destination only after a row in that table changes, not when you add the column.

Strengths
- 700+ fully managed connectors on Standard, plus 200+ activation destinations for reverse ETL.
- Per-connection schema settings: allow all new data, allow new columns only, or block all new data.
Limitations
- Since January 1, 2026, deletes count toward paid MAR on pay-as-you-go, on top of the $5 base per connection.
- Standard stops at 15 minutes, so one-minute syncs mean Enterprise or Business Critical.
Best for custom connectors and open-source ingestion: Airbyte
Airbyte comes in two flavors. There’s Airbyte Core, the open-source engine you host yourself for free, and there’s Airbyte Cloud, which Airbyte runs for you. Its catalog has 700+ connectors. If a source is missing, you can describe one in YAML with the Connector Builder instead of coding it from zero. Developer-heavy teams that want to own their pipelines will like it.
- Best for: engineering teams chasing a rare source, or ones that want to self-host and skip the license fee.
- Ingestion mode: scheduled batch and log-based CDC (Postgres, MySQL, SQL Server, MongoDB, Oracle and others), though CDC still runs as scheduled syncs and not as a continuous stream. On Cloud that’s every hour on Standard, every 15 minutes on Plus, and every 5 minutes on Pro and Enterprise Flex.
- Pricing: Standard and Plus bill credits (6 credits per million rows from APIs, 4 credits per GB from databases and files). Standard $20 a month billed monthly with 5 credits included, extra credits $5 each; Plus $189 a month for 40 credits; Pro and Enterprise Flex bill capacity (Data Workers) on a sales quote. Core is free; you pay for your own infrastructure. The Cloud trial runs 14 days or 400 credits, no credit card.
- Deployment: self-hosted Core on Kubernetes, Airbyte Cloud, or Enterprise Flex (Airbyte’s control plane, data planes in your network).
A local install was one command and a short wait. (Under the hood, the installer uses Docker to spin up a local Kubernetes cluster.) So we gave the Connector Builder something harder: an API that wasn’t in the catalog. It timed out. Twice, actually, and the second error said barely more than the first. We had the auth type right and the header name wrong. Five minutes to fix once we saw it, and from then on the stream synced without complaint.

Strengths
- Self-hosting Core is free, and if you outgrow it, pipelines move to Enterprise Flex without being rebuilt.
- A per-connection schema-change policy: propagate field changes, propagate all changes, approve every change yourself, or stop syncs.
Limitations
- Marketplace (community) connectors aren’t maintained by Airbyte, carry no support SLA and may get breaking changes without notice.
- Self-managed Airbyte checks a source for schema changes at most every 24 hours (Cloud every 15 minutes), so a change can take up to a day to show up unless you refresh it by hand.
Best for no-code pipelines with published plan prices: Hevo Data
Hevo Data is a no-code pipeline service that Hevo fully manages. Data goes from databases and SaaS apps into warehouses, there’s reverse ETL (“Activate”) for the trip back, and dbt models can kick off when a pipeline finishes. Alongside the Standard pipelines sits Hevo Edge, a separate pipeline product with an enhanced CDC connector and shorter schedules. Small and mid-size teams who like reading a price straight off the page are its crowd.
- Best for: teams that want managed ingestion with log-based CDC and a self-serve price.
- Ingestion mode: scheduled batch and log-based CDC for databases. Standard pipelines sync every 30 minutes at the fastest (for teams created after release 2.21), Edge pipelines offer presets from 5 minutes to 24 hours, and the Free and Starter plans run hourly.
- Pricing: the unit is the event, and each record inserted, updated or deleted in the destination is one. Starter costs $299 a month billed monthly ($265 billed annually) for 5M events, Professional is $849 monthly ($750 annually) for 20M, and Business Critical is priced on a quote. Free gets you 1M events a month on hourly scheduling, and the 14-day trial doesn’t ask for a credit card.
- Deployment: cloud only, in Hevo-run regions; VPC peering on Business Critical.

Strengths
- Monthly and annual prices are published for every self-serve tier, and there’s a free plan.
- Schema evolution happens automatically under a policy you set per pipeline, and Edge adds type widening.
Limitations
- There’s no self-hosted edition, and RBAC, SSO and VPC peering are Business Critical features.
- Rename a source schema and Hevo won’t map it for you. On Edge, new columns or tables that hold no data stay invisible until you refresh the schema.
Which data ingestion tools capture database changes in real time?
Need a dashboard that shows an order from a minute ago, or one source feeding several destinations? In that case log-based change data capture beats a schedule. The next two read the database’s change log nonstop, and they’re built for different people.
Best for streaming CDC into several destinations: Estuary
Estuary, which went by Estuary Flow until March 2026, keeps capturing changes from databases, SaaS apps and webhooks, then delivers them to warehouses, lakehouses, databases and Kafka-compatible consumers. Captured data lands as collections. So one capture is enough for several destinations, and when you backfill, Estuary replays from that storage rather than going back to the source.
- Best for: anyone who needs a steady stream of database changes, and often in more than one destination at once.
- Ingestion mode: streaming log-based CDC plus batch sources. Database and streaming destinations get changes as they arrive. Warehouses (Snowflake, BigQuery, Databricks, Redshift) work on a timer, though. Their default is every 30 minutes, and only a sync frequency of 0s makes Estuary push as fast as it can.
- Pricing: GB of data moved plus connector instances. Cloud pay-as-you-go: $0.50 per GB plus $100 a month per connector instance for the first six ($50 each after that), billed monthly with no minimum. The free Developer plan covers 10 GB a month and 2 connector instances, no credit card.
- Deployment: multi-tenant SaaS with regions in the US, EU and APAC; if you want a private deployment or your own cloud, that takes an annual Enterprise contract.

Strengths
- Estuary supports transactional processing and multi-table transactions. End-to-end exactly-once delivery applies when the destination supports transactions.
- Backfills and replays of history are part of the deal, and your collections can sit in your own S3, GCS or Azure bucket.
Limitations
- You’re billed on the uncompressed JSON size, counted once at capture and again for each destination. A Parquet or gzip source can therefore bill more GB than it actually holds.
- SSO, a HIPAA BAA and Python transformations need Enterprise.
Best for governed CDC across hybrid and legacy systems: Qlik Talend Cloud
Qlik Talend Cloud is what Talend turned into once Qlik bought it. One platform covers replication with log-based CDC, transformation in the target, and data quality and governance. Talend Cloud also brings Talend Studio, the Java job designer many enterprise teams already know. Large organizations with on-premises systems that aren’t going anywhere, plus rules about where data may run, are the natural buyers.
- Best for: enterprises running hybrid and legacy infrastructure, where governance and data quality matter as much as moving data from A to B.
- Ingestion mode: scheduled full loads on Starter; real-time synchronization with log-based CDC from the Standard edition up. Qlik publishes no latency figure.
- Pricing: a capacity subscription, measured by data volume (Starter, Standard) or by volume, job executions and duration (Premium, Enterprise). No edition has a published price: every one is a sales quote. The 14-day trial needs no credit card.
- Deployment: Qlik Cloud SaaS, with client-managed or hybrid deployment from Standard up.
We ran our test in Talend Studio. Even a modest mapping meant dragging components around a canvas, and point-and-click got us only partway before we had to switch to Java. Can Talend do nearly anything here? Yes, governance and on-premises connectors that most newer tools have never met included. Setting any of it up quickly is another matter.

Strengths
- Log-based CDC and hybrid or client-managed deployment from the Standard edition.
- Premium adds lineage, impact analysis, data quality and stewardship.
Limitations
- No published prices for any edition, so a budget starts with a sales call.
- Studio jobs compile to Java, and Studio’s dynamic schema doesn’t work in every component.
Which data ingestion tools stream events or route data flows?
Both of these are open-source platforms you run on your own. Downloading them costs nothing. The bill shows up as infrastructure, plus the people who know how to run it.
Best for streaming events in real time: Apache Kafka
Apache Kafka is open-source, and it’s built for event streaming. Events sit durably in topics on the brokers. Moving data in and out of Kafka is the job of Kafka Connect, and processing it is what Kafka Streams is for. None of that loads a warehouse by itself. Data comes in through a source connector and leaves through a sink connector. Debezium, an open-source CDC engine, runs on Kafka Connect. If you’d rather not run Kafka at all, Confluent Cloud sells it as a managed service.
- Best for: teams whose events have to arrive in milliseconds for several consumers, with engineers on hand who can run distributed systems.
- Ingestion mode: streaming. Kafka Connect ingests continuously, and brokers deliver messages with latencies “as low as 2ms” by the project’s own figure. Confluent Cloud clusters offer sub-100ms latency.
- Pricing: Apache Kafka is a free download; you pay for servers and operations. Confluent Cloud bills usage: Basic “starting at $0/month” (the first eCKU free, then $0.14 per eCKU-hour, plus $0.05 per GB in or out); Standard from about $385 a month by Confluent’s estimate for a cluster 70% in use; managed connectors cost extra per task-hour and per GB. New Confluent users get a one-time $400 credit for 30 days.
- Deployment: self-managed on servers, VMs, containers or cloud; managed through Confluent Cloud (AWS, Azure, Google Cloud) and other vendors.
Our Kafka ran on Windows in KRaft mode. Java 17 went on first. Then came the Kafka binary, unpacked into a short path (Windows and long file paths don’t get along). Three commands did the real work: generate a cluster ID, format the storage directories with it, and start the broker. After that we made a topic called ingest-events and let Kafka Connect fill it.
Events arrived as they were produced and stayed available for replay. Kafka preserves their order within each partition, and a consumer that falls behind can read them again without asking the producer to resend anything. Kafka’s own docs carry a warning, though: Windows “is not currently a well supported platform”, so production clusters belong on Linux.

Strengths
- Events stay in the topic after they’re read, for as long as retention keeps them, and any number of consumers can re-read them in order.
- Topics replicate across brokers, regions or datacenters; a replication factor of 3 is the common production setting.
Limitations
- The Apache download ships only example file connectors: JDBC, CDC and SaaS connectors are separate plugins to install and run, or paid managed connectors on Confluent Cloud.
- The cluster is yours to run, from brokers on Java 17+ to partitions, upgrades and monitoring. One more change worth knowing: since Kafka 4.0 (March 2025) there’s no ZooKeeper mode, only KRaft.
Best for visual flows that route files and IoT data: Apache NiFi
Apache NiFi is open-source software for dataflows. Building one looks like this: processors go onto a canvas in your browser, and you draw connections between them. Data then moves between systems, and NiFi routes, transforms and delivers it along the way. Because provenance follows each piece of data, you can look up where it came from and what was done to it. That suits teams pulling files, logs, IoT device data and database records from many sources (on-premises, more often than not), with a person watching the flow.
- Best for: on-premises teams that route data from files, devices, APIs and databases and want lineage for each record.
- Ingestion mode: flow-based, batch or streaming per processor. A processor’s run schedule defaults to 0 seconds (run as often as there’s data); CRON scheduling is also there; listener processors (HTTP, TCP, Syslog, Kafka) take data as it arrives. Only MySQL gets log-based CDC out of the box; for other databases, QueryDatabaseTable polls for changes.
- Pricing: nothing for the software, which is free and open-source, so the servers are the whole bill. Cloudera Data Flow is a managed service built on NiFi, although Cloudera doesn’t publish a price for it.
- Deployment: self-managed on Linux, Unix, Windows or macOS with Java 21. It scales from one laptop to a cluster that ZooKeeper or Kubernetes coordinates.

Strengths
- Provenance gives you the full lineage of each piece of data, right in the UI.
- Flow control is part of the product: back pressure, guaranteed delivery and prioritization. Editing a running flow works too, and it doesn’t take a redeploy.
Limitations
- Change capture beyond MySQL means polling a max-value column or adding third-party components.
- Running, securing, upgrading and clustering are all on you. Running NiFi 1.x? Support for it ended in December 2024, which means an upgrade to NiFi 2 is due.
Which data ingestion tools come with your cloud?
Is your data already sitting in AWS or Azure? Then the cloud’s own service tends to be the shortest path. It runs in your account and lands on the same invoice, and it talks to that cloud’s storage and warehouses natively. The catch: the code or configuration you write won’t work anywhere else.
Best for serverless ingestion into an AWS lake: AWS Glue
Amazon’s serverless data integration service is AWS Glue. It bundles Spark-based ETL jobs for batch or streaming, crawlers that infer schemas into the Glue Data Catalog, and zero-ETL integrations that replicate SaaS apps and databases into Amazon Redshift or the SageMaker Lakehouse. Jobs come from Glue Studio’s visual editor, which writes the code for you, or straight from PySpark or Scala. Teams whose lake already lives in S3 are the target.
- Best for: teams on AWS who land data in S3 and then catalog it for Athena, Redshift or EMR.
- Ingestion mode: batch jobs, started on a schedule, on demand or by an event. There are also streaming jobs, which never stop and write in 100-second windows by default (the lowest is about 1 to 2 seconds). Glue 6.0 adds a sub-second real-time mode for Scala jobs reading Kafka.
- Pricing: DPU-hours (1 DPU = 4 vCPU and 16 GB of memory), billed by the second with a 1-minute minimum on current versions. Pay as you go in US East: $0.44 per DPU-hour for Spark jobs on Glue 2.0 to 5.1, $0.308 per DPU-hour on Glue 6.0; zero-ETL from apps $1.50 per GB ingested plus target compute. The Data Catalog’s first million objects and requests a month are free.
- Deployment: fully managed AWS service in your account and region; reaches on-premises and VPC data stores through Glue connections.

Strengths
- Crawlers infer schemas into the Data Catalog, which Athena, Redshift and EMR query directly.
- Billed by the second, and Glue 6.0 cuts the Spark rate from $0.44 to $0.308 per DPU-hour in US East (pay as you go).
Limitations
- Streaming jobs bill the whole time they run, at 2 DPUs at minimum, and each crawler run bills at least 10 minutes.
- AWS-centric: zero-ETL from self-managed databases replicates only into Amazon Redshift. Glue 6.0’s sub-second streaming mode supports only Scala and Kafka, and partitions without enough task slots are silently dropped.
Best for Azure pipelines with on-premises sources: Azure Data Factory
Microsoft’s managed service for hybrid ETL and ELT is Azure Data Factory. A pipeline copies data from cloud and on-premises stores, and then it orchestrates the transformations, in mapping data flows on Spark or in external compute such as Databricks and stored procedures. It pays off most for teams already on Azure, and even more if they have SQL Server databases or SSIS packages to move.
- Best for: teams on Azure that move data from on-premises and cloud stores into Azure SQL, Synapse or a lake.
- Ingestion mode: scheduled and event-based batch, with schedule triggers as often as every minute, plus tumbling-window and storage-event triggers. Native CDC in mapping data flows for SQL Server, Azure SQL, SAP and others; the top-level Change Data Capture resource, which runs continuously in near real time, is still in public preview.
- Pricing: pay as you go, with no plans and no minimum. On the Azure integration runtime, the pay-as-you-go East US list prices are $1 per 1,000 activity runs for orchestration and $0.25 per DIU-hour for data movement. Mapping data flows bill per vCore-hour. New to Azure? The free account comes with a one-time $200 credit for 30 days.
- Deployment: Azure PaaS. The self-hosted integration runtime reaches on-premises and private-network sources, and the Azure-SSIS runtime runs the SSIS packages you already have.

Strengths
- Hybrid reach through the self-hosted integration runtime, with lower data movement rates on it.
- For CI/CD there’s Azure DevOps or GitHub, and for monitoring there’s Azure Monitor.
Limitations
- Expect a lot of meters on the bill (activity runs, DIU-hours, runtime hours, vCore-hours, operations, egress), and how you design the pipeline sets the total. Mapping data flows also need an 8-vCore Spark cluster at minimum.
- Microsoft now points new users to Fabric Data Factory, and the 2026 ADF updates are about migrating there.
Which data ingestion tool fits which job?
- No data engineers, SaaS and database sources, and reverse ETL later: Skyvia.
- Hundreds of sources into a warehouse with the least upkeep: Fivetran.
- A source nobody supports, or a self-hosted stack without a license fee: Airbyte.
- Managed CDC from databases with a price on the page: Hevo Data.
- Database changes streamed into more than one destination: Estuary, though warehouses get them every 30 minutes by default unless you set it faster.
- Hybrid and legacy systems under governance rules: Qlik Talend Cloud.
- Events in milliseconds, replayable by many consumers: Apache Kafka.
- Files, IoT devices and on-premises systems routed through one visual flow: Apache NiFi.
- Data already in S3, cataloged for Athena or Redshift: AWS Glue.
- Azure, with SQL Server or SSIS to bring along: Azure Data Factory.
Data ingestion tools compared: ingestion mode and price
| Tool | Ingestion mode | Fastest schedule by plan | Bill counts | Entry price |
|---|---|---|---|---|
| Skyvia | Scheduled batch, incremental | Daily (Free, Basic), hourly (Standard), every minute (Professional) | Records written per month | Free (10K records); Basic 5M records $79/mo annually, $99 monthly |
| Fivetran | Scheduled batch, log-based CDC | 15 min (Standard), 1 min (Enterprise) | Monthly Active Rows | Free (500K MAR); Standard usage-based, $5 base per connection |
| Airbyte | Scheduled batch, log-based CDC | 1 hour (Standard), 15 min (Plus), 5 min (Pro) | Credits (rows or GB); capacity on Pro | Core free; Standard $20/mo with 5 credits |
| Hevo Data | Scheduled batch, log-based CDC | 1 hour (Starter), 30 min Standard pipelines, 5 min Edge | Events | Starter $299/mo monthly, $265 annually; 14-day trial |
| Estuary | Streaming CDC, batch | Continuous; warehouses every 30 min by default | GB moved plus connector instances | Free (10 GB); $0.50/GB + $100/mo per connector |
| Qlik Talend Cloud | Scheduled loads, log-based CDC | Real-time CDC from Standard | Capacity (data volume) | Sales quote |
| Apache Kafka | Event streaming | Continuous (milliseconds) | Your servers; Confluent Cloud: eCKU-hours, GB | Free download; Confluent Basic from $0/mo |
| Apache NiFi | Flow-based batch or streaming, MySQL CDC | Continuous or per-processor schedule | Your servers | Free download |
| AWS Glue | Scheduled batch, streaming ETL | Continuous streaming; batch on schedule or event | DPU-hours, per second | Pay as you go: $0.44/DPU-hour ($0.308 on 6.0) |
| Azure Data Factory | Scheduled and event-based batch, CDC | Every minute (schedule trigger) | Activity runs, DIU-hours, vCore-hours | Pay as you go: $1 per 1,000 runs |
Batch, CDC or streaming: which mode of ingestion do you need?
Latency first. It drives nearly every other choice, whether you notice or not. Is a daily or hourly refresh good enough? For most reporting and BI it is, and then you’re in batch ingestion territory, which costs less and leaves far less to maintain. That’s home turf for Skyvia, Fivetran, Airbyte and Hevo, and your plan decides how often they run.
Replicating a production database? Then change data capture is the middle ground. Instead of rescanning tables, a CDC pipeline reads inserts, updates and deletes straight from the database’s log. The source barely feels it, and deletes get caught, which timestamp-based loads tend to miss. Every one of Fivetran, Airbyte, Hevo, Estuary and Qlik Talend Cloud reads logs. What differs is the delivery: most deliver on a schedule, while Estuary and Talend’s real-time CDC deliver continuously.
Streaming exists for events that can’t wait, like fraud checks, live inventory and IoT devices reporting every second. Kafka moves each event as it happens and holds on to it so several consumers can read it. It also eats the most infrastructure and attention. Ben Rogojan (Seattle Data Guy) said it in September 2026: “Even in 2026, real-time set-ups tend to be more expensive one way or another.”
What you’re moving matters next. A CRM such as Salesforce or HubSpot, like most SaaS applications, needs API connectors that can live with rate limits and fields that change. A production database calls for incremental loads or CDC. Files show up in assorted formats (CSV, nested JSON, Parquet), and a data lake in S3 accepts raw files that data warehouses would reject until they’re modeled. With varied data sources, a big connector catalog saves you more, and a separate ingestion pipeline per source piles up maintenance downstream.
And finally, who’s going to run it? Go with a managed service (Skyvia, Fivetran, Airbyte Cloud, Hevo, Estuary) and the servers aren’t your problem. Self-host Airbyte, Kafka or NiFi and you get control and a free license, along with the on-call shifts. Cloud-native services land in between. No servers to run, but you write code and configuration that only work in that cloud.
FAQ for Data Ingestion Tools
Can data ingestion tools transform data too?
Some make small changes on the way in, such as mapping, filtering or hashing personal data. Most of the heavy work happens after the load, inside the warehouse. Fivetran, Hevo and Airbyte run or trigger dbt there, and Azure Data Factory has mapping data flows on Spark.
What happens if a source schema changes?
It comes down to the tool’s schema-change setting. In Fivetran and Airbyte you choose per connection whether to allow new columns, allow everything or block changes; Hevo uses a policy per pipeline instead. Skyvia won’t detect metadata changes on its own, so updating the integration is your job. Check what happens to deleted columns too. Fivetran and Airbyte keep them and write NULLs from then on.
Should my team build its own ingestion pipelines in Python?
You can, as long as someone’s ready to maintain them for as long as they run. A script keeps working until an API changes its auth, a field gets renamed or volume spikes. Then somebody’s rewriting code instead of analyzing data. Libraries such as dlt (open-source) make it easier, but only a managed tool actually takes connector upkeep off your team.
Nothing beats a small pilot on your own data for finding the right tool. Prefer to run it without writing code? Skyvia’s free plan asks for no credit card. Connect a source, pick a warehouse, and the first sync lands while you watch.
