Your Teams Don't Have a Data Problem. They Have an Integration Problem.

If you work in analytics, machine learning, or application development, you know the scene: the project gets approved, the team is excited, and then weeks go by discovering where the data lives, who maintains it, and why the customer_id field means three different things depending on the system.

The model isn’t delayed because of a lack of talent. It’s delayed because the data arrives late, incomplete, and without context.

That’s where AWS Glue comes in: AWS’s serverless data integration service that discovers, prepares, moves, and combines information from multiple sources without you managing a single server. And in version 6.0, available since August 2026, the value proposition changed enough to warrant a review of the architecture you have today.

-30% processing price versus previous versions · +60 native connectors, plus reading from any REST API · single-digit millisecond latency in stateless streaming

The Silent Cost of Moving Data

Before talking about the tool, it’s worth naming the problem. In most of the organizations we work with, the cost of data integration is spread across three places nobody measures.

Cost 1: Engineering time. Writing and maintaining orchestration code, custom connectors, and retry logic. Work that doesn’t differentiate the business and consumes the team’s most expensive profiles.

Cost 2: Idle infrastructure. Spark clusters left running “just in case,” sized for the month-end peak and billing the rest of the time.

Cost 3: Decisions made blind. When an analyst can’t find the right table, they don’t stop deciding — they decide with the wrong table.

All three are fixed with the same lever: a managed integration layer, with a reliable catalog at the center.

What AWS Glue Does for Your Operation

AWS Glue brings together in a single service what are normally four separate projects:

• Data catalog. A central registry of schemas and metadata that Amazon Athena, Amazon Redshift, Amazon EMR, and your notebooks query as the single source of truth.
• Automatic discovery. Crawlers scan your sources, infer the schema, and register it without manual intervention.
• Serverless ETL. Jobs run on managed Apache Spark, billed per second of use.
• Broad connectivity. More than 60 native connectors and, since the native REST connector launched in February 2026, the ability to read from any source that exposes a REST API without packaging your own libraries.

Translated into investment-committee language, this is a shift in spending model:

Traditional model: you pay for what you provision. Clusters sized for the worst-case scenario, running all month and billing whether anyone uses them or not.

Serverless model: you pay for what you process. Billed per second of execution, with no infrastructure to manage and no idle capacity on the invoice.

What Changes with AWS Glue 6.0

Version 6.0 arrived with a modernized runtime (Apache Spark 4.1, Python 3.13, Scala 2.13) and five changes with a direct effect on cost and timelines.

1. Thirty percent lower processing price

This is the news that opens any budget conversation. The same recurring workload costs a third less than in previous versions. For a company with daily consolidation pipelines, that reorders the business case for the entire lakehouse.

2. Full Apache Iceberg v3 support

AWS Glue 6.0 implements the full Apache Iceberg v3 spec, with concrete effects on your tables’ schema and portability. What matters for your team:

• VARIANT data type with shredding: store and query JSON, logs, and events without flattening the schema. No more duplicate copies, custom parsing code, or pipelines that break every time a source system adds a field.
• Geometry and geography types: spatial analytics and location intelligence natively, with no external libraries.
• Nanosecond-precision timestamps: for IoT telemetry, scientific computing, and high-frequency financial operations.
• Unknown-type handling: schemas that evolve upstream stop breaking the process downstream.

A table written in Iceberg format can also be queried from other engines in the ecosystem without migrating data, which reduces the risk of vendor lock-in.

3. Spark Declarative Pipelines

Your team declares how the data should look, and the engine decides the execution order. Less hand-written orchestration code means less surface area for errors and a shorter ramp-up for engineers joining the project.

4. Native Python UDFs on Arrow

User-defined functions no longer pay the serialization cost between Python and the JVM. Complex PySpark transformations finish faster, and overnight processing windows get shorter.

5. Real-time streaming mode

For stateless use cases, AWS Glue 6.0 reaches single-digit millisecond latency. Fraud detection, event routing, and live personalization no longer need a separate architecture.

One detail that often decides the migration: there are no API changes. You select the version from the console, the CLI, or the SDK, and there’s an upgrade agent to move existing jobs to the new version.

The Catalog Now Understands the Business Too

In June 2026, AWS launched a preview of business context and semantic search for the Glue Data Catalog. You can now enrich your tables with glossary terms and your own metadata fields, and search for information by meaning instead of just column name.

For teams building AI agents on top of their data, the agent now responds with business-approved definitions instead of inferring them.

Three Scenarios Where This Translates Into Results

Scenario 1: Analytics. A unified catalog on top of Amazon S3 lets finance, operations, and sales query the same data with Athena or Redshift. No more meetings where every department shows up with its own sales figure.

Scenario 2: Machine learning. Feature engineering stops depending on manual extracts. Data arrives versioned, with lineage, and with granular access control via AWS Lake Formation.

Scenario 3: Application development. Product teams consume governed data through APIs and Iceberg tables, without building their own pipeline for every new feature.

How We Implement It

We work in four phases, with verifiable deliverables at each one.

1. Diagnosis (1 to 2 weeks). Inventory of sources, map of current flows, and cost of the current operation. You end this phase knowing exactly how much you spend on integration today.
2. Lakehouse design. Definition of the data model, table format, governance policies, and cost estimate with AWS Glue 6.0.
3. Implementation by domain. We start with the use case with the highest return, take it to production, and scale from there. No twelve-month projects without intermediate results.
4. Handover and support. Your team ends up operating the platform. We remain as backup, not as a dependency.

Questions We Get Before Starting

“Do we have to rewrite our current jobs?” No. AWS Glue 6.0 doesn’t require API changes, and the Spark upgrade agent assists in migrating existing jobs.

“We already use another data platform — does this replace it?” Not necessarily. Since it writes in Apache Iceberg, your tables stay queryable from other engines. The decision stops being all-or-nothing.

“How do we control who sees what?” With AWS Lake Formation enforcing granular access control, including control over write operations.

“How long until we see the first result?” In our projects, the first data domain goes into production within the first quarter. The diagnosis delivers numbers before you sign off on the implementation.

Start by Knowing What It Costs You Today

Before moving a single pipeline, it’s worth having the number on the table. Our AWS-certified team runs a free data integration diagnosis: we review your current architecture, identify the workloads consuming the most budget, and deliver a savings estimate with AWS Glue 6.0.

It’s 45 minutes with an architect, not a salesperson.