AIAIAI.FYI
LIVE

Exa gets 47% better embedding cost per million documents on AWS Inferentia2 using custom NKI kernels

Exa says agents wrote the optimizations, including a custom fused SwiGLU MLP kernel and a rewritten attention kernel, while the work carried over to Trainium2.

What happened

Exa runs a web search index. To search by meaning, it turns each document into a list of numbers, called an embedding. Exa re-embedded 30 billion documents on AWS AI chips and made the job cheaper by tuning the software that runs on them.

Why it matters to you

Embedding tens of billions of documents is costly, so small per-document savings add up for any company running search or AI agents at scale.

The takeaway

Exa says custom software tuning made its embedding work about 47% better in performance and cost per million documents.

What it means for you

If you build software

Shape-specific NKI kernels beat generic ones here, and a harness with a correctness gate and a saturation benchmark kept agent-written kernels from drifting.

If you run a business

Exa says per-document cost gains compound at tens or hundreds of billions of documents, and spreading work across hardware types and Regions reduces the risk of compute shortages.

If you invest

The source is an AWS blog post with a customer's self-reported results, and it gives no revenue or spending figures.

Get stories like this every morning