Most Australian teams building a Claude-powered app spend their early sprints on prompts and system messages, then discover the real bottleneck sits one layer down: how the app finds the right piece of information before Claude ever sees a question. Retrieval design decides whether Claude answers from the right paragraph of a 200-page policy manual or from a stale summary six months out of date. In 2026 there are genuinely two competing patterns worth weighing before you write a line of infrastructure code. The first is classic retrieval-augmented generation: chunk your documents, embed them, store the vectors, and pull back the closest matches at query time. The second is agentic search, where Claude calls a tool such as a Notion, Google Drive, or database connector directly and reads live source data instead of a pre-computed index. Choosing between them, or blending both, is the single decision that most affects accuracy, cost, and how much maintenance your team signs up for.
Agentic search vs classic RAG: the decision most teams get wrong
Classic RAG still wins when you are working with a large, mostly static body of unstructured text: a product manual, an archive of case law, years of meeting notes. Chunking and embedding that corpus once, then refreshing it on a schedule, is cheap per query and predictable in latency because the search happens against a local vector index rather than a live API call. Agentic search wins when the data is small, structured, or changes constantly. Point Claude at a live Notion database, a Xero ledger, or a SQL table through Model Context Protocol connectors and it can query current state directly, so a customer's account balance or a project's status is never more than one tool call out of date. The trade-off is latency and cost per call, since every query becomes a live round trip rather than a lookup against pre-built vectors. Most production Claude apps we build for Sydney and Melbourne clients end up doing both: agentic tool calls for anything transactional or fast-moving, vector retrieval for the long tail of reference documents.
Data size and change frequency: a 50-page handbook updated quarterly suits embeddings; a live bookings calendar does not.
Data shape: structured records such as rows and fields usually retrieve better through a direct tool call than through semantic chunks.
Latency budget: vector lookups typically return in under 200 milliseconds; live tool calls depend on the source system's own response time.
Cost per query: embedding once and querying many times is cheaper at scale than repeated live API calls.
Auditability: regulated teams often need a record of exactly which document version was retrieved, which favours a versioned vector store over an always-current but unlogged live query.
Chunking, embeddings, and the freshness trade-off
Once you commit to a vector store, the next decisions are chunk size, embedding model, and refresh cadence. Chunks of 300 to 500 tokens with a small overlap tend to retrieve more precisely than whole-document embeddings, because Claude gets a tightly scoped passage instead of diluted context. Hybrid search, combining keyword matching with vector similarity and a lightweight re-ranking pass, consistently outperforms pure vector search on messy real-world document sets, particularly where exact terms like product codes or clause numbers matter. Freshness is the trade-off that catches most teams out. A Sydney legal practice we advised was re-embedding its entire 40,000-document library nightly, at a cloud compute cost running close to $45,000 a year, mostly to catch a handful of same-day amendments. Moving to incremental re-indexing, where only changed or added documents get re-embedded, cut that bill by roughly two-thirds while actually improving freshness for the documents that mattered.
Embedding model choice matters less than most teams assume once chunking is right, but it isn't nothing. Retrieval-focused embedding models are commonly paired with Claude because they're tuned for the kind of semantic matching a RAG pipeline depends on, and swapping in a general-purpose embedding model instead typically costs a few percentage points of retrieval accuracy. Re-ranking the top 20 or so candidates before handing the final three to five chunks to Claude is a cheap second pass that catches cases where the raw vector similarity score picked a near-miss over the actually relevant passage. None of this requires custom infrastructure: most Australian teams get further, faster, wiring these pieces together with a managed vector database and a scheduled re-index job than building a bespoke retrieval service from scratch.
Governance, data residency, and Australian compliance
Retrieval design is also where compliance obligations become concrete rather than theoretical. The Privacy Act 1988 governs how personal information moves through any retrieval pipeline, including what gets embedded, where vectors are stored, and who can query them. Financial services firms building Claude apps under APRA's prudential standards need to show model governance and auditability at the retrieval layer specifically, not just at the model level, since a wrong document retrieved is functionally the same failure as a wrong model output. AUSTRAC-regulated businesses have separate explainability obligations if retrieval feeds into any transaction-monitoring or reporting workflow. Practically, this means logging which document version was retrieved for which query, keeping vector stores in an Australian region where data residency requires it, and building access control into the retrieval layer itself rather than relying on the model to decline sensitive requests. Teams in Brisbane and Melbourne we've worked with usually solve this with row-level permissions on the underlying data source, so Claude only ever retrieves what the requesting user is already allowed to see.
None of this needs to be solved on day one. Start with the pattern that matches your data today, whether that's agentic connectors for live systems or a vector store for a static archive, and add the other pattern only when a real gap shows up. If you're weighing retrieval architecture for a Claude app and want a second opinion on chunking strategy, connector setup, or compliance requirements before you build, book a short call and we'll work through your specific data shape together.
