How to Add Full-Text Search to Your App with a REST API

Two HTTP calls. Push a batch of JSON documents to an index — which is created by that first write — then POST a query string to the same index and read back ranked results. No cluster to provision, no schema to declare up front, no client library required. This post walks the whole path: documents, fielded queries, facets, sorting, snippets, and pagination, with the exact request shapes.

Step 1: a key, a base URL, an instance

Everything goes to one host, https://api.altengine.net, authenticated with an organization API key as a bearer token:

Authorization: Bearer ae_yourkeyid.your-api-key-secret

Keys carry a grant level. read lets you search and read documents; write adds putting documents; full adds deleting documents and dropping indexes. Mint the narrowest one that works — a search-only web tier wants read, your ingest job wants write.

You provision a named search instance in the console (say catalog). Inside it, every path is scoped to a namespace and an index:

/v1/search/{instance}/ns/{namespace}/idx/{index}/…

The namespace partitions data for multi-tenancy — one per customer if you need that. If you don't, use _default; it can't be an empty path segment, so it has to be spelled out. Indexes need no creation step: the first document you write into a name brings it into existence.

Step 2: push documents

A document is a string id, a list of fields, and optionally facets, a numeric rank, and a lang tag. Fields are typed and multi-valued, and the schema is dynamic — two documents in the same index can carry entirely different fields.

POST /v1/search/catalog/ns/_default/idx/films/documents

{
  "documents": [
    {
      "id": "f1",
      "fields": [
        { "name": "title",    "type": "text",   "value": "Up in the Air" },
        { "name": "genre",    "type": "atom",   "value": "drama" },
        { "name": "rating",   "type": "number", "value": 4 },
        { "name": "released", "type": "date",   "value": "2009-12-04" }
      ],
      "facets": [
        { "name": "genre", "type": "atom",   "value": "drama" },
        { "name": "year",  "type": "number", "value": 2009 }
      ]
    }
  ]
}

# → { "ids": ["f1"] }

Picking types is most of the design work, and it's short:

  • text — tokenized full text; term, phrase, and stem matching. Titles, descriptions, body copy.
  • html — same, but tags are stripped before tokenizing.
  • atom — an exact-match string, not tokenized, up to 500 bytes. Slugs, SKUs, statuses, tag values.
  • number and date (YYYY-MM-DD) — range comparisons and sorting.
  • geo — a { "lat", "lng" } point, filtered with distance().
  • tokenprefix / untokenprefix — prefix matching for autocomplete, over words or over a whole string respectively.

The one thing that trips people up: facets are not fields. fields make a value searchable; facets make it countable. Faceting on something written only as a field returns nothing at all — not an error, just an empty result, which reads like "there is no such value". To search and facet on genre, write it in both places, exactly as above.

Put up to 200 documents per request; putting an existing id replaces it. To load a real dataset, fill every batch to 200 and keep roughly 6–8 requests in flight per index — that sustains on the order of 2,000 documents a second. Push harder and writes come back 429 with a Retry-After; pause and retry the same batch, since a put is idempotent by id.

Step 3: query it

Search is a POST to the index. Only query is required, and an empty query matches everything.

POST /v1/search/catalog/ns/_default/idx/films/search

{ "query": "genre:comedy rating > 3", "limit": 20 }

The query language mirrors App Engine's Search syntax, so terms combine with an implicit AND and the operators are the ones you already know:

  • Bare term: air — field scope: title:air, genre = "sci fi"
  • Comparisons: rating > 3, rating != 1, released < 2011-02-28
  • Boolean and grouping: comedy OR drama, NOT scifi, -scifi, genre:(comedy OR drama)
  • Phrase: "very important" — stemming: ~running — geo: distance(loc, geopoint(37.7, -122.4)) < 1000

One honest caveat on ~: stemming is an instance setting and it's off by default, because it costs extra storage. With it off, ~word behaves like a plain term. Turning it on applies to documents written from then on — existing documents aren't retroactively stemmed, so re-put them if you want them included.

What comes back

{
  "total_hits": 42,
  "total_hits_exact": true,
  "returned": 1,
  "results": [
    { "id": "f1", "rank": 12345, "score": 1.7, "document": { … } }
  ],
  "cursor": "eyJvIjoyMH0",
  "facets": [ … ]
}

Trim the payload with returned_fields (an array of names) or ids_only: true when you're going to hydrate from your own database anyway.

Step 4: facets

Set facet_discover: 5 and the response carries the most common facet values over the matches. Name specific atom facets with facets: ["genre"] if you always want the same ones. When a user clicks one, send it back as a refinement:

{
  "query": "space",
  "facet_discover": 5,
  "facet_refinements": [{ "name": "genre", "value": "scifi" }]
}

Atom facets return value counts; number facets return half-open [min, max) ranges with counts. That's the whole faceted-navigation loop — discover, render, refine.

Step 5: sorting and relevance

With no sort, results come back by rank descending — the numeric field on each document, which defaults to seconds since 2011-01-01 (App Engine's rank epoch) when you omit it. Set rank deliberately if you have a popularity or recency signal.

For an explicit sort, pass an array of { expr, desc, default }. For relevance, ask for the scorer and sort on _score:

{
  "query": "jacket",
  "scorer": "match",
  "sort": [{ "expr": "_score", "desc": true, "default": 0 }]
}

scorer: "match" turns on BM25 scoring; the score lands on each result and is sortable as _score. The default matters — it's what a document missing the sort field is treated as.

Step 6: pagination and counts

limit defaults to 20 and maxes at 1000. offset works but is capped at 1000, so use it only for a shallow page picker. For anything deep, follow the cursor: pass the previous response's cursor back as cursor in the next request, and keep going until the field is absent.

Counts are deliberately separate from paging. total_hits is exact only up to total_hits_accuracy, which defaults to 20 and maxes at 10000. When total_hits_exact is false, the number is a lower bound — render it as "N+" rather than pretending it's precise. Raising the accuracy costs more work per query, so raise it to what your UI actually shows and no further.

Snippets, nearly free

Add a snippet block and each result gains a snippet map of field name to highlighted excerpt:

{ "query": "rose", "snippet": { "fields": ["body"], "max_tokens": 32 } }

# → { "id": "d1", "rank": 42,
#      "snippet": { "body": "a <b>rose</b> by any other name…" } }

Everything in an excerpt except your own pre_tag/post_tag is HTML-escaped, so stored markup comes back inert and the excerpt is safe to render. Only tokenized fields (text, html, tokenprefix) can be excerpted, and snippets are computed only when asked for.

The limits worth knowing before you build

  • 200 documents per put, 200 ids per delete, 10 GB per index, 500 bytes per atom value.
  • 100 requests per second sustained per search instance, with a matching burst; over it you get 429 RATE_LIMITED. You can lower the limit per instance, never raise it above the plan maximum.
  • Field collapsing (collapse) can't yet be combined with an explicit sort or scorer — both return a 400 today.
  • Deleting the last document in an index removes the index itself; put a document again to recreate it.

What it costs to run

Search bills on queries plus reads, writes, and stored data. Queries are flat: $0.50 per 10,000 search requests, cached or not. Reads meter the work on top of that, by rows examined — one read is up to 100 rows, so a selective query is a read or two, and a repeated identical query is served from cache and bills zero reads (a write to the index invalidates that cache, so you never read stale results). Writes are metered by rows written: a document put writes its own row plus its index rows, which is why lean documents — only the fields you actually search — are cheaper on both writes and storage.

There's no idle cluster in that list, and that's the point: an index you aren't querying costs only its storage. One pay-as-you-go plan, the first $3 of usage each month free for every organization, no card to start. Full rates on the pricing page.

Coming from App Engine Search?

The semantics are the contract: the query language, field types, facets, and sorting behave the way you remember. What changes is the call layer — these are altengine's own REST/JSON endpoints, not App Engine's language SDK. So your queries and document schemas carry over, and what you rewrite is the code that issues the HTTP request. That's a real migration, just a small and mechanical one.

Everything above, plus synonyms, query rules, field collapsing, and the full limits table, is in the Search API reference.

Start free in the console

← Back to the blog