Repository navigation
feat: answer the user identifier filter from a search index - #2944
Merged
Merged
Conversation
The `identifier` filter on `POST /users/query` read every identifier row of the project: a case-insensitive substring match has no index. At 5 million users one request took 0.5 to 2.9 s on Postgres. A user's designated identifier and display parts now store a search text on their attribute row: the value under the case folding uniqueness uses, cut to 256 characters. One index, (project_id, key, search_text, user_id) over the rows that carry it, answers a prefix match as one range per property, read in index order. The filter becomes `identifier` with the new `starts_with` operation and returns the users ordered by identifier, so a page costs the same for one match and for every user (0.01 ms at 5 million users). - Migrations add the column and the index on Postgres, Spanner and SQLite and fill the column for existing users, with the database's lower() standing in for the server's folding. - A search read is an option on the users list statement: one statement per property, merged, paged by (search text, user id). - `sorting` cannot be combined with the filter, and the value is at most 256 characters. - The CLI and the console's Users search send `starts_with`. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
A search over several properties returns a user once per property matched. Hydration keyed the rows by user id, so only one of them got the attributes, teams and owner team. The user is now loaded once and every row gets it. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…umeric term A search that failed after the first load was rethrown during render, so the error boundary replaced the list, the filters and the field being typed in. It is now said above the list, as a failed Load more is, and the rows of the last search stay. A hand-typed `?q=12345` reaches the route as a number and was dropped as not text. It is searched as the text it is. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Codex Review Summary: ✅ CompletedThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
Contributor
🦋 Changeset detectedLatest commit: 010bd23 The changes in this PR will be included in the next version bump. This PR includes changesets to release 25 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
main took the migration numbers this branch used (the events indexes), so the search text migration moves behind them: 000042 on Postgres, 000046 on Spanner and 000029 on SQLite.
Contributor
There was a problem hiding this comment.
Note
Copilot was unable to complete this review at the selected tier due to model policy or plan restrictions.
🟡 Changes recommended
A unit test currently mutates the offered operations by always adding starts_with, which can obscure what the registry actually exposes and make the assertion less trustworthy until the expectation is rewritten without altering offered.
1 open issue
What changed in this PR
Switch user identifier querying from case-insensitive substring scanning (contains) to an indexed, case-insensitive prefix search (starts_with) by persisting normalized search_text on designated user attributes, updating the API contract, and aligning Console/CLI behavior and tests.
Changes:
- Add
user_attributes.search_text(backfilled) plus a cross-dialect(project_id, key, search_text, user_id)index and storage-layer prefix search with keyset paging. - Update OpenAPI and generated clients/validators to add
starts_withand restrict theidentifierfilter accordingly (including ordering and validation rules). - Update Console and CLI search UX/docs/tests to use
identifier=starts_with:....
| File | Description |
|---|---|
| packages/go/api/oas_validators_gen.go | Regenerates Go client validators to accept the new starts_with filter operation. |
| packages/go/api/oas_schemas_gen.go | Regenerates Go client schemas/enums to include starts_with and updated identifier filtering docs. |
| packages/go/api/oas_json_gen.go | Regenerates Go client JSON (un)marshalling to support starts_with. |
| docs/design/decisions.md | Updates design decision text to reflect prefix identifier search and identifier-ordered results. |
| docs/adrs/058-user-identity-designation-and-references.md | Adds/updates ADR notes describing persisted search_text on designated attributes for search/indexing. |
| docs/adrs/031-openapi-querying.md | Amends OpenAPI querying ADR to document starts_with semantics and related constraints. |
| apps/console/src/routes/_authed/users/users.spec.tsx | Updates Console users-page tests for the new starts_with search behavior and related UX cases. |
| apps/console/src/routes/_authed/users/index.tsx | Adjusts Users route behavior to handle search failures more gracefully and treat q consistently. |
| apps/console/src/features/users/list.ts | Switches the Users list filter builder from contains to starts_with for identifier search. |
| apps/cli/tests/unit/commands/resources.test.ts | Updates CLI unit tests around filter-operation exposure/expectations now that starts_with exists in shared enums. |
| apps/cli/tests/unit/commands/resources-list.test.ts | Updates CLI resources-list tests to expect users.identifier to support starts_with. |
| apps/cli/src/commands/resources.ts | Updates CLI resource/operation metadata handling to account for starts_with being globally defined but field-gated. |
| apps/cli/skills/zitadel-cli/SKILL.md | Updates the CLI skill/contract documentation to describe identifier=starts_with:... usage. |
| apps/cli/README.md | Regenerates CLI README/help content to reflect the new identifier filter operation. |
| apps/cli/command-reference.json | Regenerates CLI command reference data to advertise starts_with for identifier filtering. |
| apps/api/internal/storage/user/search.go | Implements shared storage search helpers for prefix bounds, ordering, paging, and per-arm merging. |
| apps/api/internal/storage/user/search_test.go | Adds unit coverage for search bounds/cursors/merge behavior and search-specific option validation. |
| apps/api/internal/storage/user/schema.go | Updates storage schema bindings to support search-related fields instead of substring identifier predicates. |
| apps/api/internal/storage/user/list.go | Extends list hydration/grouping to handle repeated users returned across search arms. |
| apps/api/internal/storage/stmttest/user_test.go | Updates statement tests to validate the new search-based identifier filtering, ordering, and paging semantics. |
| apps/api/internal/storage/stmttest/authz_list_predicate_test.go | Adjusts authz list predicate statement tests to work with the new search narrowing behavior. |
| apps/api/internal/storage/dialect/sqlite/user.go | Adds SQLite persistence and query paths for search_text-backed prefix search over user attributes. |
| apps/api/internal/storage/dialect/sqlite/migration/sql/000027_user_attributes_search_text.sql | Introduces SQLite migration to add/backfill search_text and create the supporting index. |
| apps/api/internal/storage/dialect/sqlite/migration/search_text_test.go | Adds SQLite migration tests validating search_text backfill rules for designated attributes. |
| apps/api/internal/storage/dialect/spanner/user.go | Adds Spanner persistence and query paths for search_text-backed prefix search. |
| apps/api/internal/storage/dialect/spanner/migration/sql/000044_user_attributes_search_text.sql | Introduces Spanner migration to add/backfill search_text and create the supporting index. |
| apps/api/internal/storage/dialect/postgres/user.go | Adds Postgres persistence and query paths for search_text-backed prefix search and keyset paging. |
| apps/api/internal/storage/dialect/postgres/migration/sql/000040_user_attributes_search_text.sql | Introduces Postgres migration to add/backfill search_text and create the supporting index. |
| apps/api/internal/storage/dialect/postgres/migration/search_text_test.go | Adds Postgres migration tests verifying correct search_text backfill behavior. |
| apps/api/internal/service/user.go | Routes identifier filters through the new search option, enforcing the starts_with contract and constraints. |
| apps/api/internal/service/user_test.go | Adds/updates service-level tests for starts_with validation, constraints, and error cases. |
| apps/api/internal/service/statement.go | Introduces/extends statement-layer structures for user search options and arm-based search execution. |
| apps/api/internal/service/list.go | Updates list/service logic to recognize starts_with and enforce where it is allowed. |
| apps/api/internal/domain/user.go | Extends domain models to carry matched/search text needed for ordering and paging. |
| apps/api/internal/domain/attribute.go | Adds domain-level search_text derivation/constraints and marking of searchable attributes. |
| apps/api/internal/domain/attribute_test.go | Adds unit tests for attribute search_text derivation, constraints, and marking behavior. |
| apps/api/internal/bootstrap/users/import.go | Marks imported user attributes as searchable based on schema designations to populate search_text. |
| apps/api/internal/api/integration_test/user_list_test.go | Adds integration coverage for identifier starts_with search behavior, ordering, and paging. |
| apps/api/internal/api/generated/oas_validators_gen.go | Regenerates server OpenAPI validators to accept starts_with where applicable. |
| apps/api/internal/api/generated/oas_schemas_gen.go | Regenerates server OpenAPI schemas/enums to include starts_with and updated docs. |
| apps/api/internal/api/generated/oas_json_gen.go | Regenerates server OpenAPI JSON (un)marshalling to support starts_with. |
| apps/api/api/openapi/endpoints/users/query/user-filter-field.yaml | Updates the OpenAPI definition for the users query identifier filter to use starts_with and document constraints. |
| apps/api/api/openapi/components/schemas/filter-operation.yaml | Adds starts_with to the shared OpenAPI filter-operation enum. |
| apps/api/api/openapi/bundled/openapi.json | Updates the bundled OpenAPI artifact to reflect the new filter operation and identifier search contract. |
| .changeset/user-identifier-filter.md | Updates release notes to describe the identifier filter change to starts_with and associated package impact. |
Review effort: Balanced
The test added `starts_with` to the set of offered operations before comparing it, so the set no longer said what the registry offers. The expectation now allows for it instead, and a second test names the one field that offers it. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Context
identifiercontainsfilter reads every identifier row of the project, because no index serves a case-insensitive substring match. This PR targets feat: find users by identifier in the API, the CLI and the console #2688's branch; it moves tomainonce feat: find users by identifier in the API, the CLI and the console #2688 merges.Summary
user_attributes.search_text): the value under the case folding uniqueness uses, cut to 256 characters. One index,(project_id, key, search_text, user_id)over the rows that carry it, answers a prefix match as one range per property, read in index order. Migrations add both on Postgres, Spanner and SQLite and fill the column for existing users.identifieronPOST /users/querytakes the newstarts_withoperation instead ofcontains. The users come back ordered by identifier, thenid;sortingcannot be combined with the filter, it may be given once, and its value is at most 256 characters.starts_withjoins the sharedfilter-operationenum, and only a field that names it accepts it (ADR 031 amendment, ADR 058 note).UserQueryOptions.Search): one statement per property, merged, paged by (search text, user id). The CLI and the console's Users search sendstarts_with.id = ''sentinel. Two console ones are fixed in a commit of their own: a failed search no longer replaces the Users screen, and a hand-typed?q=12345is searched.Validation
moon run server:generate,server:openapi,workspace:check-openapi-rules,server:format,server:lint,server:check-generate-prunedgo test -count=1 ./...inapps/apigo test -tags sqlite_integration -count=1 ./internal/api/integration_test/ ./internal/storage/...go test -tags postgres_integration -count=1 -p 1 ./internal/api/integration_test/... ./internal/storage/...against a throwaway local Postgres 18 (notserver:test-postgres)moon run console:typecheck console:lint console:test,moon run cli:typecheck cli:lint cli:testNot run: the Spanner lane (its migration and statements compile; only CI's emulator has run them), the e2e lanes and the docs browser tests. In the full CLI suite two timing specs of
tests/integration/status.spec.tsexceeded their 10 s bound on a busy laptop; they pass on their own.Release notes / changeset
.changeset/user-identifier-filter.md, feat: find users by identifier in the API, the CLI and the console #2688's, edited: it now namesstarts_withand adds@zitadel/api(minor) for the new enum value.Notes
acme.comdoes not findann@acme.com. That is the price of an index that is the same on all three databases.sorting.lower(), which differs from the server's folding for a few letters outside ASCII (ß, final sigma) and is ASCII-only on SQLite. Such a value is found again after the user's next write.mainadded.schema_urlper revision that designates it.x-displayparts carry search text too, which feat(api): find users and teams in a project with one search request #2935 uses to find people by name. Theidentifierfilter reads identifiers only.🤖 Generated with Claude Code