Skip to content

perf(plugins): list every schema's tables in one query on Oracle, Dameng, HANA, Trino, CockroachDB and Redshift - #3708

Open
datlechin wants to merge 4 commits into
mainfrom
perf/3661-large-catalog
Open

datlechin wants to merge 4 commits into
mainfrom
perf/3661-large-catalog

Conversation

@datlechin

@datlechin datlechin commented Oct 20, 2026 •

Copy link
Copy Markdown
Member

Summary

  • Open Quickly, sidebar search, MCP schema search and the backup scope list other schemas' tables through CatalogTableListing. With no all-schema answer from the driver it went straight to one query per schema. It now lists the schemas, drops the system ones, and asks fetchTables(inSchemas:) once. Per schema only when the driver has no set read, or the set read fails for a reason other than a lost connection.
  • fetchTables(inSchemas:) on Oracle, Dameng, SAP HANA, Trino, CockroachDB and Redshift: the per-schema query widened to IN (...), schema projected on every row. Oracle and Dameng split at 1,000 names (ORA-01795). Redshift still adds external tables per external schema. Trino falls back to per-schema materialized-view reads when the combined one fails.
  • Naming the schemas keeps Oracle's read off SYS and the other Oracle-maintained schemas, the reason feat(plugins): list every schema's tables in one query on SQL Server and DuckDB #3096 left Oracle on the fallback.
  • Oracle, PostgreSQL-family and Trino literal and identifier quoting match by code unit. A quote followed by a combining mark (x'́) was left undoubled: PostgreSQL 17 rejects that listing with a syntax error, and in a set read one such schema name would fail every schema.
  • apple/scripts/probes/build-large-catalog-fixtures.sh builds the Measure the sidebar, Open Quickly and autocomplete on 100 schemas and 50,000 tables #3661 fixtures: PostgreSQL 100 schemas x 500 tables, Oracle the same in a gvenzl/oracle-free container, MySQL one database of 50,000 tables.
  • New TableProTrinoCoreTests target in TableProCore.
  • No PluginKit change. Oracle, Dameng, HANA and Trino are registry plugins and need a plugin release to reach users. CockroachDB and Redshift ship in the bundled PostgreSQL plugin.

Server time for the other schemas' tables on the PostgreSQL fixture, PostgreSQL 17.11 local, information_schema query CockroachDB and Redshift use, 3 runs on a loaded machine:

Read Queries Time
Per schema 100 2,488 to 3,382 ms
Set read 1 54 to 77 ms

Test plan

  • CatalogTableListingTests: every schema through the set read (schema list, then one read with the excluded schemas left out); a failed set read falls back per schema and names only the failing schema; a lost connection in the set read fails the listing without per-schema reads.
  • TableProOracleCoreTests (swift test): 47/47 pass, including the set query, the 1,000-name split, the owner column, the combining-mark escape and SYS qualification of the new query.
  • TableProTrinoCoreTests (swift test): 3/3 pass.
  • PostgreSQLDriverTests: Cockroach and Redshift set queries and rows, combining-mark quoting; new builders added to the literal-quoting matrix.
  • Probe: PostgreSQL fixture built in 234 s; MySQL path built 1,200 tables; Oracle path not run (no Docker here).
  • Not run locally: the Xcode build, Unit plan, AllPlugins and iOS steps. A disk guard from another project deleted DerivedData's shared module caches every minute while the disk was under 40 GB. Standalone swiftc -typecheck of the PostgreSQL, Oracle, Trino, HANA and Dameng plugin sources and the changed PostgreSQL tests passed. CI covers the rest.

Not in this PR

Docs: TableProApp/docs#67

Refs #3661

# Conflicts:
#	apple/Packages/TableProOracle/Sources/TableProOracleCore/OracleSchemaQueries.swift

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant