Skip to content

Improve metadata lookup during restore - #9290

Open
dyemanov wants to merge 1 commit into
masterfrom
work/gbak-meta-dict
Open

dyemanov wants to merge 1 commit into
masterfrom
work/gbak-meta-dict

Conversation

@dyemanov

Copy link
Copy Markdown
Member

The original code used O(n) linear searches through linked lists (relations / packages / procedures / miss_privs) for object lookups during get_user_privilege processing.

This PR replaces linear linked-list traversals with a tree-based dictionary lookups via MetaDictionary, keyed by (object_type, QualifiedMetaString).

The measured effect is up to 20% restore time decrease for artificially generated metadata and 9-10% decrease for real-world database containing 1,941,409 user privileges. This is for metadata-only restore, of course. The numbers are not very impressive, but even a small boost wouldn't hurt, I suppose.

@AlexPeshkoff

Copy link
Copy Markdown
Member

The only question is - why tree? Did you try with hash?

@dyemanov

Copy link
Copy Markdown
Member Author

The only question is - why tree? Did you try with hash?

To compete with the tree speed-wise, the hash table should be rather large (way more than 10000 buckets). I tried our HashTable with SIZE=9973 and it was noticeably slower than the tree. Given that the hash table is statically preallocated and BurpGlobals is instantiated per every parallel worker it didn't look fine to me.

Another option could be std::unordered_map with Firebird::PoolAllocator -- as it extends dynamically -- but the quick test shows the same performance as the suggested GenericMap-based implementation. I'm not sure it's worth investing more time into such micro-optimizations.

@tomaszdubiel18

Copy link
Copy Markdown

Hello. I have some spare ChatGPT usage so I checked this. If you don't want me to do this, let me know.
Below are the results of analysis performed by AI.

Summary: metadata lookup is measurably cheaper with this change. No functional regression was found in the tested cases.

Compared 14e1466 with ed2c21d (Firebird 6.0.0.2459, Linux x64, Release). For a synthetic metadata-only backup with 1,500 tables and 150,000 additional SELECT grants, five alternating measured pairs gave an embedded restore median of 11.21 → 9.61 s (14.3% shorter). The runs were pinned to one CPU; baseline-only A/A control medians differed by 4.4%. Peak RSS increased by approximately 3.5 MiB for this dataset.

TCP restore to the same unchanged server showed 3.9% lower median elapsed time and 46.7% less CPU in the gbak client. TCP elapsed-time measurements were noisy, so that percentage is only indicative. No reliable elapsed-time gain was established for the small database or the 1,500-table case without additional grants.

Functional checks passed for full and metadata-only restore, -parallel 4, Services, a Firebird 5.0.4 backup, tablespace remapping, cross-schema/type name collisions, packaged routines, quoted Unicode names, and empty/no-table databases. Selected metadata, complete privilege rows and data matched between builds; effective permissions and expected access denials also passed. ServerMode was Super; embedded was tested separately. Windows and other ServerModes were not tested. The author's real-world database was not available, so its reported percentage was not independently reproduced.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants