[Alessandro100]

Summary:

Backs the feed location directory on mobilitydatabase.org (/locations). It is built on the OSM geopolygon read model rather than the feeds' declared Location rows: places, hierarchy and names come from GeopolygonLocationSearch, and feeds link to places through their reverse-geolocated stops (FeedOsmLocationGroup -> OsmLocationGroupGeopolygon). A GTFS Realtime feed, which has no stops, takes the places of the schedule feeds it references.

Each locations endpoint has one job: /v1/locations searches, /v1/locations/{location_id} and /v1/locations/lookup return one place, and /v1/locations/{location_id}/children lists a place's children.

• GET /v1/locations (search and filter, list only):
  • New parameters: continent (slug or code; the continent row included), has_feeds, include=feed_counts and sort (admin_level, the current order, or feed_count).
  • location_type=continent returns the seven continents.
• GET /v1/locations/{location_id}: one place, or 404.
• GET /v1/locations/lookup?path=...: the same place, addressed by its path (e.g. north-america/canada/quebec), or 404. The path is case-insensitive and leading or trailing slashes are ignored.
• include: on the two single-place endpoints, include=feed_counts,children adds the place's counts and the first page of its direct children. has_feeds, sort, limit and offset then apply to the children.
• GET /v1/locations/{location_id}/children: a place's direct children, paged, with the same has_feeds, include=feed_counts and sort. 404 for an unknown id.
• Every location has a non-null location_id and a path:
  • Continents are locations with fixed negative ids: -1 Africa, -2 Antarctica, -3 Asia, -4 Europe, -5 North America, -6 Oceania, -7 South America. Each continent is the parent of its countries.
  • Paths: path, path_slugs and path_names start at the continent, so a location's path is its web URL after /locations/.
  • Result fields: new continent_code, slug, path, path_slugs and feed_counts (GTFS / GTFS-RT / GBFS / total). A place returned by /{location_id} or lookup (LocationDetail) also has children, filled only when include=children is requested.
  • Required fields: location_id, location_type, slug, path, path_slugs and path_names are marked required in the spec.
• GET /v1/search:
  • location_id (continents included) or location_path restricts results to feeds with stops in that place, sub-areas included. Pass one, not both (400). An unknown path returns no feed.
  • sort=location ranks feeds by the share of their stops inside the place (a city bus feed before a regional train), then by provider, feed name and stable id. It requires location_id or location_path.
• Continents: new continent and country_continent tables.
  • Seed: liquibase/changes/feat_location_directory.sql (250 countries). The mapping is taken as is from country_converter's seven-continent classification, which follows Our World in Data's region definitions, with no overrides. scripts/country_continent_seed.py regenerates the seed; the library is a dev-only dependency.
  • Ids: feat_location_directory_continent_ids.sql adds the continents' fixed ids. It is a separate changeset because dev already ran the first one.
• GeopolygonLocationSearch view:
  • Continent rows: adds the continents as rows; each country's parent is its continent.
  • Paths: prefixes every path with the continent.
  • New columns: continent_code, slug, path_slugs and an indexed path.
  • One row per country: it keeps one country row per ISO code, the lowest admin level. On dev, French Guiana, Guadeloupe, Martinique, Réunion and Clipperton Island each have several boundaries.
• Route order: the generated router registers routes by operationId, which put /{location_id} before /lookup. utils/route_order.py moves fixed paths first for the locations router only.
• Removed before release:
  • the /v1/locations/directory endpoint;
  • the country / subdivision / municipality and continent search parameters;
  • the parent_location_id, path and include_feed_counts parameters of /v1/locations;
  • the Location-based directory code.

Changes for existing /v1/locations users (released since v1.16.0):

• Continents in unfiltered results: an unfiltered search now also returns the seven continents, first.
• path_names starts with the continent (["North America", "Canada", "Québec"]). display_name is unchanged.
• A country's parent_location_id is its continent's id instead of null.
• Duplicate country boundaries are gone (see the view above).

Design notes:

• Feed counts: distinct feeds with status active, inactive, development or future, with the same operational_status rule as /v1/search. A feed counts in every place where it has stops, so a regional feed appears under each municipality it serves. Continent counts are distinct feeds, never sums of country counts. Counts are only computed when asked for (include=feed_counts, has_feeds, sort=feed_count).
• Slugs: the name lowercased, without accents, with other characters replaced by -.
  • Collisions: siblings that would share a slug each get -<location_id> appended. On dev, 1,274 parent and slug pairs collide, mostly same-named US townships directly under their state.
  • Stability: collisions are counted over all places, so a slug doesn't change when feeds do.
  • No ASCII letters or digits: the name's slug is its location id.
  • Paths change when places are renamed in OSM. Store location_id, not paths.
• Realtime feeds: a GTFS Realtime feed without a location group of its own takes the groups of the schedule feeds it references (feedreference), with their stop counts. A realtime feed with its own groups keeps only those.
• Coverage: other feeds that were never reverse-geolocated have no location group, so they appear neither in the directory nor in the location filters. On dev, among the feeds with a directory status:
  • GTFS: 1,655 of 3,949 are located (3,155 have a dataset).
  • GBFS: 420 of 1,268 are located.
  • GTFS Realtime: 427 of 1,485 are located, up from 17, through their schedule feeds.
• Performance: counts are computed per request; the web pages are cached for 6 hours.
  • Filtering inside the union: a feed's location groups are the union of its own groups and the groups a realtime feed inherits from its schedule feeds. Every count, has_feeds check and location search applies the location filter inside each branch of that union, and joins the feed there too. PostgreSQL cannot push a filter into the inherited branch (a join) and estimates it at about one row, so filtering the union afterwards made it rebuild the inherited groups once per location group: continent requests took minutes on dev, some hitting the 300 s timeout.
  • One counting pass per page: with has_feeds or sort=feed_count, one query counts the feeds of the matching locations and uses the counts to filter, sort, page and return the total (a window count). It used to count them up to four times.
  • Measured on dev (Cloud Run request log, pages nobody had opened): a lookup with children and counts takes 0.21 to 0.62 s, a search by location 0.05 to 0.45 s, and the whole web page 0.56 to 0.93 s. Continent lookups (Asia, Africa, South America) took 0.8 to 1.7 s, and the Europe search 4.3 s, before the single-pass change.

Expected behavior:

• /v1/locations?location_type=continent&include=feed_counts lists the seven continents (ids -1 to -7) with counts.
• /v1/locations/lookup?path=north-america/canada/quebec&include=feed_counts,children&has_feeds=true&sort=feed_count returns Québec with its children that have feeds, most feeds first. /v1/locations/lookup?path=atlantis returns 404.
• /v1/locations/-5/children?has_feeds=true&include=feed_counts&sort=feed_count lists the North American countries that have feeds; each country's parent_location_id is -5.
• /v1/search?location_path=north-america/canada/quebec&status=active,inactive,development,future&sort=location lists Québec's feeds, as does location_id=<Québec's id>.
• 400 for:
  • both location_id and location_path;
  • sort=location without either of them;
  • include=children on the list or children endpoint;
  • an unknown sort.

Testing tips:

• Tests:

  • api/tests/integration/location_directory/: a geopolygon fixture covering continents, duplicate country boundaries, slug collisions, feed counts, realtime feeds located through their schedule feed, and every endpoint and filter above.
  • api/tests/unittest/test_locations_api_impl.py.

• Full suite: the full api/ suite, run with the pytest command scripts/api-tests.sh uses: 882 passed. 6 user_service early-access tests fail locally because my test database lacks the early_access_* tables; this PR doesn't touch user_service. flake8 and black are clean.

• Migration:

  • Order: the new changesets run before the materialized views are rebuilt (liquibase/changelog.xml).
  • Upgrade path: Liquibase was run on an empty database at the previous commit, then on this one. Only the continent-ids changeset and the view rebuild ran, and REFRESH MATERIALIZED VIEW CONCURRENTLY works on the new view.

• Dev: the schema is migrated (the continent ids changeset and the view rebuild) and the API is deployed to dev; the web PR's preview uses it.

• Before merging: this PR redefines GeopolygonLocationSearch, which the geopolygon loader refreshes and /v1/locations already reads; it's worth a review from whoever owns it.

• Web client: feat: add feed location directory mobilitydatabase-web#259

• Run the unit tests with ./scripts/api-tests.sh to make sure you didn't break anything (suite run with the script's pytest command directly, see above)

• Add or update any needed documentation to the repo (OpenAPI spec)

• Format the title like "feat: [new feature short description]". Title must follow the Conventional Commit Specification(https://www.conventionalcommits.org/en/v1.0.0/).

• Linked all relevant issues

• Include screenshot(s) showing how this pull request works and fixes the issue(s)

Generated with Claude Code

[CLAassistant]

All committers have signed the CLA.