diff --git a/.claude/skills/uts-to-kotlin/SKILL.md b/.claude/skills/uts-to-kotlin/SKILL.md index f7f5368b8..9905d2b18 100644 --- a/.claude/skills/uts-to-kotlin/SKILL.md +++ b/.claude/skills/uts-to-kotlin/SKILL.md @@ -1,10 +1,10 @@ --- -description: "Translate the UTS pseudocode test specs in a whole module directory into runnable Kotlin tests in the owning ably-java module (:java for realtime/rest, :liveobjects for objects; :uts hosts the shared infra + smoke examples). Takes a UTS module directory (e.g. /uts/objects), validates its structure, resolves the target ably-java module, lets you pick a tier (unit/integration/proxy) and which specs, then derives a Kotlin test per spec. Usage: /uts-to-kotlin " +description: "Translate the UTS pseudocode test specs in a whole module directory into runnable Kotlin tests in the owning ably-java module (:core for realtime/rest, :liveobjects for objects; :uts hosts the shared infra + smoke examples). Takes a UTS module directory (e.g. /uts/objects), validates its structure, resolves the target ably-java module, lets you pick a tier (unit/integration/proxy) and which specs, then derives a Kotlin test per spec. Usage: /uts-to-kotlin " allowed-tools: Bash, Read, Edit, Write, WebFetch --- Translate the UTS pseudocode test specs under the **module directory** `$ARGUMENTS` into runnable Kotlin -tests in the owning ably-java module (`:java` for realtime/rest, `:liveobjects` for objects; `:uts` hosts +tests in the owning ably-java module (`:core` for realtime/rest, `:liveobjects` for objects; `:uts` hosts the shared infra + smoke examples). `$ARGUMENTS` is a UTS *module* directory — a directory sitting directly under the spec repo's `uts/`, @@ -49,7 +49,7 @@ It prints one JSON object. **If `ok` is `false`, relay `message` to the user and `NOT_A_UTS_MODULE_PATH` (not a `.../uts/` directory), `DIR_NOT_FOUND`, `NO_TIER_DIRS` (no `unit/` or `integration/`). On success it gives `sourceModule`, `mapped`, `translationNotes`, and a `tiers` object with one entry per tier (`unit` / `integration` / `proxy`), each carrying `present`, -`sourceDir`, `targetDir`, `package`, `module` (the owning Gradle module — `:java` / `:liveobjects` / +`sourceDir`, `targetDir`, `package`, `module` (the owning Gradle module — `:core` / `:liveobjects` / `:uts`), and `specs` (a list of `{file, className}`). Everything downstream reads from this output — treat it as the single source of truth and don't recompute paths or names by hand. @@ -66,10 +66,10 @@ don't always match (e.g. `objects` → `liveobjects`), which is why it's explici Each tier's mapping value is **one repo-root-relative path** (never a machine-absolute `/Users/...` path). The resolver derives everything from it: `targetDir` is the path, `package` is the path after `src/test/kotlin/` with `/` → `.`, and `module` is the owning Gradle module from the path's first segment -(`lib/` → `:java`, `liveobjects/` → `:liveobjects`, `uts/` → `:uts`). Use the resolver's `targetDir` / +(`lib/` → `:core`, `liveobjects/` → `:liveobjects`, `uts/` → `:uts`). Use the resolver's `targetDir` / `package` / `module` output directly — don't recompute. The `objects` entry is **hand-maintained**: its tiers live in `:liveobjects`'s own test source set (its specs assert on `:liveobjects` internals only -visible to that module's own tests) under `liveobjects/src/test/kotlin/io/ably/lib/liveobjects/uts/{unit, +visible to that module's own tests) under `liveobjects/src/test/kotlin/io/ably/pubsub/liveobjects/uts/{unit, integration,proxy}` — a sibling layout that differs from the `--create` template below, so `--create` never regenerates it. @@ -84,10 +84,10 @@ regenerates it. python3 .claude/skills/uts-to-kotlin/scripts/resolve_uts.py "" --create ``` - This adds full `lib/`-rooted (`:java`) paths under `packages` — - `lib/src/test/kotlin/io/ably/lib/uts/unit/`, `.../integration/standard/`, and + This adds full `lib/`-rooted (`:core`) paths under `packages` — + `lib/src/test/kotlin/io/ably/pubsub/uts/unit/`, `.../integration/standard/`, and `.../integration/proxy/` — and re-prints the resolved output. **`--create` only scaffolds - `:java`-hosted modules;** a module whose tiers live in another Gradle module (like `objects` → + `:core`-hosted modules;** a module whose tiers live in another Gradle module (like `objects` → `:liveobjects`, a sibling `uts/{unit,integration,proxy}` layout) still needs a hand-edit afterwards. (`` must be a simple module base name — letters, digits, underscore; the script returns `BAD_TARGET_NAME` otherwise, so just ask again.) @@ -181,12 +181,12 @@ integration tests** section; **proxy** → the proxy subsections of the **Integr > multi-point-per-method shape. Infrastructure lives in `:uts`'s **main** source set — -`uts/src/main/kotlin/io/ably/lib/uts/infra/` — so other Gradle modules can consume it via -`testImplementation(project(":uts"))`. Kotlin packages are `io.ably.lib.uts.infra.*`, so imports in +`uts/src/main/kotlin/io/ably/pubsub/uts/infra/` — so other Gradle modules can consume it via +`testImplementation(project(":uts"))`. Kotlin packages are `io.ably.pubsub.uts.infra.*`, so imports in generated tests are unaffected. It is split by tier: -- `infra/Utils.kt` — shared async helpers (`awaitState`, `awaitChannelState`, `pollUntil`), package `io.ably.lib.uts.infra`. -- `infra/unit/` — unit-test mocks/factories (`ClientFactories.kt`, `MockWebSocket.kt`, `MockHttpClient.kt`, `FakeClock.kt`, `MockEvent.kt`, the `PendingConnection`/`PendingRequest` pairs, and `Utils.kt` with the `ConnectionDetails { }` builder), package `io.ably.lib.uts.infra.unit`. +- `infra/Utils.kt` — shared async helpers (`awaitState`, `awaitChannelState`, `pollUntil`), package `io.ably.pubsub.uts.infra`. +- `infra/unit/` — unit-test mocks/factories (`ClientFactories.kt`, `MockWebSocket.kt`, `MockHttpClient.kt`, `FakeClock.kt`, `MockEvent.kt`, the `PendingConnection`/`PendingRequest` pairs, and `Utils.kt` with the `ConnectionDetails { }` builder), package `io.ably.pubsub.uts.infra.unit`. - `infra/integration/` + `infra/integration/proxy/` — direct-sandbox + proxy helpers (`SandboxApp.kt`, `ProxyManager.kt`, `ProxySession.kt`) — see the **Integration tests** section. For a **unit** test, read all files under `infra/unit/` plus `infra/Utils.kt` before generating any code (you need exact method signatures). @@ -194,12 +194,12 @@ For a **unit** test, read all files under `infra/unit/` plus `infra/Utils.kt` be **Module-local helpers.** Every module's tiers live in that module's own test source set (the resolver's `targetDir` / `module`), so also read any module-local helpers alongside the target — they sit in the tier's `targetDir`. For **objects/unit** (module `:liveobjects`) that's -`liveobjects/src/test/kotlin/io/ably/lib/liveobjects/uts/unit/Helpers.kt` (package -`io.ably.lib.liveobjects.uts.unit`) — `setupSyncedChannel`, the `build_*` message builders (typed `Wire*` +`liveobjects/src/test/kotlin/io/ably/pubsub/liveobjects/uts/unit/Helpers.kt` (package +`io.ably.pubsub.liveobjects.uts.unit`) — `setupSyncedChannel`, the `build_*` message builders (typed `Wire*` constructions, no JSON/reflection), `STANDARD_POOL_OBJECTS` and the canonical serial constants. Access convention for that suite: **public-tier specs use only the public API + helpers**; only the five internal-graph specs (`internal_live_counter`, `internal_live_map`, `object_id`, `objects_pool`, -`parent_references`) and documented deviations may reference `io.ably.lib.liveobjects` `internal` members — +`parent_references`) and documented deviations may reference `io.ably.pubsub.liveobjects` `internal` members — their symbol map is `references/objects-mapping.md` §17. ## Step 4 — Generate the Kotlin test file @@ -208,11 +208,11 @@ Apply the translation rules below, then write the file. ### Client construction -Use `TestRealtimeClient { }` and `TestRestClient { }` from `io.ably.lib`. These are DSL builders that extend `DebugOptions` — all `ClientOptions` fields (`autoConnect`, `recover`, `disconnectedRetryTimeout`, etc.) are settable directly inside the block. The default API key is `"appId.keyId:keySecret"`. +Use `TestRealtimeClient { }` and `TestHttpClient { }` from `io.ably.pubsub.uts.infra.unit`. These are DSL builders that extend `DebugOptions` — all `ClientOptions` fields (`autoConnect`, `recover`, `disconnectedRetryTimeout`, etc.) are settable directly inside the block. The default API key is `"appId.keyId:keySecret"`. | Pseudocode | Kotlin | |---|---| -| `Rest(options: ClientOptions(key: "..."))` | `TestRestClient { key = "..." }` | +| `Rest(options: ClientOptions(key: "..."))` (the HTTP client) | `TestHttpClient { key = "..." }` | | `Realtime(options: ClientOptions(key: "...", autoConnect: false))` | `TestRealtimeClient { key = "..."; autoConnect = false }` | | `Realtime(options: ClientOptions(autoConnect: false))` | `TestRealtimeClient { autoConnect = false }` | | Installing mocks | `install(mock)` inside the builder block | @@ -270,7 +270,7 @@ val refuseJob = launch { For HTTP: ```kotlin val mockHttp = MockHttpClient { onRequest = { req -> req.respondWith(200, body) } } -val client = TestRestClient { install(mockHttp) } +val client = TestHttpClient { install(mockHttp) } ``` ### Inspecting outgoing frames @@ -393,22 +393,22 @@ This scaffold is for the **unit** tier — it wires the mocked transport (`infra **Proxy integration tests** section instead (`SandboxApp` / `ProxySession` wiring), not from this template. The `package` is always the resolver's `package` for the chosen tier (Step 2), and the -`io.ably.lib.uts.infra.*` imports stay valid whatever module the tier lands in. For **objects/unit** the -same scaffold applies with the resolver's `io.ably.lib.liveobjects.uts.unit` package, and channel/objects +`io.ably.pubsub.uts.infra.*` imports stay valid whatever module the tier lands in. For **objects/unit** the +same scaffold applies with the resolver's `io.ably.pubsub.liveobjects.uts.unit` package, and channel/objects setup goes through the module-local helpers (`setupSyncedChannel` etc. from the same package — see Step 3) rather than raw `MockWebSocket` wiring. ```kotlin package // the resolver's package for the chosen tier (Step 2) -import io.ably.lib.uts.infra.unit.* // TestRealtimeClient/TestRestClient, MockWebSocket, MockHttpClient, FakeClock, CONNECTED_MESSAGE, ConnectionDetails { } builder -import io.ably.lib.uts.infra.awaitState -import io.ably.lib.uts.infra.awaitChannelState // if testing channels -import io.ably.lib.uts.infra.pollUntil // if polling on a predicate -import io.ably.lib.realtime.ChannelState // if testing channels -import io.ably.lib.realtime.ConnectionState -import io.ably.lib.types.ErrorInfo -import io.ably.lib.types.ProtocolMessage +import io.ably.pubsub.uts.infra.unit.* // TestRealtimeClient/TestHttpClient, MockWebSocket, MockHttpClient, FakeClock, CONNECTED_MESSAGE, ConnectionDetails { } builder +import io.ably.pubsub.uts.infra.awaitState +import io.ably.pubsub.uts.infra.awaitChannelState // if testing channels +import io.ably.pubsub.uts.infra.pollUntil // if polling on a predicate +import io.ably.pubsub.realtime.ChannelState // if testing channels +import io.ably.pubsub.realtime.ConnectionState +import io.ably.pubsub.types.ErrorInfo +import io.ably.pubsub.types.ProtocolMessage import kotlinx.coroutines.launch import kotlinx.coroutines.test.runTest import kotlin.test.* @@ -455,7 +455,7 @@ class { Compile the module the resolver reports for the chosen tier (its `module` field — Step A): ```bash -./gradlew :java:compileTestKotlin # realtime / rest tiers (module :java) +./gradlew :core:compileTestKotlin # realtime / rest tiers (module :core) ./gradlew :liveobjects:compileTestKotlin # objects tiers (module :liveobjects) ``` @@ -482,16 +482,16 @@ Use the per-tier task of the module the resolver reports for the tier (its `modu resolver's `package` + the spec's `className` for the `--tests` filter: ```bash -# realtime / rest unit (module :java) → io.ably.lib.uts.unit.* -./gradlew :java:runUtsUnitTests --tests "." +# realtime / rest unit (module :core) → io.ably.pubsub.uts.unit.* +./gradlew :core:runUtsUnitTests --tests "." -# realtime / rest integration + proxy (module :java) → io.ably.lib.uts.integration.* -./gradlew :java:runUtsIntegrationTests --tests "." +# realtime / rest integration + proxy (module :core) → io.ably.pubsub.uts.integration.* +./gradlew :core:runUtsIntegrationTests --tests "." -# objects unit (module :liveobjects) → io.ably.lib.liveobjects.uts.unit.* +# objects unit (module :liveobjects) → io.ably.pubsub.liveobjects.uts.unit.* ./gradlew :liveobjects:runLiveObjectsUnitTests --tests "." -# objects integration + proxy (module :liveobjects) → io.ably.lib.liveobjects.uts.{integration,proxy}.* +# objects integration + proxy (module :liveobjects) → io.ably.pubsub.liveobjects.uts.{integration,proxy}.* ./gradlew :liveobjects:runLiveObjectsIntegrationTests --tests "." ``` @@ -557,8 +557,8 @@ fun `RTLC7c2 - LOCAL source does not write siteTimeserials`() = runTest { ### Deviations file Append to the deviations file that belongs to the tier's module: -`lib/src/test/kotlin/io/ably/lib/uts/deviations.md` for realtime/rest tiers (module `:java`), or -`liveobjects/src/test/kotlin/io/ably/lib/liveobjects/uts/deviations.md` for objects/unit. Use the manual's +`lib/src/test/kotlin/io/ably/pubsub/uts/deviations.md` for realtime/rest tiers (module `:core`), or +`liveobjects/src/test/kotlin/io/ably/pubsub/liveobjects/uts/deviations.md` for objects/unit. Use the manual's **Recording deviations** entry format and sections. The ably-java-specific mapping: a **UTS Spec Error** (test fails fast — fix in the spec) goes under the manual's *UTS Spec Errors* section; an **SDK deviation** (env-gated/adapted — fix in the SDK) goes under *Failing Tests* / *Adapted Tests*. @@ -645,7 +645,7 @@ For each test case, verify: Deviations are discovered by running, so this check applies in evaluate mode. For any place where the generated test diverges from the spec pseudocode (adapted assertion, env-gated skip, or omitted step): - [ ] A `// DEVIATION:` comment explains why -- [ ] The deviation is recorded in the tier's module deviations file (see "Deviations file" above: `lib/src/test/kotlin/io/ably/lib/uts/deviations.md` for realtime/rest tiers, `liveobjects/src/test/kotlin/io/ably/lib/liveobjects/uts/deviations.md` for objects tiers) +- [ ] The deviation is recorded in the tier's module deviations file (see "Deviations file" above: `lib/src/test/kotlin/io/ably/pubsub/uts/deviations.md` for realtime/rest tiers, `liveobjects/src/test/kotlin/io/ably/pubsub/liveobjects/uts/deviations.md` for objects tiers) If you find gaps during this review, fix them, then **re-run the audit script** until `missingInKotlin` / `orphanInKotlin` are empty and every `perTest` entry reconciles, and re-run Step 5 (compile) — and, in @@ -679,7 +679,7 @@ A **direct-sandbox** spec (no `create_proxy_session`, no rules — just happy-pa **Client wiring** — point the client at the sandbox endpoint: ```kotlin -private fun newClient(useBinaryProtocol: Boolean): AblyRealtime = TestRealtimeClient { +private fun newClient(useBinaryProtocol: Boolean): PubSubRealtimeClient = TestRealtimeClient { key = app.defaultKey endpoint = SandboxApp.sandboxEndpoint // nonprod:sandbox this.useBinaryProtocol = useBinaryProtocol @@ -723,13 +723,13 @@ Use generous timeouts (10–30s) — real network is involved. Everything else i ### Infrastructure -Three helpers live under `uts/src/main/kotlin/io/ably/lib/uts/infra/integration/`. **Read the ones your tier uses before translating an integration spec** — they hold the exact method signatures. `SandboxApp` serves **both** tiers; `ProxyManager` and `ProxySession` are **proxy-only**. +Three helpers live under `uts/src/main/kotlin/io/ably/pubsub/uts/infra/integration/`. **Read the ones your tier uses before translating an integration spec** — they hold the exact method signatures. `SandboxApp` serves **both** tiers; `ProxyManager` and `ProxySession` are **proxy-only**. -- **`ProxyManager`** (`infra/integration/proxy/ProxyManager.kt`, package `io.ably.lib.uts.infra.integration.proxy`) — downloads/starts the shared `uts-proxy` process. Call `ProxyManager.ensureProxy()` once per suite in setup. +- **`ProxyManager`** (`infra/integration/proxy/ProxyManager.kt`, package `io.ably.pubsub.uts.infra.integration.proxy`) — downloads/starts the shared `uts-proxy` process. Call `ProxyManager.ensureProxy()` once per suite in setup. - **`ProxySession`** (`infra/integration/proxy/ProxySession.kt`, same package) — one programmable session wrapping the proxy control API; also defines the `connectThroughProxy` extension and the rule-builder helpers. -- **`SandboxApp`** (`infra/integration/SandboxApp.kt`, package `io.ably.lib.uts.infra.integration`) — provisions/deletes a sandbox test app from the shared `test-app-setup.json` in ably-common. `SandboxApp.create()` returns a `SandboxApp` with `appId`, `defaultKey`, and `keys` (`defaultKey` is a full-capability `appId.keyId:keySecret`); `app.delete()` tears it down. Provision in suite setup, delete in teardown. Also owns `SandboxApp.sandboxEndpoint` (`nonprod:sandbox`), which direct-sandbox clients set as their `endpoint`, and `SandboxApp.sandboxHost` (`sandbox.realtime.ably-nonprod.net`, the hostname that endpoint resolves to) — the default upstream target of every `ProxySession`. +- **`SandboxApp`** (`infra/integration/SandboxApp.kt`, package `io.ably.pubsub.uts.infra.integration`) — provisions/deletes a sandbox test app from the shared `test-app-setup.json` in ably-common. `SandboxApp.create()` returns a `SandboxApp` with `appId`, `defaultKey`, and `keys` (`defaultKey` is a full-capability `appId.keyId:keySecret`); `app.delete()` tears it down. Provision in suite setup, delete in teardown. Also owns `SandboxApp.sandboxEndpoint` (`nonprod:sandbox`), which direct-sandbox clients set as their `endpoint`, and `SandboxApp.sandboxHost` (`sandbox.realtime.ably-nonprod.net`, the hostname that endpoint resolves to) — the default upstream target of every `ProxySession`. -Import what the tier needs: a **direct-sandbox** test imports `io.ably.lib.uts.infra.integration.SandboxApp` plus `io.ably.lib.uts.infra.unit.TestRealtimeClient` and `io.ably.lib.uts.infra.{awaitState, pollUntil}`; a **proxy** test additionally imports `io.ably.lib.uts.infra.integration.proxy.{ProxyManager, ProxySession, connectThroughProxy}`. +Import what the tier needs: a **direct-sandbox** test imports `io.ably.pubsub.uts.infra.integration.SandboxApp` plus `io.ably.pubsub.uts.infra.unit.TestRealtimeClient` and `io.ably.pubsub.uts.infra.{awaitState, pollUntil}`; a **proxy** test additionally imports `io.ably.pubsub.uts.infra.integration.proxy.{ProxyManager, ProxySession, connectThroughProxy}`. `ensureProxy()`, the `ProxySession` methods, and the `SandboxApp` methods are all **`suspend`** functions. Per-test bodies use `runTest { }`; JUnit5 `@BeforeAll`/`@AfterAll` (with `@TestInstance(Lifecycle.PER_CLASS)`) wrap their suspend calls in `runBlocking { }`. @@ -807,14 +807,14 @@ val client = TestRealtimeClient { | `tls: false` | `tls = false` | | `useBinaryProtocol: false` | already the `ClientOptionsBuilder` default — left untouched | -It does **not** touch `autoConnect`, so set that yourself. Setting explicit hosts disables fallback hosts automatically, so don't add `fallbackHosts`. +It does **not** touch `autoConnect`, so set that yourself. A hostname endpoint has no default fallback hosts (REC2c2), so don't add `fallbackHosts`. ### Auth in proxy tests A spec that needs to observe (re-)authentication uses an `authCallback`. Where the pseudocode "generates a JWT from the key parts", the idiomatic ably-java equivalent is a **locally-signed `TokenRequest`** from the same sandbox key — no JWT library required. The realtime client exchanges it for a token through the proxy: ```kotlin -val tokenSigner = AblyRest(app.defaultKey) // local signing only; no network +val tokenSigner = TestHttpClient { key = app.defaultKey } // local signing only; no network val authCallbackCount = AtomicInteger(0) val authCallback = Auth.TokenCallback { params -> authCallbackCount.incrementAndGet() @@ -827,7 +827,7 @@ val client = TestRealtimeClient { } ``` -`TokenParams`/`TokenRequest` are nested in `io.ably.lib.rest.Auth`; `AuthDetails` is nested in `io.ably.lib.types.ProtocolMessage`. +`TokenParams`/`TokenRequest` are nested in `io.ably.pubsub.http.Auth`; `AuthDetails` is nested in `io.ably.pubsub.types.ProtocolMessage`. ### Rule factory helpers @@ -873,6 +873,6 @@ assertNotNull(queryParams["resume"]) 3. Each test creates its own `ProxySession` and closes it (and the client) in `finally`. 4. Use `awaitState` / `awaitChannelState` for state assertions; verify via SDK state **and** the proxy log where useful. 5. Use generous timeouts (10–30s) — real network is involved: `awaitState(client, ConnectionState.connected, 15.seconds)`. -6. Don't set `fallbackHosts`; explicit hosts already disable fallbacks. +6. Don't set `fallbackHosts`; a hostname endpoint already has no fallbacks (REC2c2). Step 5 (compile) still applies; Step 6 (run) applies only in evaluate mode (Step E). Note that proxy tests hit the live sandbox and download the proxy binary on first run, so they are slower and require network access. diff --git a/.claude/skills/uts-to-kotlin/references/objects-mapping.md b/.claude/skills/uts-to-kotlin/references/objects-mapping.md index c97b2a8f9..4826ce72b 100644 --- a/.claude/skills/uts-to-kotlin/references/objects-mapping.md +++ b/.claude/skills/uts-to-kotlin/references/objects-mapping.md @@ -41,12 +41,12 @@ different things**, and a third *internal* layer underneath. Keep them straight: | Layer | Spec name | ably-js | ably-java | Package | |---|---|---|---|---| -| **Creation value type** — immutable blueprint you pass *into* `set` | `LiveMap` / `LiveCounter` (the `RTLMV*` / `RTLCV*` classes) | `LiveMap.create()` / `LiveCounter.create()` | `LiveMap` / `LiveCounter` | `io.ably.lib.liveobjects.value` | -| **Public read/write view** — what you navigate & subscribe on | `PathObject`, `Instance` | `PathObject`, `Instance` | typed hierarchy (§4, §5) | base in `io.ably.lib.liveobjects.path` / `.instance`; **typed subtypes in `.path.types` / `.instance.types`** | -| **Internal graph object** — the live CRDT node | `InternalLiveMap` / `InternalLiveCounter` (`RTLM*` / `RTLC*`), `ObjectsPool` | internal | `InternalLiveMap` / `InternalLiveCounter` / `ObjectsPool` — **same names as the spec** (`internal` classes, `:liveobjects` module) | `io.ably.lib.liveobjects[.value.livemap/.value.livecounter]` — not public; access rules §13, full symbol map §17 | +| **Creation value type** — immutable blueprint you pass *into* `set` | `LiveMap` / `LiveCounter` (the `RTLMV*` / `RTLCV*` classes) | `LiveMap.create()` / `LiveCounter.create()` | `LiveMap` / `LiveCounter` | `io.ably.pubsub.liveobjects.value` | +| **Public read/write view** — what you navigate & subscribe on | `PathObject`, `Instance` | `PathObject`, `Instance` | typed hierarchy (§4, §5) | base in `io.ably.pubsub.liveobjects.path` / `.instance`; **typed subtypes in `.path.types` / `.instance.types`** | +| **Internal graph object** — the live CRDT node | `InternalLiveMap` / `InternalLiveCounter` (`RTLM*` / `RTLC*`), `ObjectsPool` | internal | `InternalLiveMap` / `InternalLiveCounter` / `ObjectsPool` — **same names as the spec** (`internal` classes, `:liveobjects` module) | `io.ably.pubsub.liveobjects[.value.livemap/.value.livecounter]` — not public; access rules §13, full symbol map §17 | So when a spec says `counter = LiveCounter.create(5)` and passes it to `set`, that's the **value type** -(`io.ably.lib.liveobjects.value.LiveCounter`). When a spec says "the resolved value is an +(`io.ably.pubsub.liveobjects.value.LiveCounter`). When a spec says "the resolved value is an `InternalLiveCounter` with `.data == 5`", that's the **internal graph node** (§13). When a spec navigates `root.get("counter").value()`, that's the **public view** (`PathObject`). @@ -97,10 +97,10 @@ In the spec/ably-js a `PathObject` is polymorphic: `get`, `at`, `value`, `set`, all hang off the one object. In ably-java the base `PathObject` exposes **only** the type-agnostic methods; everything type-specific is moved onto a sub-interface you reach via an `as*` cast. -**Base `PathObject`** (`io.ably.lib.liveobjects.path.PathObject`) — always available. The typed sub-types +**Base `PathObject`** (`io.ably.pubsub.liveobjects.path.PathObject`) — always available. The typed sub-types returned by the `as*` casts (`LiveMapPathObject`, `LiveCounterPathObject`, `NumberPathObject`, `StringPathObject`, `BooleanPathObject`, `BinaryPathObject`, `JsonObjectPathObject`, `JsonArrayPathObject`) -live in **`io.ably.lib.liveobjects.path.types`** (not `.path`) — import them from there. +live in **`io.ably.pubsub.liveobjects.path.types`** (not `.path`) — import them from there. | Spec / ably-js | ably-java | |---|---| @@ -207,7 +207,7 @@ Same partition as `PathObject`, with two differences: the base is **abstract / n directly (`RTTS7e`), and the casts **throw** on mismatch instead of degrading (`RTTS9d`) — because an `Instance` wraps an already-resolved value of known type. -**Base `Instance`** (`io.ably.lib.liveobjects.instance.Instance`): +**Base `Instance`** (`io.ably.pubsub.liveobjects.instance.Instance`): | Spec / ably-js | ably-java | |---|---| @@ -254,7 +254,7 @@ immutable creation value types. | Spec / ably-js | ably-java | |---|---| -| `LiveCounter.create()` | `LiveCounter.create(): LiveCounter` (`io.ably.lib.liveobjects.value`) | +| `LiveCounter.create()` | `LiveCounter.create(): LiveCounter` (`io.ably.pubsub.liveobjects.value`) | | `LiveCounter.create(5)` | `LiveCounter.create(5)` (arg is `Number`) | | `LiveMap.create()` | `LiveMap.create(): LiveMap` | | `LiveMap.create({ a: 1, b: "x" })` | `LiveMap.create(mapOf("a" to LiveMapValue.of(1), "b" to LiveMapValue.of("x")))` — entries are `Map` | @@ -370,13 +370,13 @@ depth (`RTPO19c1`). | remove all | `` channel.`object`.offAll() `` | Listener: `ObjectStateChange.Listener.onStateChanged(ObjectStateEvent)`. Enum `ObjectStateEvent { SYNCING, -SYNCED }` (`io.ably.lib.liveobjects.state`). +SYNCED }` (`io.ably.pubsub.liveobjects.state`). --- ## 10. `ValueType` & type discrimination -ably-js uses string-literal type tags; ably-java has an enum `io.ably.lib.liveobjects.ValueType`: +ably-js uses string-literal type tags; ably-java has an enum `io.ably.pubsub.liveobjects.ValueType`: | Spec value category | `ValueType` | |---|---| @@ -394,7 +394,7 @@ non-null. Use `getType()` for "what is it" assertions and the matching `as*` cas The spec's `PublicAPI::ObjectMessage` / `PublicAPI::ObjectOperation` (the `PAOM*` / `PAOOP*` types, delivered to subscription listeners) map to ably-java interfaces with getters (package -`io.ably.lib.liveobjects.message`). The `PublicAPI::` prefix is dropped — ably-java exposes them as +`io.ably.pubsub.liveobjects.message`). The `PublicAPI::` prefix is dropped — ably-java exposes them as `ObjectMessage` / `ObjectOperation`. > **Getter-only, no *public* constructor — use the `buildPublicObjectMessage` helper.** In normal use you @@ -520,10 +520,10 @@ Several **unit** specs assert on the **internal CRDT graph**, not the public API that is otherwise `internal`. Build the source with the op builders and assert the public getters (§11). In ably-java these are **not public**. They live in the `:liveobjects` module as `Internal*` / `Default*` / `Wire*` / -`ResolvedValue` / `Leaf` / `MapRef` / `CounterRef` classes (package `io.ably.lib.liveobjects.*`, +`ResolvedValue` / `Leaf` / `MapRef` / `CounterRef` classes (package `io.ably.pubsub.liveobjects.*`, implementation source set), visible only to that module's own tests. Because of this, **all objects tiers translate into `:liveobjects`'s own test source set** — -`liveobjects/src/test/kotlin/io/ably/lib/liveobjects/uts/{unit,integration,proxy}/` — so the unit tier +`liveobjects/src/test/kotlin/io/ably/pubsub/liveobjects/uts/{unit,integration,proxy}/` — so the unit tier sees module internals directly, and the integration/proxy tiers reach the plugin through the same module. Consequences when translating: @@ -548,8 +548,8 @@ Consequences when translating: Every objects unit spec opens with `setup_synced_channel` and constructs protocol/object messages with the `build_*` helpers. These are implemented in -`liveobjects/src/test/kotlin/io/ably/lib/liveobjects/uts/unit/Helpers.kt` (package -`io.ably.lib.liveobjects.uts.unit`; transport bootstrap via the shared `io.ably.lib.uts.infra.*` infra helpers, +`liveobjects/src/test/kotlin/io/ably/pubsub/liveobjects/uts/unit/Helpers.kt` (package +`io.ably.pubsub.liveobjects.uts.unit`; transport bootstrap via the shared `io.ably.pubsub.uts.infra.*` infra helpers, message construction via the typed `Wire*` DTOs — §17.10) — **call them; don't hand-roll the mock setup or message wire forms.** @@ -599,8 +599,8 @@ wire `action` / `semantics` are integer enum codes — the builders emit the cod Some objects **integration** specs (tier `integration/standard`) seed object state over REST *before* the realtime client connects, via the spec's `## REST Fixture Provisioning` helper `provision_objects_via_rest`. Its ably-java translation lives in -`liveobjects/src/test/kotlin/io/ably/lib/liveobjects/uts/integration/Helpers.kt` (package -`io.ably.lib.liveobjects.uts.integration`) — **call it; don't hand-roll the REST request or payload +`liveobjects/src/test/kotlin/io/ably/pubsub/liveobjects/uts/integration/Helpers.kt` (package +`io.ably.pubsub.liveobjects.uts.integration`) — **call it; don't hand-roll the REST request or payload JSON.** (Currently only `objects/integration/RTPO15` uses it.) It lives inside `:liveobjects`'s own tests now (like the whole objects suite), but still uses only the public `AblyRest` — the "public API only for REST provisioning" convention survives even though the module's internals are visible. @@ -711,8 +711,8 @@ This section maps the **5 internal-graph unit specs** — `internal_live_counter internal CRDT state, so they cannot translate against the public API — background in `JAVA_LIVEOBJECTS_INTERNAL_METHODS_ACCESS_REPORT.md` at the repo root) — onto the internal engine. Like the rest of the objects suite, they go into -**`liveobjects/src/test/kotlin/io/ably/lib/liveobjects/uts/unit/`** (package -`io.ably.lib.liveobjects.uts.unit`, task `:liveobjects:runLiveObjectsUnitTests`), where the test +**`liveobjects/src/test/kotlin/io/ably/pubsub/liveobjects/uts/unit/`** (package +`io.ably.pubsub.liveobjects.uts.unit`, task `:liveobjects:runLiveObjectsUnitTests`), where the test compilation is associated with `main`, so every `internal` declaration is directly visible — no reflection. @@ -726,7 +726,7 @@ reflection. The internal classes have `private constructor`s; each pairs with an `internal` companion factory that needs a `DefaultRealtimeObject`. Build one from the mocked adapter (helper already exists in -`liveobjects/src/test/kotlin/io/ably/lib/liveobjects/unit/TestHelpers.kt`): +`liveobjects/src/test/kotlin/io/ably/pubsub/liveobjects/unit/TestHelpers.kt`): ```kotlin val ro = DefaultRealtimeObject("test", getMockAblyClientAdapter()) diff --git a/.claude/skills/uts-to-kotlin/scripts/resolve_uts.py b/.claude/skills/uts-to-kotlin/scripts/resolve_uts.py index 773f839bf..ff9de0c36 100644 --- a/.claude/skills/uts-to-kotlin/scripts/resolve_uts.py +++ b/.claude/skills/uts-to-kotlin/scripts/resolve_uts.py @@ -34,10 +34,10 @@ SKILL_DIR = Path(__file__).resolve().parent.parent MAPPING = SKILL_DIR / "uts-package-mapping.json" TIERS = ("unit", "integration", "proxy") -# Owning Gradle module, keyed by a target dir's first path segment. The `lib` -> `:java` -# pair is the load-bearing non-obvious mapping (the `:java` module's build file wires +# Owning Gradle module, keyed by a target dir's first path segment. The `lib` -> `:core` +# pair is the load-bearing non-obvious mapping (the `:core` module's build file wires # `../lib/src/...` srcDirs). -MODULE_BY_PREFIX = {"lib": ":java", "liveobjects": ":liveobjects", "uts": ":uts"} +MODULE_BY_PREFIX = {"lib": ":core", "liveobjects": ":liveobjects", "uts": ":uts"} def fail(code, message): @@ -95,7 +95,7 @@ def main(): "--create", metavar="NAME", help="add a mapping for this source module using NAME as the ably-java " - "module base name, then resolve. Scaffolds full lib/-rooted (:java) paths " + "module base name, then resolve. Scaffolds full lib/-rooted (:core) paths " "only — a module whose tiers live in another Gradle module (like objects " "-> :liveobjects) still needs a hand-edit afterwards.", ) @@ -131,10 +131,10 @@ def main(): f"--create target {target!r} must be a simple module base name " f"(letters/digits/underscore, e.g. 'liveobjects') so it forms a " f"valid path and Kotlin package.") - # Full repo-root-relative paths using the realtime/`lib` (:java) template. + # Full repo-root-relative paths using the realtime/`lib` (:core) template. # A module whose tiers live in another Gradle module (like objects -> - # :liveobjects) still needs a hand-edit — --create only scaffolds :java-hosted modules. - base = "lib/src/test/kotlin/io/ably/lib/uts" + # :liveobjects) still needs a hand-edit — --create only scaffolds :core-hosted modules. + base = "lib/src/test/kotlin/io/ably/pubsub/uts" new_entry = { "unit": f"{base}/unit/{target}", "integration": f"{base}/integration/standard/{target}", diff --git a/.claude/skills/uts-to-kotlin/uts-package-mapping.json b/.claude/skills/uts-to-kotlin/uts-package-mapping.json index f334189e1..62ca0a1d5 100644 --- a/.claude/skills/uts-to-kotlin/uts-package-mapping.json +++ b/.claude/skills/uts-to-kotlin/uts-package-mapping.json @@ -1,21 +1,21 @@ { - "_comment": "Maps each UTS spec module (a dir under specification/uts/) to its target test directory per tier. Each tier value is ONE repo-root-relative path (never machine-absolute); the Kotlin package is the path after 'src/test/kotlin/' with '/' -> '.'; the owning Gradle module is the path's first segment (lib/ -> :java, liveobjects/ -> :liveobjects, uts/ -> :uts). Every tier path MUST keep a module segment after the tier (e.g. 'unit/realtime', never bare 'unit') so a derived package can never collide with a :uts smoke package (io.ably.lib.uts.unit / .integration.standard / .integration.proxy) — an invariant currently held only by construction. An optional 'notes' field points (relative to this skill dir) to a per-module ably-js -> ably-java translation reference, read before translating that module. Used by the uts-to-kotlin skill (scripts/resolve_uts.py).", + "_comment": "Maps each UTS spec module (a dir under specification/uts/) to its target test directory per tier. Each tier value is ONE repo-root-relative path (never machine-absolute); the Kotlin package is the path after 'src/test/kotlin/' with '/' -> '.'; the owning Gradle module is the path's first segment (lib/ -> :core, liveobjects/ -> :liveobjects, uts/ -> :uts). Every tier path MUST keep a module segment after the tier (e.g. 'unit/realtime', never bare 'unit') so a derived package can never collide with a :uts smoke package (io.ably.pubsub.uts.unit / .integration.standard / .integration.proxy) — an invariant currently held only by construction. An optional 'notes' field points (relative to this skill dir) to a per-module ably-js -> ably-java translation reference, read before translating that module. Used by the uts-to-kotlin skill (scripts/resolve_uts.py).", "packages": { "realtime": { - "unit": "lib/src/test/kotlin/io/ably/lib/uts/unit/realtime", - "integration": "lib/src/test/kotlin/io/ably/lib/uts/integration/standard/realtime", - "proxy": "lib/src/test/kotlin/io/ably/lib/uts/integration/proxy/realtime" + "unit": "lib/src/test/kotlin/io/ably/pubsub/uts/unit/realtime", + "integration": "lib/src/test/kotlin/io/ably/pubsub/uts/integration/standard/realtime", + "proxy": "lib/src/test/kotlin/io/ably/pubsub/uts/integration/proxy/realtime" }, "objects": { - "unit": "liveobjects/src/test/kotlin/io/ably/lib/liveobjects/uts/unit", - "integration": "liveobjects/src/test/kotlin/io/ably/lib/liveobjects/uts/integration", - "proxy": "liveobjects/src/test/kotlin/io/ably/lib/liveobjects/uts/proxy", + "unit": "liveobjects/src/test/kotlin/io/ably/pubsub/liveobjects/uts/unit", + "integration": "liveobjects/src/test/kotlin/io/ably/pubsub/liveobjects/uts/integration", + "proxy": "liveobjects/src/test/kotlin/io/ably/pubsub/liveobjects/uts/proxy", "notes": "references/objects-mapping.md" }, "rest": { - "unit": "lib/src/test/kotlin/io/ably/lib/uts/unit/rest", - "integration": "lib/src/test/kotlin/io/ably/lib/uts/integration/standard/rest", - "proxy": "lib/src/test/kotlin/io/ably/lib/uts/integration/proxy/rest" + "unit": "lib/src/test/kotlin/io/ably/pubsub/uts/unit/rest", + "integration": "lib/src/test/kotlin/io/ably/pubsub/uts/integration/standard/rest", + "proxy": "lib/src/test/kotlin/io/ably/pubsub/uts/integration/proxy/rest" } } } diff --git a/.github/workflows/check.yml b/.github/workflows/check.yml index 7c8e819ab..b5f518d6d 100644 --- a/.github/workflows/check.yml +++ b/.github/workflows/check.yml @@ -28,9 +28,12 @@ jobs: # A second UTS leg through the server door's builders (see the uts.side handling in # uts/.../ClientFactories.kt): the builders stamp a side-declaring agent entry and pass # everything else through, so conformance must be identical on both legs; SideModesTest - # fails a leg whose stamp does not match. There is no device leg on the JVM — the device - # door is an Android artifact, covered by the instrumentation tests in emulate.yml. - - run: ./gradlew :uts:runUtsUnitTests -Duts.side=server + # fails a leg whose stamp does not match. The leg covers every module whose UTS specs + # build clients through the factories: :core and :liveobjects (plus the :uts self-tests). + # There is no device leg on the JVM — the device door is not wired into the factories' seam; + # its stamping contract is covered by the device module's own tests (jvmTest and the + # instrumentation tests in emulate.yml). + - run: ./gradlew :core:runUtsUnitTests :uts:runUtsUnitTests :liveobjects:runLiveObjectsUnitTests -Duts.side=server # Continuously proves the release pre-flight and that every published module # builds a publishable artifact set, so version/coordinate regressions surface diff --git a/.github/workflows/emulate.yml b/.github/workflows/emulate.yml index c7bfe302f..e69086878 100644 --- a/.github/workflows/emulate.yml +++ b/.github/workflows/emulate.yml @@ -14,6 +14,8 @@ jobs: strategy: fail-fast: false matrix: + # core-android declares minSdk 19, but API 19-23 (and the HttpURLConnection network engine + # used there, -PhttpURLConnection) are not covered by this emulator matrix. android-api-level: [ 24, 29, 35 ] steps: @@ -50,7 +52,7 @@ jobs: arch: ${{ steps.get-avd-arch.outputs.arch }} target: default # Print emulator logs if tests fail - script: ./gradlew :core-android:connectedAndroidTest :device:connectedAndroidTest ${{ matrix.android-api-level == 19 && '-PhttpURLConnection' || '' }} || (adb logcat -d System.out:I && exit 1) + script: ./gradlew :core-android:connectedAndroidTest :device:connectedAndroidTest || (adb logcat -d System.out:I && exit 1) - uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4 if: always() diff --git a/.github/workflows/integration-test.yml b/.github/workflows/integration-test.yml index 348bbe0f3..40f4934f3 100644 --- a/.github/workflows/integration-test.yml +++ b/.github/workflows/integration-test.yml @@ -146,5 +146,6 @@ jobs: - run: ./gradlew :core:runUtsIntegrationTests :uts:runUtsIntegrationTests # A second leg through the server door's builders — see the uts.side handling in - # uts/.../ClientFactories.kt and the matching leg in check.yml. - - run: ./gradlew :uts:runUtsIntegrationTests -Duts.side=server + # uts/.../ClientFactories.kt and the matching leg in check.yml. The :liveobjects run is + # narrowed to its UTS specs (the module's own integration tests do not use the factories). + - run: ./gradlew :core:runUtsIntegrationTests :uts:runUtsIntegrationTests :liveobjects:runLiveObjectsIntegrationTests --tests 'io.ably.pubsub.liveobjects.uts.*' -Duts.side=server diff --git a/CHANGELOG.md b/CHANGELOG.md index 1ed0dd270..62dc5294f 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,24 @@ # Change Log +## [2.0.0] (unreleased) + +See [UPGRADING.md](./UPGRADING.md) for the full 1.x to 2.0 migration guide. + +### Breaking changes + +- The SDK is split into one artifact per side of the connection: `io.ably.pubsub:server` for backends and `io.ably.pubsub:device` for end-user devices (Android and JVM). Clients are built through `PubSubServer` or `PubSubDevice`; the `AblyRealtime` and `AblyRest` constructors are no longer public. Use exactly one door per app and never depend on `io.ably.pubsub:core` or `core-android` directly. +- Coordinates and packages move from `io.ably:*` to `io.ably.pubsub:*` and from `io.ably.lib.*` to `io.ably.pubsub.*`. `AblyRealtime` and `AblyRest` become `PubSubRealtimeClient` and `PubSubHttpClient`. +- REST-named types become HTTP-named (`io.ably.pubsub.http.*`, `HttpRequester`, `PubSubBase`, `HttpAnnotations`). +- Deprecated 1.x API is removed, including the `CompletionListener` publish overloads, `Auth.authorise` and `Auth.renew`, `fromCipherKey`, `Connection.recoveryKey`, `fromJSON` and `RegistrationToken.Type.GCM`. +- `ClientOptions.environment`, `restHost`, `realtimeHost` and `fallbackHostsUseDefault` are replaced by `endpoint` (REC1, REC2c). Default hosts move to `main.realtime.ably.net`. +- `Channels.release()` throws a checked `AblyException` (90011, status 400) unless the channel is `INITIALIZED`, `DETACHED` or `FAILED` (RTS4c-e). +- The `Ably-Agent` header and realtime `agent` parameter start with `ably-pubsub-java/` instead of `ably-java/`, with a side entry (`ably-pubsub-server` or `ably-pubsub-device`). +- Server clients using token authentication need a token carrying the signed `x-ably-clientType=server` claim (today a JWT), otherwise the connection is rejected with 40167. +- The device artifact requires Android API level 24 (1.x supported API level 19). +- Door `build()` methods pass a shallow copy of the options to the client (maps and arrays are copied, nested objects such as `proxy` or callbacks are shared). +- Android push registration token types are persisted by name. A stored FCM token is migrated; a stored GCM token is discarded and the device registers with FCM again. +- Server-initiated re-authentication emits `UPDATE` once `renewAuth` completes. + ## [1.8.2](https://github.com/ably/ably-java/tree/v1.8.2) [Full Changelog](https://github.com/ably/ably-java/compare/v1.8.1...v1.8.2) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index d873e68f2..eb6cf51e8 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -21,6 +21,10 @@ but on Windows there is a batch file: gradlew.bat +Applications consume the SDK through `io.ably.pubsub:server` or `io.ably.pubsub:device`; `core` and +`core-android` are internal implementation artifacts. If you are changing public behaviour, update +[UPGRADING.md](./UPGRADING.md) too. + The JRE-specific library JAR is built with: ./gradlew core:jar @@ -33,18 +37,18 @@ The Android-specific library AAR is built with: ## Adding a New Network Engine Implementation -Currently, `ably-java` supports two different engines for network operations (HTTP calls and WebSocket connections): +Currently, `ably-pubsub-java` supports two different engines for network operations (HTTP calls and WebSocket connections): -- **Default Engine**: Utilizes the built-in `HttpUrlConnection` for HTTP calls and the TooTallNate/Java-WebSocket library for WebSocket connections. -- **OkHttp Engine**: Utilizes the OkHttp library for both HTTP and WebSocket connections. +- **OkHttp Engine** (`network-client-okhttp`): Utilizes the OkHttp library for both HTTP and WebSocket connections. This is what the published `core` and `core-android` artifacts pull in. +- **Default Engine** (`network-client-default`): Utilizes the built-in `HttpUrlConnection` for HTTP calls and the TooTallNate/Java-WebSocket library for WebSocket connections. Build with `-PhttpURLConnection` to use it in this repository. -These engines are designed to be swappable. By default, the library comes with the default engine, but you can easily replace it with the OkHttp engine: +These engines are designed to be swappable. For an application, swap the engine under the door you depend on, for example on a server: ```kotlin -implementation("io.ably:ably-java:$ABLY_VERSION") { - exclude(group = "io.ably", module = "network-client-default") +implementation("io.ably.pubsub:server:$ABLY_VERSION") { + exclude(group = "io.ably.pubsub", module = "network-client-okhttp") } -runtimeOnly("io.ably:network-client-okhttp:$ABLY_VERSION") +runtimeOnly("io.ably.pubsub:network-client-default:$ABLY_VERSION") ``` ### Publishing to local Maven repository for development and testing @@ -126,7 +130,7 @@ The gradle project files can be imported to create projects in IntelliJ IDEA, Ec The top-level ably-java project can be imported into IntelliJ IDEA, enabling development of both the core and core-android projects. This has been tested with IntelliJ IDEA Ultimate 2017.2. To import into IDEA: - do File->New->Project from Existing Sources... -- select ably-pubsub-java/settings.gradle +- select ably-pubsub-java/settings.gradle.kts - in the import dialog, check "Use auto-import" and uncheck "Create separate module per source set" - select "ok" @@ -160,13 +164,13 @@ Interactive run/debug configurations for the core project can be created as foll #### Importing into Android studio -Android studio does not include the components required to support development of the core project, it is not capable of importing the multi-level ably-java gradle project. It is possible to import the core-android project as a standalone project into Android Studio by deleting the top-level settings.gradle file, which effectively decouples the core-android and core projects. +Android studio does not include the components required to support development of the core project, it is not capable of importing the multi-level ably-java gradle project. It is possible to import the core-android project as a standalone project into Android Studio by deleting the top-level settings.gradle.kts file, which effectively decouples the core-android and core projects. This has been tested with Android Studio 3.0.1. To import into Android Studio: - do Import project (Gradle, Eclipse ADT, etc); -- select ably-pubsub-java/core-android/build.gradle; +- select ably-pubsub-java/core-android/build.gradle.kts; - select OK to Gradle Sync. This creates a single core-android project and module. @@ -227,17 +231,17 @@ signing.secretKeyRingFile=/Users/username/.ably/ably-java-secring.gpg ``` - Run `./gradlew core-android:assembleRelease` or `./gradlew core-android:assembleDebug`. -## Using `ably-java` / `ably-android` locally in other projects +## Using `core-android` locally in other projects -You may wish to make changes to Ably Java or Ably Android, and test it immediately in a separate project. For example, during development for [Ably Flutter](https://github.com/ably/ably-flutter) which depends on `ably-android`, a bug was found in `ably-android`. A small fix was done, the AAR was built and tested in [Ably Flutter](https://github.com/ably/ably-flutter). +You may wish to make changes to Ably Java or Ably Android, and test it immediately in a separate project. For example, during development for [Ably Flutter](https://github.com/ably/ably-flutter), which depended on the 1.x `ably-android`, a bug was found in `ably-android`. A small fix was done, the AAR was built and tested in [Ably Flutter](https://github.com/ably/ably-flutter). - Build the AAR: See [Building an Android Archive (AAR) file locally](#building-an-android-archive-aar-file-locally) -- Open the directory printed from the output of that command. Inside that folder, get the `ably-android-x.y.z.aar`, and place it your Android project's `libs/` directory. Create this directory if it doesn't exist. +- Open the directory printed from the output of that command. Inside that folder, get the `core-android-release.aar` (or `core-android-debug.aar`; artifact `io.ably.pubsub:core-android`), and place it your Android project's `libs/` directory. Create this directory if it doesn't exist. - Add an `implementation` dependency on the `.aar`: ```groovy -implementation files('libs/ably-android-1.8.2.aar') +implementation files('libs/core-android-release.aar') ``` -- Add the `implementation` (not `testImplementation`) dependencies found in `dependencies.gradle` to your project. This is because the `.aar` does not contain dependencies. +- Add the `implementation` (not `testImplementation`) dependencies declared in `gradle/libs.versions.toml` and `core-android/build.gradle.kts` to your project. This is because the `.aar` does not contain dependencies. - Build/run your application. ## Release Process diff --git a/README.md b/README.md index 8e44c8ee2..bb104d5b4 100644 --- a/README.md +++ b/README.md @@ -1,224 +1,345 @@ ![Ably Pub/Sub Java Header](images/javaSDK-github.png) -[![Latest Version](https://img.shields.io/maven-central/v/io.ably/ably-java)](https://central.sonatype.com/artifact/io.ably/ably-java) -[![License](https://badgen.net/github/license/ably/ably-pubsub-java)](https://github.com/ably/ably-pubsub-java/blob/main/LICENSE) + +[![Maven Central: server](https://img.shields.io/maven-central/v/io.ably.pubsub/server?label=io.ably.pubsub%3Aserver)](https://central.sonatype.com/artifact/io.ably.pubsub/server) +[![Maven Central: device](https://img.shields.io/maven-central/v/io.ably.pubsub/device?label=io.ably.pubsub%3Adevice)](https://central.sonatype.com/artifact/io.ably.pubsub/device) +[![License](https://badgen.net/github/license/ably/ably-pubsub-java)](./LICENSE) # Ably Pub/Sub Java SDK -Build any realtime experience using Ably’s Pub/Sub Java SDK. Supported on all popular platforms and frameworks, including Kotlin and Android. +The Java and Kotlin SDK for [Ably Pub/Sub](https://ably.com/docs/basics): publish and subscribe to +channels, and use message history, presence and push notifications, from servers, Android apps and +JVM desktop clients. -Ably Pub/Sub provides flexible APIs that deliver features such as pub-sub messaging, message history, presence, and push notifications. Utilizing Ably’s realtime messaging platform, applications benefit from its highly performant, reliable, and scalable infrastructure. +## Overview -Find out more: +The SDK is published as two packages, one for each side of the connection. You choose the package +by where your code runs, and build your client through that package's entry point. -* [Ably Pub/Sub docs.](https://ably.com/docs/basics) -* [Ably Pub/Sub examples.](https://ably.com/examples?product=pubsub) +| Package | Use it for | Entry point | Platforms | +|---------|-----------|-------------|-----------| +| `io.ably.pubsub:server` | Servers and other trusted backend environments | `PubSubServer.httpClientBuilder()`, `PubSubServer.realtimeClientBuilder()` | JVM | +| `io.ably.pubsub:device` | Android apps and other end-user runtimes | `PubSubDevice.clientBuilder()` | Android, JVM | ---- +## Requirements -> [!NOTE] -> **This branch carries the in-development 2.0 device/server package split.** The SDK is being -> restructured into new artifacts that declare which side of the network they run on, so that -> traffic classifies correctly on MAU-priced accounts: -> -> | Artifact | For | Entry point | -> |----------|-----|-------------| -> | `io.ably.pubsub:device` (aar) | Devices: Android apps and other end-user runtimes | `PubSubDevice.clientBuilder(...)` | -> | `io.ably.pubsub:server` (jar) | Servers and other trusted backend environments | `PubSubServer.httpClientBuilder(...)` / `PubSubServer.realtimeClientBuilder(...)` | -> | `io.ably.pubsub:core`, `io.ably.pubsub:core-android` | Internal implementation artifacts — do not depend on these directly | — | -> -> Nothing from this branch is published yet. The `io.ably:ably-java` and `io.ably:ably-android` -> 1.x artifacts continue to work and will receive security and critical fixes from a maintenance -> branch for one year after the 2.0 release. The installation instructions below still describe 1.x. +| Platform | Requirement | +|----------|-------------| +| Java | 8 or later (the SDK targets Java 8 bytecode). | +| Android | Android 7.0 (API level 24) or later, for `io.ably.pubsub:device`. | +| Kotlin | Optional. The Java API is usable from Kotlin; the `device` package is built with Kotlin 2.1.10. | ---- +## Installation -## Getting started +Use exactly one of `server` or `device` in an app; see [One package per app](#one-package-per-app). -Everything you need to get started with Ably: +### Server -* [Getting started with Pub/Sub using Java.](https://ably.com/docs/getting-started/java) -* [SDK Setup for Java.](https://ably.com/docs/getting-started/setup?lang=java) +Gradle (Kotlin DSL): ---- +```kotlin +dependencies { + implementation("io.ably.pubsub:server:2.0.0") +} +``` -## Supported platforms +Gradle (Groovy): -Ably aims to support a wide range of platforms. If you experience any compatibility issues, open an issue in the repository or contact [Ably support](https://ably.com/support). +```groovy +dependencies { + implementation 'io.ably.pubsub:server:2.0.0' +} +``` + +Maven: + +```xml + + io.ably.pubsub + server + 2.0.0 + +``` -The following platforms are supported: +### Device -| Platform | Support | -|----------|---------| -| Java | >= 1.8 (JRE 8 or later) | -| Kotlin | All versions (>= 1.0 supported), but we recommend >= 1.8 for best compatibility. | -| Android | >=4.4 (API level 19) | +`device` is a Kotlin Multiplatform artifact, published as `device` (root), `device-android` and +`device-jvm`. A Gradle build declares only `io.ably.pubsub:device`; Gradle module metadata selects the +Android or JVM variant for you. -> [!IMPORTANT] -> SDK versions < 1.2.35 will be [deprecated](https://ably.com/docs/platform/deprecate/protocol-v1) from November 1, 2025. +Android, Gradle (Kotlin DSL): ---- +```kotlin +dependencies { + implementation("io.ably.pubsub:device:2.0.0") +} +``` -## Installation +Android, Gradle (Groovy): -The Java SDK is available as a [Maven dependency](https://mvnrepository.com/artifact/io.ably/ably-java). To get started with your project, install the package: +```groovy +dependencies { + implementation 'io.ably.pubsub:device:2.0.0' +} +``` -### Install for Maven: +JVM desktop client, Gradle (Kotlin DSL or Groovy; the coordinate is the same): + +```kotlin +dependencies { + implementation("io.ably.pubsub:device:2.0.0") +} +``` + +JVM desktop client, Maven. Maven cannot read Gradle module metadata, so name the JVM artifact directly: ```xml - io.ably - ably-java - 1.8.2 + io.ably.pubsub + device-jvm + 2.0.0 ``` -### Install for Gradle: +### One package per app -```gradle -implementation 'io.ably:ably-java:1.8.2' -implementation 'org.slf4j:slf4j-simple:2.0.7' -``` +- Depend on exactly one of `io.ably.pubsub:server` or `io.ably.pubsub:device` in each app. +- Never depend on `io.ably.pubsub:core` or `io.ably.pubsub:core-android`. They are internal + implementation artifacts, and declaring them next to a package puts duplicate classes on the + classpath. +- If one module of your system runs on a device and another on a server, give each module its own + package. + +### Migrating from 1.x + +Remove the 1.x `io.ably:ably-java` and `io.ably:ably-android` dependencies when you add a 2.0 +package. See [UPGRADING.md](./UPGRADING.md) for the artifact mapping, package renames, behaviour +changes and removed APIs. + +## Quick start -Run the following to instantiate a client: +### Server + +Use an API key on a server. The following code connects, subscribes to a channel and publishes a +message to it. ```java +import io.ably.pubsub.realtime.Channel; +import io.ably.pubsub.realtime.ConnectionEvent; import io.ably.pubsub.realtime.PubSubRealtimeClient; import io.ably.pubsub.server.PubSubServer; +import io.ably.pubsub.types.AblyException; -PubSubRealtimeClient realtime = PubSubServer.realtimeClientBuilder() - .key(apiKey) - .build(); +public class ServerQuickStart { + public static void main(String[] args) throws AblyException { + PubSubRealtimeClient realtime = PubSubServer.realtimeClientBuilder() + .key("your-ably-api-key") + .build(); + + realtime.connection.on(ConnectionEvent.connected, change -> { + try { + Channel channel = realtime.channels.get("test-channel"); + channel.subscribe(message -> System.out.println("Received: " + message.data)); + channel.publish("test-event", "hello world"); + } catch (AblyException e) { + e.printStackTrace(); + } + }); + } +} ``` -On an end-user device, depend on `io.ably.pubsub:device` and build through -`PubSubDevice.clientBuilder()` instead; the builder you call declares which side of the -connection your code runs on. +To publish over HTTP without a realtime connection, build an HTTP client instead: ---- +```java +import io.ably.pubsub.http.PubSubHttpClient; +import io.ably.pubsub.server.PubSubServer; +import io.ably.pubsub.types.AblyException; -## Usage +public class ServerPublish { + public static void main(String[] args) throws AblyException { + PubSubHttpClient http = PubSubServer.httpClientBuilder() + .key("your-ably-api-key") + .build(); -The following code connects to Ably's realtime messaging service, subscribes to a channel to receive messages, and publishes a test message to that same channel. + http.channels.get("test-channel").publish("test-event", "hello world"); + } +} +``` +> [!IMPORTANT] +> Clients from `PubSubServer` declare themselves as servers. Authenticate them with an API key. +> Token authentication on a realtime connection works only with a token that carries the signed +> `x-ably-clientType=server` claim, which today means a JWT; otherwise Ably rejects the connection +> with error 40167. See [Server clients and token authentication](./UPGRADING.md#server-clients-and-token-authentication). -```java -// Initialize Ably Realtime client -PubSubRealtimeClient realtimeClient = PubSubServer.realtimeClientBuilder() - .key("your-ably-api-key") - .clientId("me") - .build(); +### Device -// Wait for connection to be established -realtimeClient.connection.on(ConnectionEvent.connected, connectionStateChange -> { - System.out.println("Connected to Ably"); - - // Get a reference to the 'test-channel' channel - Channel channel = realtimeClient.channels.get("test-channel"); - - // Subscribe to all messages published to this channel - channel.subscribe(message -> { - System.out.println("Received message: " + message.data); - }); - - // Publish a test message to the channel - channel.publish("test-event", "hello world"); -}); -``` ---- +Authenticate a device with tokens issued by your own server, through `authUrl` or `authCallback`. +Never embed an API key in an app that runs on an end-user device. -## Live Objects +```java +import io.ably.pubsub.device.PubSubDevice; +import io.ably.pubsub.realtime.Channel; +import io.ably.pubsub.realtime.PubSubRealtimeClient; +import io.ably.pubsub.types.AblyException; -[Ably Live Objects](https://ably.com/docs/liveobjects) provide realtime, collaborative data structures that automatically synchronize state across all connected clients. Build interactive applications with shared data that updates instantly across devices. +public class DeviceQuickStart { + public static void main(String[] args) throws AblyException { + PubSubRealtimeClient realtime = PubSubDevice.clientBuilder() + .authUrl("https://your-server.example.com/ably-auth") + .clientId("me") + .build(); -For a comprehensive guide, check the [Ably Live Objects documentation](https://ably.com/docs/liveobjects), starting with the [Java quickstart](https://ably.com/docs/liveobjects/quickstart/java). + // The client connects automatically. Subscribing attaches the channel. + Channel channel = realtime.channels.get("test-channel"); + channel.subscribe(message -> System.out.println("Received: " + message.data)); + channel.publish("test-event", "hello world"); + } +} +``` -### Install Live Objects +The same in Kotlin: -Add the following dependency to your `build.gradle` file: +```kotlin +import io.ably.pubsub.device.PubSubDevice -```groovy -dependencies { - runtimeOnly("io.ably:liveobjects:1.8.2") -} +val realtime = PubSubDevice.clientBuilder() + .authUrl("https://your-server.example.com/ably-auth") + .clientId("me") + .build() + +val channel = realtime.channels.get("test-channel") +channel.subscribe { message -> println("Received: ${message.data}") } +channel.publish("test-event", "hello world") ``` -### Example app +## Common configuration + +Both entry points accept the same options through their builders, or through a `ClientOptions` +object passed to `PubSubServer.realtimeClientBuilder(options)`, `PubSubServer.httpClientBuilder(options)` +or `PubSubDevice.clientBuilder(options)`. -The Ably Live Objects [Example app](./examples) is an interactive demo showcasing Live Objects with realtime color voting and collaborative task management. +### Endpoint + +`endpoint` selects the Ably host for both REST requests and the realtime connection. It replaces +the 1.x `environment`, `restHost` and `realtimeHost` options. + +```java +PubSubRealtimeClient realtime = PubSubServer.realtimeClientBuilder() + .key("your-ably-api-key") + .endpoint("nonprod:sandbox") + .build(); +``` -The example app demonstrates: -- **Color Voting**: Realtime voting system with live vote counts synchronized across all devices -- **Task Management**: Collaborative task management where users can add, edit, and delete tasks that sync in realtime +Never put a port in `endpoint`. Set `port` or `tlsPort` on `ClientOptions` instead. See +[Choose the host with `endpoint`](./UPGRADING.md#6-choose-the-host-with-endpoint) for how each value resolves. -To run the example app, follow the setup instructions in the [examples README](./examples/README.md). +### Network client -## Proxy support +By default, the published packages use OkHttp for HTTP and WebSocket connections. To use the +built-in `HttpURLConnection` engine instead, swap the engine under the package you depend on. For +example, on a server: -You can add proxy support to the Ably Java SDK by configuring `ProxyOptions` in your client setup, enabling connectivity through corporate firewalls and secured networks. +```kotlin +dependencies { + implementation("io.ably.pubsub:server:2.0.0") { + exclude(group = "io.ably.pubsub", module = "network-client-okhttp") + } + runtimeOnly("io.ably.pubsub:network-client-default:2.0.0") +} +``` -
-Proxy support setup details. +See [CONTRIBUTING.md](./CONTRIBUTING.md) for details of the engines. -Enable proxy support by specifying proxy settings in the ClientOptions when initializing your Ably client. +### Proxy -The following example sets up a proxy using the Pub/Sub Java SDK: +Pass a `ProxyOptions` object to the builder to connect through a proxy. ```java +import io.ably.pubsub.http.HttpAuth; import io.ably.pubsub.realtime.PubSubRealtimeClient; -import io.ably.pubsub.transport.Defaults; -import io.ably.pubsub.types.ClientOptions; +import io.ably.pubsub.server.PubSubServer; +import io.ably.pubsub.types.AblyException; import io.ably.pubsub.types.ProxyOptions; -import io.ably.pubsub.http.HttpAuth; -import io.ably.pubsub.http.PubSubHttpClient; public class AblyWithProxy { - public static void main(String[] args) throws Exception { - // Configure Ably Client options - ClientOptions options = new ClientOptions(); - - // Setup proxy settings + public static void main(String[] args) throws AblyException { ProxyOptions proxy = new ProxyOptions(); - proxy.host = "your-proxy-host"; // Replace with your proxy host - proxy.port = 8080; // Replace with your proxy port - - // Optional: If the proxy requires authentication - proxy.username = "your-username"; // Replace with proxy username - proxy.password = "your-password"; // Replace with proxy password - proxy.prefAuthType = HttpAuth.Type.BASIC; // Choose your preferred authentication type (e.g., BASIC or DIGEST) - - // Create an instance of Ably with the proxy settings attached - PubSubHttpClient ably = PubSubServer.httpClientBuilder() + proxy.host = "your-proxy-host"; + proxy.port = 8080; + + // Optional: only if the proxy requires authentication. + proxy.username = "your-username"; + proxy.password = "your-password"; + proxy.prefAuthType = HttpAuth.Type.BASIC; + + PubSubRealtimeClient realtime = PubSubServer.realtimeClientBuilder() .key("your-ably-api-key") .proxy(proxy) .build(); + } +} +``` + +### Logging + +Set `logLevel` to one of the constants in `io.ably.pubsub.util.Log` (`VERBOSE`, `DEBUG`, `INFO`, +`WARN`, `ERROR` or `NONE`; the default is `WARN`). Set `logHandler` to receive log output +yourself; by default it goes to standard output. - // Alternatively, for real-time connections - PubSubRealtimeClient realtimeClient = PubSubServer.realtimeClientBuilder() +```java +import io.ably.pubsub.realtime.PubSubRealtimeClient; +import io.ably.pubsub.server.PubSubServer; +import io.ably.pubsub.types.AblyException; +import io.ably.pubsub.util.Log; + +public class AblyWithLogging { + public static void main(String[] args) throws AblyException { + PubSubRealtimeClient realtime = PubSubServer.realtimeClientBuilder() .key("your-ably-api-key") - .proxy(proxy) + .logLevel(Log.DEBUG) + .logHandler((severity, tag, msg, tr) -> System.err.println(tag + ": " + msg)) .build(); - - // Use the Ably client as usual } } ``` -
+## Live Objects + +[Ably LiveObjects](https://ably.com/docs/liveobjects) provides realtime, collaborative data +structures that synchronize state across clients. It ships as a separate plugin, +`io.ably.pubsub:liveobjects`, that works with either package. Add it at runtime and the client +detects it on the classpath; no other setup is needed. + +```kotlin +dependencies { + runtimeOnly("io.ably.pubsub:liveobjects:2.0.0") +} +``` ---- +Attach the channel with the `object_subscribe` and `object_publish` modes to use objects on it. See +the [LiveObjects documentation](https://ably.com/docs/liveobjects) and the +[Java quickstart](https://ably.com/docs/liveobjects/quickstart/java), and run the +[example app](./examples/README.md) for a working demo. -## Contribute +## Further reading -Read the [CONTRIBUTING.md](./CONTRIBUTING.md) guidelines to contribute to Ably. +- [Getting started with Pub/Sub using Java](https://ably.com/docs/getting-started/java) +- [SDK setup for Java](https://ably.com/docs/getting-started/setup?lang=java) +- [Ably Pub/Sub documentation](https://ably.com/docs/basics) and [examples](https://ably.com/examples?product=pubsub) +- [UPGRADING.md](./UPGRADING.md): migrating from 1.x +- [CHANGELOG.md](./CHANGELOG.md): release notes; all Ably releases are listed on [changelog.ably.com](https://changelog.ably.com) +- [examples/](./examples): the LiveObjects example app ---- +## Contributing -## Releases +Read [CONTRIBUTING.md](./CONTRIBUTING.md) before contributing. -The [CHANGELOG.md](./CHANGELOG.md) contains details of the latest releases for this SDK. You can also view all Ably releases on [changelog.ably.com](https://changelog.ably.com). +## Support, feedback and troubleshooting ---- +For help, visit Ably's [support page](https://ably.com/support). Report bugs and compatibility +problems on [GitHub Issues](https://github.com/ably/ably-pubsub-java/issues). -## Support, feedback, and troubleshooting +## License -For help or technical support, visit Ably's [support page](https://ably.com/support) or [GitHub Issues](https://github.com/ably/ably-pubsub-java/issues) for community-reported bugs and discussions. +Licensed under the Apache License, Version 2.0. See [LICENSE](./LICENSE). diff --git a/UPGRADING.md b/UPGRADING.md index 88d0ee8c3..42c8a7877 100644 --- a/UPGRADING.md +++ b/UPGRADING.md @@ -13,6 +13,12 @@ runs on an end-user device or on infrastructure you control. | `io.ably.lib.rest.*` | `io.ably.pubsub.http.*` | | `AblyRealtime`, `AblyRest` | `PubSubRealtimeClient`, `PubSubHttpClient` | +Remove `io.ably:ably-java` and `io.ably:ably-android` (1.x) from your build when you add the new +artifact. Depend on exactly one door per app, and never on +`io.ably.pubsub:core` or `io.ably.pubsub:core-android` directly: they are internal implementation +artifacts. Read [Behaviour changes](#4-behaviour-changes) before upgrading, since several 1.x calls +behave differently at runtime. + ## 1. Choose the artifact for your side | Artifact | For | Entry point | @@ -24,6 +30,12 @@ Pick by where the code runs, not by which client you want: both doors give you a and connectionless operations (history, presence queries, token requests, `request()`, `batchPublish()`) are available on it either way. +Use exactly one of `io.ably.pubsub:server` or `io.ably.pubsub:device` in an app. On Android, `server` +brings `core` and `device` brings `core-android`, which contain the same `io.ably.pubsub` classes, so +declaring both, or declaring `core` or `core-android` yourself, puts duplicate classes on the +classpath; on the JVM both resolve to `core`. Either way, declaring both makes your side ambiguous. If one module of your app runs on +a device and another on a server, give each module its own door and keep them in separate artifacts. + Gradle: ```gradle @@ -93,14 +105,19 @@ PubSubHttpClient http = PubSubServer.httpClientBuilder() import io.ably.pubsub.device.PubSubDevice val client = PubSubDevice.clientBuilder() - .key("xVLyHw.MHOCLg:...") + .authUrl("https://your-server.example.com/ably-auth") .clientId("bob") .build() ``` -Note that the options object is handed to the client rather than copied, and `build()` replaces its -`agents` map with the stamped one; pass `ClientOptions.copy()` if either matters. Your own `agents` -entries are preserved — only the side entry is applied last and cannot be overridden. +Never embed an API key in an application that runs on an end-user device. Authenticate devices with +tokens issued by your own server, through `authUrl(...)` or `authCallback(...)`. + +The door's `build()` hands the client a copy of the options (`ClientOptions.copy()`). Reusing a builder, +or changing the `ClientOptions` you passed to `clientBuilder(options)` after `build()`, no longer +affects clients that were already built. Your own `agents` entries are preserved; the door's side +entry is applied to the copy and cannot be overridden, and a caller-supplied identifier for the +opposite side (for example `ably-pubsub-server` on a device client) is removed. `RealtimeClientFactory` and `HttpClientFactory` exist in the clients' own packages as the seam the doors construct through. They are not application API; use the builders. @@ -117,22 +134,117 @@ it into `io.ably.pubsub.http`: | `io.ably.lib.rest.Auth` | `io.ably.pubsub.http.Auth` | | `io.ably.lib.rest.DeviceDetails` | `io.ably.pubsub.http.DeviceDetails` | | `io.ably.lib.rest.RestAnnotations` | `io.ably.pubsub.http.HttpAnnotations` | +| `io.ably.lib.rest.AblyBase` | `io.ably.pubsub.http.PubSubBase` | +| `io.ably.lib.rest.ChannelBase`, `io.ably.lib.rest.MessageEditsMixin` | `io.ably.pubsub.http.ChannelBase`, `io.ably.pubsub.http.MessageEditsMixin` | +| `io.ably.lib.http.Http` | `io.ably.pubsub.http.HttpRequester` | | `io.ably.lib.http.*` (the rest) | `io.ably.pubsub.http.*` | +| `io.ably.lib.network.*` | `io.ably.pubsub.network.*` | +| `io.ably.lib.liveobjects.*` | `io.ably.pubsub.liveobjects.*` | +| `Defaults.HOST_REST`, `Defaults.HOST_REALTIME` | Removed. Use `endpoint` (`Defaults.ENDPOINT` is the default, `main`); see [Choose the host with `endpoint`](#6-choose-the-host-with-endpoint) | +| `Defaults.HOST_FALLBACKS` | Kept, but now holds the default endpoint's fallback domains, `main.[a-e].fallback.ably-realtime.com` | +| pubsub-adapter `com.ably.pubsub.RestClient`, `RestChannel`, `RestPresence` and `RestClient.createWrapperSdkProxy` | Removed with no HTTP counterpart. The adapter's `Client`, `Channel`, `Channels`, `Presence`, `RealtimeClient`, `RealtimeChannel` and `RealtimePresence` interfaces remain, and `RealtimeClient.createWrapperSdkProxy(WrapperSdkProxyOptions)` is kept. Use `PubSubHttpClient` directly for HTTP-only access. | | `io.ably.lib.types.*`, `io.ably.lib.realtime.*`, `io.ably.lib.push.*`, … | `io.ably.pubsub.types.*`, `io.ably.pubsub.realtime.*`, `io.ably.pubsub.push.*`, … | -For most codebases the import rewrite is mechanical. Rewrite the REST package first, since it is -the one that does not simply map by prefix: +`ClientOptions.idempotentRestPublishing` deliberately keeps its name, which is the specification's (TO3n). + +For most codebases the import rewrite is mechanical. Apply the specific renames first and the generic +package rules last, and keep the word boundaries so that identifiers merely containing `AblyRest` or +`AblyRealtime` are left alone. `perl` behaves the same on macOS and Linux (BSD `sed` has no `\b`): ```bash -# GNU sed; on macOS use `sed -i ''` -find src -name '*.java' -o -name '*.kt' | xargs sed -i \ - -e 's/io\.ably\.lib\.rest/io.ably.pubsub.http/g' \ - -e 's/io\.ably\.lib/io.ably.pubsub/g' \ - -e 's/AblyRealtime/PubSubRealtimeClient/g' \ - -e 's/AblyRest/PubSubHttpClient/g' +find src \( -name '*.java' -o -name '*.kt' \) -print0 | xargs -0 perl -pi -e ' + s/\bio\.ably\.lib\.http\.Http\b/io.ably.pubsub.http.HttpRequester/g; + s/\bAblyBase\b/PubSubBase/g; + s/\bRestAnnotations\b/HttpAnnotations/g; + s/\bio\.ably\.lib\.rest\b/io.ably.pubsub.http/g; + s/\bio\.ably\.lib\b/io.ably.pubsub/g; + s/\bAblyRealtime\b/PubSubRealtimeClient/g; + s/\bAblyRest\b/PubSubHttpClient/g; +' +``` + +The script renames only the fully qualified `io.ably.lib.http.Http`. Usages of the simple name `Http` +are left alone (a word-bounded rule would clobber unrelated `Http` identifiers), so rename them to +`HttpRequester` by hand. + +The script rewrites names only. You still need to replace `new AblyRealtime(...)` and `new AblyRest(...)` +with the door builders (see above), and review the [behaviour changes](#4-behaviour-changes) by hand. + +## 4. Behaviour changes + +### `Channels.release()` no longer detaches the channel (RTS4c-e) + +A channel can be released only when it is `INITIALIZED`, `DETACHED` or `FAILED`. In any other state +(`ATTACHING`, `ATTACHED`, `DETACHING`, `SUSPENDED`) the realtime `Channels.release()` throws an +`AblyException` with code 90011 (status 400) and leaves the channel unchanged. Releasing a channel +that does not exist is a no-op. The method now declares the checked `AblyException`, so Java callers +must handle it. Detach first and wait for the detach to complete. Releasing a channel on the HTTP +client is unchanged. + +**Before (1.x):** + +```java +ably.channels.release("some-channel"); // detached, then removed ``` -## 4. Deprecated API removed +**After (2.0):** + +```java +Channel channel = realtime.channels.get("some-channel"); +channel.once(ChannelState.detached, stateChange -> { + try { + realtime.channels.release("some-channel"); + } catch (AblyException e) { + // 90011 if the channel is no longer in a releasable state + } +}); +channel.detach(); // throws AblyException: call it from a method that declares it, or catch it +``` + +Closing the client detaches its channels, and the client and its channels become collectable together, +so you do not need to release them one by one. + +### Requests identify as `ably-pubsub-java` + +The `Ably-Agent` header, and the `agent` query parameter of the realtime connection, now start with +`ably-pubsub-java/` instead of `ably-java/`, and carry a bare `ably-pubsub-server` +or `ably-pubsub-device` entry according to the door used. Update anything that matches on +`ably-java/`, such as proxy rules or log queries. Entries you add through `agents(...)` are kept. +The door adds its side entry to a copy of your options, removes the opposite side's entry, and the +side entry cannot be overridden. After `ably-pubsub-java/`, the additional entries +(yours and the side entry) are sent in alphabetical order, followed by the platform entry. + +### Server clients and token authentication + +Clients from `PubSubServer` declare themselves as servers. Authenticate them with an API key. Token +authentication on a realtime connection works only with a token that carries the signed +`x-ably-clientType=server` claim, which today means a JWT; otherwise Ably rejects the connection with +error 40167. Tokens without the signed claim (for example Ably native tokens, whether from `authCallback`, +`token` or `tokenDetails`) are rejected. Clients from `PubSubDevice` are not affected: on a device, authenticate with tokens issued by +your own server (`authCallback` or `authUrl`), never with an API key. + +### Options are copied when a client is built + +`build()` on each door passes a copy of the options to the client, so a builder can be reused and +the options handed to `clientBuilder(options)` can be reassigned afterwards without affecting clients +that were already built. The copy is shallow: the `headers`, `agents`, `fallbackHosts` and +`transportParams` maps and arrays are copied, but nested objects such as `proxy`, `authParams`, +callbacks and listeners are shared. See the end of [section 2](#2-build-clients-through-the-door-builders). + +### Other changes + +- **Re-authentication.** When Ably asks a connected client to re-authenticate, the client now emits + `UPDATE` once `renewAuth` completes, so the event reports the outcome of the re-authentication + instead of firing before it has finished. +- **Android push registration token.** The push registration token type is now persisted by name + rather than by enum ordinal. A token stored by an earlier 1.x release as FCM is migrated and kept. + A stored GCM token (GCM is no longer supported), or an unrecognised stored value, is discarded, and + the device registers with FCM again on next activation. +- **Log tags.** Log tags are class names, so they changed with the packages (for example + `io.ably.lib.rest.AblyRest` became `io.ably.pubsub.http.PubSubHttpClient`). Update logcat filters + and `LogHandler` matchers that match on tags. + +## 5. Deprecated API removed 2.0 drops every member 1.x had marked deprecated. @@ -142,19 +254,19 @@ find src -name '*.java' -o -name '*.kt' | xargs sed -i \ | `ChannelOptions.fromCipherKey(byte[])`, `fromCipherKey(String)` | `ChannelOptions.withCipherKey(…)` | | `Connection.recoveryKey` | `Connection.createRecoveryKey()` | | `Connection.emit/on/once(ConnectionState, …)` | The `ConnectionEvent`-keyed `EventEmitter` methods, e.g. `connection.on(ConnectionEvent.connected, listener)` | -| `Channel.publish(…, CompletionListener)`, `publishAsync(…, CompletionListener)` | The `Callback` overloads, e.g. `publish(name, data, callback)` | +| Realtime `Channel.publish(…, CompletionListener)` | The `Callback` overloads, e.g. `publish(name, data, callback)` | +| HTTP client `Channel.publishAsync(…, CompletionListener)` | `publishAsync(name, data, Callback)`; the HTTP channel has no `publish(…, callback)` overload | | `Channel.sync()` | Nothing. It was a no-op logging stub, intended only for internal testing per RTP19. | | `Auth.authorise(…)` | `Auth.authorize(…)` | | `Auth.renew()` | `Auth.renewAuth(…)`, which reports completion instead of returning early | | `TokenDetails.fromJSON(…)`, `TokenRequest.fromJSON(…)` | `fromJsonElement(…)` | -| `ClientOptions.fallbackHostsUseDefault` | Drop it. Default fallback hosts apply automatically; set `fallbackHosts` only for custom hosts. | +| `ClientOptions.fallbackHostsUseDefault` | Drop it. Default fallback hosts apply automatically, except with a hostname endpoint, where you must pass `fallbackHosts` explicitly (see [endpoint](#6-choose-the-host-with-endpoint)). | | `RegistrationToken.Type.GCM` (Android) | `RegistrationToken.Type.FCM` | -## 5. Choose the host with `endpoint` +## 6. Choose the host with `endpoint` -2.0 replaces `ClientOptions.environment`, `restHost` and `realtimeHost` with a single -`ClientOptions.endpoint` option, and the builder methods `environment(…)`, `restHost(…)` and -`realtimeHost(…)` with `endpoint(…)`. REST requests and the realtime connection now always use the +2.0 replaces (REC1) the `ClientOptions.environment`, `restHost` and `realtimeHost` fields with a single +`ClientOptions.endpoint` field; the door builders also offer `endpoint(…)`. REST requests and the realtime connection now always use the same host. | 1.x | 2.0 | @@ -164,10 +276,16 @@ same host. | `environment = "acme"` (dedicated cluster) | `endpoint = "acme"` | | `restHost` / `realtimeHost = "localhost"` | `endpoint = "localhost"` | | `restHost` and `realtimeHost` set to the same custom host | `endpoint = ""` | +| `restHost = h` with `fallbackHostsUseDefault = true` | `endpoint = h` plus an explicit `fallbackHosts` of `main.a.fallback.ably-realtime.com` to `main.e.fallback.ably-realtime.com` (the five `main.[a-e]` hosts). A hostname endpoint has no default fallback hosts (REC2c2). | +| `restHost` with a non-default `port` or `tlsPort` | `endpoint = ""` plus `port` / `tlsPort` | | `restHost` and `realtimeHost` set to different hosts | Not supported; use one host that serves both, or contact Ably | | Custom `fallbackHosts` | Unchanged | -How `endpoint` resolves: +Never put a port in `endpoint`. Set `port` or `tlsPort` instead. A value such as `localhost:8080` +is not recognised as a hostname (it contains no `.` or `::` and is not exactly `localhost`), so it is +treated as a routing policy name and resolves to `localhost:8080.realtime.ably.net`. + +How `endpoint` resolves (REC1, REC2c): | `endpoint` | Primary host | Default fallback hosts | |------------|--------------|------------------------| @@ -198,4 +316,5 @@ PubSubRealtimeClient realtime = PubSubServer.realtimeClientBuilder() **Firewalls and proxies.** 1.x connected to `rest.ably.io`, `realtime.ably.io` and `[a-e].ably-realtime.com` by default. If your network restricts outbound traffic, allow -`*.realtime.ably.net` and `*.fallback.ably-realtime.com` before upgrading. +`*.realtime.ably.net` and `*.fallback.ably-realtime.com` before upgrading. If you use a `nonprod:` +endpoint, also allow `*.realtime.ably-nonprod.net` and `*.fallback.ably-realtime-nonprod.com`. diff --git a/build.gradle.kts b/build.gradle.kts index 398076611..8d7c46fff 100644 --- a/build.gradle.kts +++ b/build.gradle.kts @@ -1,4 +1,6 @@ import com.vanniktech.maven.publish.MavenPublishBaseExtension +import org.gradle.api.publish.PublishingExtension +import org.gradle.api.publish.maven.MavenPublication // Top-level build file where you can add configuration options common to all sub-projects/modules. @@ -41,7 +43,12 @@ subprojects { val expectedReleaseArtifacts = sortedSetOf( "io.ably.pubsub:core:jar", "io.ably.pubsub:core-android:aar", + // `device` is Kotlin Multiplatform: `device` is the root (metadata) publication and the + // real artifacts are the `device-jvm` and `device-android` target publications below; + // the root entry is modelled as `jar` only because it has no packaging of its own. "io.ably.pubsub:device:jar", + "io.ably.pubsub:device-jvm:jar", + "io.ably.pubsub:device-android:aar", "io.ably.pubsub:server:jar", "io.ably.pubsub:liveobjects:jar", "io.ably.pubsub:pubsub-adapter:jar", @@ -68,6 +75,20 @@ tasks.register("verifyReleaseArtifacts") { } val group = p.findProperty("GROUP") actual.add("$group:$artifactId:$packaging") + // A Kotlin Multiplatform module (`device`) does not publish as a single jar: its + // POM_ARTIFACT_ID names only the root metadata publication (the one a plain + // dependency resolves through Gradle module metadata), and the real artifacts are + // the per-target publications (`device-jvm`, `device-android`). Record those from the + // module's actual publications so a dropped target fails the check. + if (p.pluginManager.hasPlugin("org.jetbrains.kotlin.multiplatform")) { + p.extensions.getByType().publications + .withType() + .filter { it.artifactId != artifactId } + .forEach { pub -> + val ext = if (pub.artifactId.endsWith("-android")) "aar" else "jar" + actual.add("$group:${pub.artifactId}:$ext") + } + } } if (actual != expectedReleaseArtifacts) { error( diff --git a/core-android/build.gradle.kts b/core-android/build.gradle.kts index e714135c5..b5a63d49f 100644 --- a/core-android/build.gradle.kts +++ b/core-android/build.gradle.kts @@ -8,7 +8,6 @@ android { defaultConfig { minSdk = 19 compileSdk = 34 - buildConfigField("String", "LIBRARY_NAME", "\"android\"") buildConfigField("String", "VERSION", "\"${property("VERSION_NAME")}\"") testInstrumentationRunner = "android.support.test.runner.AndroidJUnitRunner" testInstrumentationRunnerArguments["class"] = "io.ably.pubsub.test.android.AndroidPushTest" diff --git a/core-android/src/main/java/io/ably/pubsub/push/LocalDevice.java b/core-android/src/main/java/io/ably/pubsub/push/LocalDevice.java index 5a05bf3d5..d14683d50 100644 --- a/core-android/src/main/java/io/ably/pubsub/push/LocalDevice.java +++ b/core-android/src/main/java/io/ably/pubsub/push/LocalDevice.java @@ -64,8 +64,7 @@ private void loadPersisted() { this.clientId = storage.get(SharedPrefKeys.CLIENT_ID, null); this.deviceIdentityToken = storage.get(SharedPrefKeys.DEVICE_TOKEN, null); - RegistrationToken.Type type = RegistrationToken.Type.fromOrdinal( - storage.get(SharedPrefKeys.TOKEN_TYPE, -1)); + RegistrationToken.Type type = readPersistedTokenType(storage); Log.d(TAG, "loadPersisted(): token type = " + type); if(type != null) { @@ -79,6 +78,46 @@ private void loadPersisted() { } } + /** + * Reads the persisted registration token type. It is stored by name; releases that predate the + * removal of GCM stored the enum ordinal (an int) under the same key. A legacy FCM (1) is + * migrated to the name form, and anything else (legacy GCM (0), unknown ordinals, garbage) is + * dropped together with the stored token so that the device registers again. + */ + static RegistrationToken.Type readPersistedTokenType(Storage storage) { + String name = null; + Integer legacyOrdinal = null; + try { + name = storage.get(SharedPrefKeys.TOKEN_TYPE, (String) null); + } catch (ClassCastException e) { + try { + legacyOrdinal = storage.get(SharedPrefKeys.TOKEN_TYPE, -1); + } catch (ClassCastException e2) { + /* neither a string nor an int: treated as garbage below */ + } + } + + if (name == null && legacyOrdinal == null) { + return null; /* nothing persisted */ + } + + RegistrationToken.Type type = null; + if (name != null) { + type = RegistrationToken.Type.fromName(name); + } else if (legacyOrdinal == LEGACY_FCM_ORDINAL) { + type = RegistrationToken.Type.FCM; + storage.put(SharedPrefKeys.TOKEN_TYPE, type.toName()); + } + + if (type == null) { + storage.clear(new String[]{SharedPrefKeys.TOKEN_TYPE, SharedPrefKeys.TOKEN}); + } + return type; + } + + /** Ordinal of FCM in the {GCM, FCM} enum of releases that persisted the type by ordinal. */ + private static final int LEGACY_FCM_ORDINAL = 1; + RegistrationToken getRegistrationToken() { JsonObject recipient = push.recipient; if(recipient == null) { @@ -107,7 +146,7 @@ private void clearRegistrationToken() { void setAndPersistRegistrationToken(RegistrationToken token) { Log.v(TAG, "setAndPersistRegistrationToken(): token=" + token); setRegistrationToken(token); - storage.put(SharedPrefKeys.TOKEN_TYPE, token.type.ordinal()); + storage.put(SharedPrefKeys.TOKEN_TYPE, token.type.toName()); storage.put(SharedPrefKeys.TOKEN, token.token); } diff --git a/core-android/src/test/java/io/ably/pubsub/push/LocalDeviceTokenTypeMigrationTest.java b/core-android/src/test/java/io/ably/pubsub/push/LocalDeviceTokenTypeMigrationTest.java new file mode 100644 index 000000000..4e8d59181 --- /dev/null +++ b/core-android/src/test/java/io/ably/pubsub/push/LocalDeviceTokenTypeMigrationTest.java @@ -0,0 +1,97 @@ +package io.ably.pubsub.push; + +import io.ably.pubsub.types.RegistrationToken; +import org.junit.Test; + +import java.util.HashMap; +import java.util.Map; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertFalse; +import static org.junit.Assert.assertNull; + +public class LocalDeviceTokenTypeMigrationTest { + private static final String TOKEN_TYPE = "ABLY_REGISTRATION_TOKEN_TYPE"; + private static final String TOKEN = "ABLY_REGISTRATION_TOKEN"; + + /** Mimics SharedPreferences: reading a key with the wrong type throws ClassCastException. */ + private static class FakeStorage implements Storage { + final Map map = new HashMap<>(); + + @Override + public void put(String key, String value) { + map.put(key, value); + } + + @Override + public void put(String key, int value) { + map.put(key, value); + } + + @Override + public String get(String key, String defaultValue) { + Object v = map.get(key); + return v != null ? (String) v : defaultValue; + } + + @Override + public int get(String key, int defaultValue) { + Object v = map.get(key); + return v != null ? (Integer) v : defaultValue; + } + + @Override + public void clear(String[] keys) { + for (String key : keys) { + map.remove(key); + } + } + } + + @Test + public void nameRoundTrip() { + FakeStorage storage = new FakeStorage(); + storage.put(TOKEN_TYPE, RegistrationToken.Type.FCM.toName()); + assertEquals(RegistrationToken.Type.FCM, LocalDevice.readPersistedTokenType(storage)); + } + + @Test + public void nothingPersisted() { + assertNull(LocalDevice.readPersistedTokenType(new FakeStorage())); + } + + @Test + public void legacyFcmOrdinalIsKeptAndRewrittenByName() { + FakeStorage storage = new FakeStorage(); + storage.put(TOKEN_TYPE, 1); + storage.put(TOKEN, "t"); + assertEquals(RegistrationToken.Type.FCM, LocalDevice.readPersistedTokenType(storage)); + assertEquals("fcm", storage.map.get(TOKEN_TYPE)); + assertEquals("t", storage.map.get(TOKEN)); + } + + @Test + public void legacyGcmOrdinalIsDropped() { + FakeStorage storage = new FakeStorage(); + storage.put(TOKEN_TYPE, 0); + storage.put(TOKEN, "t"); + assertNull(LocalDevice.readPersistedTokenType(storage)); + assertFalse(storage.map.containsKey(TOKEN_TYPE)); + assertFalse(storage.map.containsKey(TOKEN)); + } + + @Test + public void garbageIsDropped() { + FakeStorage storage = new FakeStorage(); + storage.put(TOKEN_TYPE, 7); + storage.put(TOKEN, "t"); + assertNull(LocalDevice.readPersistedTokenType(storage)); + assertFalse(storage.map.containsKey(TOKEN)); + + storage.put(TOKEN_TYPE, "gcm"); + storage.put(TOKEN, "t"); + assertNull(LocalDevice.readPersistedTokenType(storage)); + assertFalse(storage.map.containsKey(TOKEN_TYPE)); + assertFalse(storage.map.containsKey(TOKEN)); + } +} diff --git a/core/build.gradle.kts b/core/build.gradle.kts index 9ec4947b5..b36bf2b90 100644 --- a/core/build.gradle.kts +++ b/core/build.gradle.kts @@ -38,7 +38,6 @@ dependencies { buildConfig { useJavaOutput() packageName = "io.ably.pubsub" - buildConfigField("String", "LIBRARY_NAME", "\"java\"") buildConfigField("String", "VERSION", "\"${property("VERSION_NAME")}\"") } @@ -123,11 +122,27 @@ tasks.register("runUnitTests") { outputs.upToDateWhen { false } } +/* +Which package's entry points the UTS specs construct clients through: `core` (default) or +`server` (the io.ably.pubsub:server builders). Gradle does not forward -D system properties to +the forked test JVM, so propagate it explicitly, accepting either `-Duts.side=server` on the +Gradle invocation or the `UTS_SIDE` environment variable. See ClientFactories.kt in :uts. +*/ +fun Test.forwardUtsSide() { + systemProperty( + "uts.side", + providers.systemProperty("uts.side") + .orElse(providers.environmentVariable("UTS_SIDE")) + .getOrElse("core"), + ) +} + tasks.register("runUtsUnitTests") { useJUnitPlatform() filter { includeTestsMatching("io.ably.pubsub.uts.unit.*") } + forwardUtsSide() jvmArgs("--add-opens", "java.base/java.time=ALL-UNNAMED") jvmArgs("--add-opens", "java.base/java.lang=ALL-UNNAMED") beforeTest(closureOf { logger.lifecycle("-> $this") }) @@ -139,6 +154,7 @@ tasks.register("runUtsIntegrationTests") { filter { includeTestsMatching("io.ably.pubsub.uts.integration.*") } + forwardUtsSide() jvmArgs("--add-opens", "java.base/java.time=ALL-UNNAMED") jvmArgs("--add-opens", "java.base/java.lang=ALL-UNNAMED") beforeTest(closureOf { logger.lifecycle("-> $this") }) diff --git a/core/gradle.properties b/core/gradle.properties index c315e84b2..c7758a2e0 100644 --- a/core/gradle.properties +++ b/core/gradle.properties @@ -1,6 +1,6 @@ POM_ARTIFACT_ID=core POM_NAME=Ably Pub/Sub core (JVM) -POM_DESCRIPTION=Internal implementation artifact for Ably's own Pub/Sub device/server packages. Not intended for direct use: depend on io.ably.pubsub:server (JVM) or io.ably.pubsub:device (Android) instead. +POM_DESCRIPTION=Internal implementation artifact for Ably's own Pub/Sub device/server packages. Not intended for direct use: depend on io.ably.pubsub:server (JVM servers) or io.ably.pubsub:device (Android and JVM end-user devices) instead. POM_PACKAGING=jar # The Kotlin plugin is applied for TEST-ONLY Kotlin sources; keep kotlin-stdlib out of :core's diff --git a/deploy/check-archives.sh b/deploy/check-archives.sh deleted file mode 100755 index 2a3d14c19..000000000 --- a/deploy/check-archives.sh +++ /dev/null @@ -1,78 +0,0 @@ -#!/bin/bash - -# Usage: CheckFileListExistance dir prefix filelist -CheckFileListExistance() { - for file in $3; do - local fileName=${file%.*} - local ext=${file##*.} - local filePath="${1}/${2}-${ABLY_VERSION}" - if [ "$fileName" != "" ]; then - filePath+="-${fileName}" - fi - filePath+=".${ext}" - if [ ! -f "${filePath}" -o ! -f "${filePath}.md5" -o ! -f "${filePath}.sha1" ]; then - echo "Required file or its hash not found: ${filePath}" - exit 1 - fi - if [ "${ext}" = "jar" -o "${ext}" = "aar" ]; then - if ! unzip -t ${filePath} >/dev/null 2>&1; then - echo "Corrupted archive: ${filePath}" - exit 1 - fi - fi - done -} - - -ABLY_HOME="${0%/*}/.." -ABLY_VERSION=`grep version "${ABLY_HOME}/build.gradle" | sed $'s/.*\'\(.*\)\'.*/\\\1/'` - -case "${ABLY_VERSION}" in - [0-9][0-9.]*|[0-9][0-9.]*-beta|[0-9][0-9.]*-pre) - ;; - *) - echo "Incorrect version: ${ABLY_VERSION}" - exit 1 - ;; -esac - -ABLY_JAVA_RELEASE_DIR="${ABLY_HOME}/core/build/release/${ABLY_VERSION}/io/ably/ably-java/${ABLY_VERSION}" -ABLY_ANDROID_RELEASE_DIR="${ABLY_HOME}/core-android/build/release/${ABLY_VERSION}/io/ably/ably-android/${ABLY_VERSION}" - -if [ ! -d "${ABLY_JAVA_RELEASE_DIR}" ]; then - echo "Directory for Java release ${ABLY_JAVA_RELEASE_DIR} doesn't exist" - exit 1 -fi - -if [ ! -d "${ABLY_ANDROID_RELEASE_DIR}" ]; then - echo "Directory for Android release ${ABLY_ANDROID_RELEASE_DIR} doesn't exist" - exit 1 -fi - -JAVA_FILES=".jar .pom sources.jar javadoc.jar" -ANDROID_FILES=".aar .pom sources.jar javadoc.jar" - -(CheckFileListExistance "${ABLY_JAVA_RELEASE_DIR}" "ably-java" "${JAVA_FILES}" && - CheckFileListExistance "${ABLY_ANDROID_RELEASE_DIR}" "ably-android" "${ANDROID_FILES}") || exit 1 - -ABLY_JAVA_JAR="${ABLY_JAVA_RELEASE_DIR}/ably-java-${ABLY_VERSION}.jar" -if [ `unzip -v ${ABLY_JAVA_JAR} | grep BuildConfig | wc -l` -ne 1 ]; then - echo "Java release failed BuildConfig test" - exit 1 -fi - -ABLY_ANDROID_AAR="${ABLY_ANDROID_RELEASE_DIR}/ably-android-${ABLY_VERSION}.aar" -if ! unzip -p "${ABLY_ANDROID_AAR}" classes.jar >/tmp/android-classes.jar 2>/dev/null; then - echo "There is no classes.jar in ${ABLY_ANDROID_AAR} file" - exit 1 -fi - -if [ `unzip -v /tmp/android-classes.jar | grep BuildConfig | wc -l` -ne 1 ]; then - echo "Android release failed BuildConfig test" - rm /tmp/android-classes.jar - exit 1 -fi -rm /tmp/android-classes.jar - -echo "All tests passed" - diff --git a/device/src/commonMain/kotlin/io/ably/pubsub/device/PubSubDevice.kt b/device/src/commonMain/kotlin/io/ably/pubsub/device/PubSubDevice.kt index 84d47ba71..6c3ffc954 100644 --- a/device/src/commonMain/kotlin/io/ably/pubsub/device/PubSubDevice.kt +++ b/device/src/commonMain/kotlin/io/ably/pubsub/device/PubSubDevice.kt @@ -25,9 +25,12 @@ import io.ably.pubsub.util.Side * history, presence queries, token requests, `request()` and `batchPublish()` — are all available on * the client this returns. * + * Authenticate with an auth endpoint you host (or an `authCallback`) rather than an API key: never + * embed an API key in an application that runs on an end-user device. + * * ``` * val client = PubSubDevice.clientBuilder() - * .key("xVLyHw.MHOCLg:...") + * .authUrl("https://example.com/ably-auth") * .clientId("bob") * .build() * ``` @@ -53,10 +56,10 @@ public object PubSubDevice { * options for it. * * The builder's setters still apply on top of what is passed here, and [ClientBuilder.build] - * stamps the side-declaring agent entry as usual. The options object is handed to the - * client rather than copied, so later changes to it are not isolated from the built client, - * and [ClientBuilder.build] replaces its `agents` map with the stamped one; pass - * [ClientOptions.copy] if either matters. + * stamps the side-declaring agent entry as usual. [ClientBuilder.build] snapshots the + * options ([ClientOptions.copy]), so neither later builder calls nor later changes to the + * object passed here affect a client already built, and the passed object's own `agents` + * map is never modified. * * @param options the options to start from. * @return a new builder over [options]. @@ -68,7 +71,7 @@ public object PubSubDevice { * Builds a device client. Obtain one from [PubSubDevice.clientBuilder]. */ public class ClientBuilder internal constructor( - /** Accumulates the calls made on this builder; handed to the client as-is by [build]. */ + /** Accumulates the calls made on this builder; [build] hands the client a snapshot of it. */ private val options: ClientOptions = ClientOptions(), ) { @@ -416,6 +419,6 @@ public object PubSubDevice { * authentication parameters were supplied. */ public fun build(): PubSubRealtimeClient = - RealtimeClientFactory.create(Side.injectSideAgent(options, Side.DEVICE_AGENT_IDENTIFIER)) + RealtimeClientFactory.create(Side.injectSideAgent(options.copy(), Side.DEVICE_AGENT_IDENTIFIER)) } } diff --git a/device/src/commonTest/kotlin/io/ably/pubsub/device/PubSubDeviceTest.kt b/device/src/commonTest/kotlin/io/ably/pubsub/device/PubSubDeviceTest.kt index 5c9718644..9d1c36e48 100644 --- a/device/src/commonTest/kotlin/io/ably/pubsub/device/PubSubDeviceTest.kt +++ b/device/src/commonTest/kotlin/io/ably/pubsub/device/PubSubDeviceTest.kt @@ -1,8 +1,18 @@ package io.ably.pubsub.device +import com.sun.net.httpserver.HttpServer +import io.ably.pubsub.types.AblyException +import io.ably.pubsub.types.ClientOptions import io.ably.pubsub.util.Side import org.junit.Assert import org.junit.Test +import java.net.InetAddress +import java.net.InetSocketAddress +import java.net.ServerSocket +import java.net.URLDecoder +import java.util.concurrent.CountDownLatch +import java.util.concurrent.TimeUnit +import java.util.concurrent.atomic.AtomicReference /** * The agent entries asserted here are what the platform reads to classify traffic on @@ -46,6 +56,144 @@ class PubSubDeviceTest { Assert.assertEquals("not-the-real-form", callerAgents.get(Side.DEVICE_AGENT_IDENTIFIER)) } + @Test + fun sideFlags_arePinnedToTheirWireLiterals() { + // Literal strings, not the constants: the platform classifies traffic by these exact tokens. + val agents = PubSubDevice.clientBuilder().key(FAKE_KEY).autoConnect(false).build().options.agents + Assert.assertTrue(agents.containsKey("ably-pubsub-device")) + Assert.assertFalse(agents.containsKey("ably-pubsub-server")) + } + + @Test + fun callerSuppliedServerEntry_isStripped() { + val callerAgents: Map = mapOf("ably-pubsub-server" to null, "some-sdk" to "1.2.3") + val agents = PubSubDevice.clientBuilder().key(FAKE_KEY).autoConnect(false) + .agents(callerAgents).build().options.agents + Assert.assertFalse(agents.containsKey("ably-pubsub-server")) + Assert.assertTrue(agents.containsKey("ably-pubsub-device")) + Assert.assertEquals("1.2.3", agents["some-sdk"]) + Assert.assertTrue("the caller's map is not modified", callerAgents.containsKey("ably-pubsub-server")) + } + + @Test + fun reusingABuilder_doesNotAffectAnEarlierClient() { + val builder = PubSubDevice.clientBuilder().key(FAKE_KEY).autoConnect(false) + val first = builder.build() + + builder.key("other.key:secret").agents(mapOf("later-sdk" to "9")).tls(false) + val second = builder.build() + + Assert.assertEquals(FAKE_KEY, first.options.key) + Assert.assertTrue(first.options.tls) + Assert.assertFalse(first.options.agents.containsKey("later-sdk")) + Assert.assertEquals("other.key:secret", second.options.key) + Assert.assertTrue(second.options.agents.containsKey("later-sdk")) + } + + @Test + fun changingPassedOptionsAfterBuild_doesNotAffectTheClient() { + val options = ClientOptions(FAKE_KEY) + options.autoConnect = false + val client = PubSubDevice.clientBuilder(options).build() + + options.clientId = "changed" + options.agents = HashMap() + + Assert.assertNotSame(options, client.options) + Assert.assertNull(client.options.clientId) + assertDeviceFlag(client.options.agents) + } + + @Test + fun noAuthParameters_getTheCoreConstructorsOwnError() { + try { + PubSubDevice.clientBuilder().build() + Assert.fail("expected the core's initialization error") + } catch (e: AblyException) { + Assert.assertEquals(40000, e.errorInfo.code) + } + } + + /** Wire-level: the Ably-Agent header actually sent over HTTP carries the device flag as a bare token. */ + @Test + fun httpRequests_carryTheDeviceAgentHeaderOnTheWire() { + val observed = AtomicReference() + val httpServer = HttpServer.create(InetSocketAddress("127.0.0.1", 0), 0) + httpServer.createContext("/time") { exchange -> + observed.set(exchange.requestHeaders.getFirst("Ably-Agent")) + val body = "[1234567890000]".toByteArray(Charsets.UTF_8) + exchange.responseHeaders.add("Content-Type", "application/json") + exchange.sendResponseHeaders(200, body.size.toLong()) + exchange.responseBody.write(body) + exchange.close() + } + httpServer.start() + try { + val client = PubSubDevice.clientBuilder() + .key(FAKE_KEY) + .autoConnect(false) + .tls(false) + .endpoint("127.0.0.1") + .port(httpServer.address.port) + .build() + client.time() + + val header = observed.get() + Assert.assertNotNull("no Ably-Agent header observed", header) + Assert.assertTrue("missing bare device flag in: $header", header!!.split(" ").contains("ably-pubsub-device")) + Assert.assertFalse("side flag must be versionless in: $header", header.contains("ably-pubsub-device/")) + Assert.assertFalse("must not carry the server flag: $header", header.contains("ably-pubsub-server")) + Assert.assertTrue("missing core base identifier in: $header", header.contains("ably-pubsub-java/")) + } finally { + httpServer.stop(0) + } + } + + /** Wire-level: the `agent` query param of the WebSocket upgrade request carries the device flag. */ + @Test + fun webSocketConnections_carryTheDeviceAgentParamOnTheWire() { + ServerSocket(0, 1, InetAddress.getByName("127.0.0.1")).use { listener -> + val requestLine = AtomicReference() + val seen = CountDownLatch(1) + val acceptor = Thread { + try { + listener.accept().use { socket -> + requestLine.set(socket.getInputStream().bufferedReader(Charsets.UTF_8).readLine()) + } + } catch (ignored: Exception) { + // the test fails on the missing request line + } finally { + seen.countDown() + } + } + acceptor.isDaemon = true + acceptor.start() + + val client = PubSubDevice.clientBuilder() + .key(FAKE_KEY) + .tls(false) + .endpoint("127.0.0.1") + .port(listener.localPort) + .build() + try { + Assert.assertTrue("no WebSocket upgrade request seen", seen.await(15, TimeUnit.SECONDS)) + } finally { + client.close() + } + + val line = requestLine.get() + Assert.assertNotNull("no request line observed", line) + val query = if (line!!.contains("?")) line.substring(line.indexOf('?') + 1, line.lastIndexOf(' ')) else "" + val agent = query.split("&") + .firstOrNull { it.startsWith("agent=") } + ?.let { URLDecoder.decode(it.removePrefix("agent="), "UTF-8") } + Assert.assertNotNull("no agent query param in: $line", agent) + Assert.assertTrue("missing core base identifier in: $agent", agent!!.contains("ably-pubsub-java/")) + Assert.assertTrue("missing bare device flag in: $agent", agent.split(" ").contains("ably-pubsub-device")) + Assert.assertFalse("must not carry the server flag: $agent", agent.contains("ably-pubsub-server")) + } + } + companion object { private const val FAKE_KEY = "fakeAppId.fakeKeyId:fakeKeySecret" diff --git a/examples/README.md b/examples/README.md index f9e58fc35..ee178ee82 100644 --- a/examples/README.md +++ b/examples/README.md @@ -38,7 +38,7 @@ EXAMPLES_ABLY_KEY=xxxx:yyyyyy 3. Configure an Emulator or Device - Set up an emulator or connect a physical Android device. - - Ensure the device is configured with at least Android 5.0 (API 21). + - Ensure the device is configured with at least Android 10 (API 29), the example app's `minSdk`. The `io.ably.pubsub:device` artifact it depends on requires API 24 or later. 4. Run the App diff --git a/lib/src/main/java/io/ably/pubsub/debug/DebugOptions.java b/lib/src/main/java/io/ably/pubsub/debug/DebugOptions.java index 32ebf5833..147510873 100644 --- a/lib/src/main/java/io/ably/pubsub/debug/DebugOptions.java +++ b/lib/src/main/java/io/ably/pubsub/debug/DebugOptions.java @@ -38,56 +38,16 @@ public interface RawHttpListener { public WebSocketEngineFactory webSocketEngineFactory; public Clock clock; + @Override public DebugOptions copy() { DebugOptions copied = new DebugOptions(); + copyInto(copied); copied.protocolListener = protocolListener; copied.httpListener = httpListener; copied.transportFactory = transportFactory; copied.httpEngine = httpEngine; copied.webSocketEngineFactory = webSocketEngineFactory; copied.clock = clock; - copied.clientId = clientId; - copied.logLevel = logLevel; - copied.logHandler = logHandler; - copied.tls = tls; - copied.endpoint = endpoint; - copied.port = port; - copied.tlsPort = tlsPort; - copied.autoConnect = autoConnect; - copied.useBinaryProtocol = useBinaryProtocol; - copied.queueMessages = queueMessages; - copied.echoMessages = echoMessages; - copied.recover = recover; - copied.proxy = proxy; - copied.idempotentRestPublishing = idempotentRestPublishing; - copied.httpOpenTimeout = httpOpenTimeout; - copied.httpRequestTimeout = httpRequestTimeout; - copied.httpMaxRetryDuration = httpMaxRetryDuration; - copied.httpMaxRetryCount = httpMaxRetryCount; - copied.realtimeRequestTimeout = realtimeRequestTimeout; - copied.disconnectedRetryTimeout = disconnectedRetryTimeout; - copied.suspendedRetryTimeout = suspendedRetryTimeout; - copied.fallbackRetryTimeout = fallbackRetryTimeout; - copied.defaultTokenParams = defaultTokenParams; - copied.channelRetryTimeout = channelRetryTimeout; - copied.asyncHttpThreadpoolSize = asyncHttpThreadpoolSize; - copied.pushFullWait = pushFullWait; - copied.localStorage = localStorage; - copied.addRequestIds = addRequestIds; - copied.authCallback = authCallback; - copied.authUrl = authUrl; - copied.authMethod = authMethod; - copied.key = key; - copied.token = token; - copied.tokenDetails = tokenDetails; - copied.authHeaders = authHeaders; - copied.authParams = authParams; - copied.queryTime = queryTime; - copied.useTokenAuth = useTokenAuth; - copied.headers = headers; - copied.fallbackHosts = fallbackHosts; - copied.transportParams = transportParams; - copied.agents = agents; return copied; } } diff --git a/lib/src/main/java/io/ably/pubsub/http/HttpCore.java b/lib/src/main/java/io/ably/pubsub/http/HttpCore.java index 1adf15dc3..a960f7167 100644 --- a/lib/src/main/java/io/ably/pubsub/http/HttpCore.java +++ b/lib/src/main/java/io/ably/pubsub/http/HttpCore.java @@ -71,10 +71,11 @@ public class HttpCore { /** * This field is used for analytics purposes. *

- * It holds additional agents that should be added after the Realtime/HTTP client is initialized. + * It holds additional agents that should be added after the realtime client is initialized. * - **Static agents** are set in `ClientOptions`. * - **Dynamic agents** are added later by higher-level SDKs like Chat or Asset Tracking - * and are provided in the `createWrapperSdkProxy` call. + * and are provided in the pubsub-adapter's `RealtimeClient.createWrapperSdkProxy` call. + * The wrapper SDK proxy is available on the realtime client only; the HTTP client has none. */ private Map dynamicAgents; diff --git a/lib/src/main/java/io/ably/pubsub/types/ClientOptions.java b/lib/src/main/java/io/ably/pubsub/types/ClientOptions.java index 1efa71418..199a2cf64 100644 --- a/lib/src/main/java/io/ably/pubsub/types/ClientOptions.java +++ b/lib/src/main/java/io/ably/pubsub/types/ClientOptions.java @@ -7,6 +7,7 @@ import io.ably.pubsub.util.Log; import io.ably.pubsub.util.Log.LogHandler; +import java.util.HashMap; import java.util.Map; /** @@ -83,6 +84,10 @@ public ClientOptions(String key) throws AblyException { * * The default is {@code main}. *

+ * Never put a port in {@code endpoint}; set {@link #port} or {@link #tlsPort} instead. A value such as + * {@code localhost:8080} is not recognised as a hostname, so it is treated as a routing policy name and + * resolves to {@code localhost:8080.realtime.ably.net}. + *

* Spec: REC1 */ public String endpoint; @@ -312,10 +317,25 @@ public ClientOptions(String key) throws AblyException { /** * Internal method * - * @return copy of client options + * @return copy of client options. The {@code headers} and {@code agents} maps and the + * {@code fallbackHosts} and {@code transportParams} arrays are copied, so changing + * them on the original afterwards does not reach the copy; other object-valued + * options (callbacks, storage, proxy, ...) are shared. */ public ClientOptions copy() { - ClientOptions copied = new ClientOptions(); + return copyInto(new ClientOptions()); + } + + /** + * Internal method + *

+ * Copies every option of this object onto {@code copied}, so that a subclass's + * {@code copy()} can reuse the field list and add its own. + * + * @param copied the instance to fill in. + * @return {@code copied}. + */ + protected ClientOptions copyInto(ClientOptions copied) { copied.clientId = clientId; copied.logLevel = logLevel; copied.logHandler = logHandler; @@ -354,10 +374,10 @@ public ClientOptions copy() { copied.authParams = authParams; copied.queryTime = queryTime; copied.useTokenAuth = useTokenAuth; - copied.headers = headers; - copied.fallbackHosts = fallbackHosts; - copied.transportParams = transportParams; - copied.agents = agents; + copied.headers = headers == null ? null : new HashMap<>(headers); + copied.fallbackHosts = fallbackHosts == null ? null : fallbackHosts.clone(); + copied.transportParams = transportParams == null ? null : transportParams.clone(); + copied.agents = agents == null ? null : new HashMap<>(agents); return copied; } diff --git a/lib/src/main/java/io/ably/pubsub/util/AgentHeaderCreator.java b/lib/src/main/java/io/ably/pubsub/util/AgentHeaderCreator.java index daaaff38e..21199a3ca 100644 --- a/lib/src/main/java/io/ably/pubsub/util/AgentHeaderCreator.java +++ b/lib/src/main/java/io/ably/pubsub/util/AgentHeaderCreator.java @@ -3,6 +3,7 @@ import io.ably.pubsub.transport.Defaults; import java.util.Map; +import java.util.TreeMap; public class AgentHeaderCreator { /** @@ -33,8 +34,10 @@ public static String create(Map additionalAgents, PlatformAgentP private static String getAdditionalAgentEntries(Map additionalAgents) { StringBuilder additionalAgentsBuilder = new StringBuilder(); - for (String additionalAgentName : additionalAgents.keySet()) { - String additionalAgentVersion = additionalAgents.get(additionalAgentName); + // Sorted by name so the header is deterministic whatever the map's own iteration order. + for (Map.Entry additionalAgent : new TreeMap<>(additionalAgents).entrySet()) { + String additionalAgentName = additionalAgent.getKey(); + String additionalAgentVersion = additionalAgent.getValue(); additionalAgentsBuilder.append(additionalAgentName); if (additionalAgentVersion != null) { additionalAgentsBuilder.append(AGENT_DIVIDER); diff --git a/lib/src/main/java/io/ably/pubsub/util/Side.java b/lib/src/main/java/io/ably/pubsub/util/Side.java index 79a0c8d85..fffac60f4 100644 --- a/lib/src/main/java/io/ably/pubsub/util/Side.java +++ b/lib/src/main/java/io/ably/pubsub/util/Side.java @@ -7,13 +7,13 @@ /** * Internal helper shared by the {@code io.ably.pubsub:device} and {@code io.ably.pubsub:server} - * door artifacts. It is compiled into each artifact's output from a shared source directory - * rather than published, so that the two artifacts can share this code without a third - * artifact existing for it to live in. + * door artifacts. It lives in this module's sources and ships in {@code io.ably.pubsub:core} + * (and the Android core) as {@code io.ably.pubsub.util.Side}, so both doors depend on it + * through the core artifact they already require. It is public only because the doors live in + * other packages; it is not part of the supported API. *

- * The package split keeps {@code io.ably.pubsub:core} itself as the shared core, so nothing here may - * grow into a general abstraction over the core: it exists only to stamp the side a package - * declares. + * Nothing here may grow into a general abstraction over the core: it exists only to stamp the + * side a package declares. */ public final class Side { private Side() {} @@ -43,7 +43,11 @@ private Side() {} /** * Injects a side agent identifier into the provided {@code ClientOptions} instance. * The method adds the specified agent identifier to the {@code agents} map in the given - * {@code ClientOptions} object. If the {@code agents} map does not exist, a new one is created. + * {@code ClientOptions} object, and removes the opposite side's identifier, so a client + * never declares both sides. If the {@code agents} map does not exist, a new one is created. + *

+ * This mutates {@code options} (it replaces {@code options.agents}); callers that must not + * alter their own object pass a {@link ClientOptions#copy()}. * * @param options the {@code ClientOptions} instance to which the agent identifier will be added. * If {@code null}, the method returns {@code null}. @@ -60,6 +64,8 @@ public static ClientOptions injectSideAgent(ClientOptions options, String identi if (options.agents != null) { agents.putAll(options.agents); } + agents.remove(DEVICE_AGENT_IDENTIFIER.equals(identifier) ? SERVER_AGENT_IDENTIFIER + : SERVER_AGENT_IDENTIFIER.equals(identifier) ? DEVICE_AGENT_IDENTIFIER : null); agents.put(identifier, null); options.agents = agents; return options; diff --git a/lib/src/test/java/io/ably/pubsub/test/realtime/RealtimeHttpHeaderTest.java b/lib/src/test/java/io/ably/pubsub/test/realtime/RealtimeHttpHeaderTest.java index 079cea482..a7715078b 100644 --- a/lib/src/test/java/io/ably/pubsub/test/realtime/RealtimeHttpHeaderTest.java +++ b/lib/src/test/java/io/ably/pubsub/test/realtime/RealtimeHttpHeaderTest.java @@ -1,6 +1,7 @@ package io.ably.pubsub.test.realtime; import fi.iki.elonen.NanoHTTPD; +import io.ably.pubsub.BuildConfig; import io.ably.pubsub.realtime.PubSubRealtimeClient; import io.ably.pubsub.realtime.RealtimeClientFactory; import io.ably.pubsub.test.common.ParameterizedTest; @@ -89,7 +90,7 @@ public void realtime_websocket_param_test() { * Defaults.ABLY_AGENT_PARAM, as ultimately the request param has been derived from those values. */ assertEquals("Verify correct lib version", requestParameters.get("agent"), - Collections.singletonList("ably-pubsub-java/2.0.0 jre/" + System.getProperty("java.version"))); + Collections.singletonList("ably-pubsub-java/" + BuildConfig.VERSION + " jre/" + System.getProperty("java.version"))); /* Spec RTN2a */ assertEquals("Verify correct format", requestParameters.get("format"), diff --git a/lib/src/test/java/io/ably/pubsub/types/ClientOptionsTest.java b/lib/src/test/java/io/ably/pubsub/types/ClientOptionsTest.java index a8d05e62f..5fd55665b 100644 --- a/lib/src/test/java/io/ably/pubsub/types/ClientOptionsTest.java +++ b/lib/src/test/java/io/ably/pubsub/types/ClientOptionsTest.java @@ -1,10 +1,19 @@ package io.ably.pubsub.types; import static org.junit.Assert.assertArrayEquals; -import static org.junit.Assert.assertSame; +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertNotNull; +import static org.junit.Assert.assertNotSame; import static org.junit.Assert.assertTrue; +import io.ably.pubsub.debug.DebugOptions; + +import java.lang.reflect.Field; +import java.lang.reflect.Modifier; +import java.lang.reflect.Proxy; import java.util.HashMap; +import java.util.Map; +import java.util.Objects; import org.junit.Test; @@ -32,9 +41,93 @@ public void copy_carries_headers_fallbackHosts_transportParams_and_agents() { ClientOptions copied = clientOptions.copy(); // Then - assertSame(clientOptions.headers, copied.headers); + assertEquals(clientOptions.headers, copied.headers); assertArrayEquals(clientOptions.fallbackHosts, copied.fallbackHosts); - assertSame(clientOptions.transportParams, copied.transportParams); - assertSame(clientOptions.agents, copied.agents); + assertArrayEquals(clientOptions.transportParams, copied.transportParams); + assertEquals(clientOptions.agents, copied.agents); + } + + @Test + public void copy_isolates_collections_from_later_changes_to_the_original() { + // Given + clientOptions.headers = new HashMap<>(); + clientOptions.agents = new HashMap<>(); + clientOptions.agents.put("some-sdk", "1.2.3"); + clientOptions.fallbackHosts = new String[]{"a.example.com"}; + clientOptions.transportParams = new Param[]{new Param("remainPresentFor", "1000")}; + ClientOptions copied = clientOptions.copy(); + + // When + clientOptions.agents.put("later", "9"); + clientOptions.headers.put("X-Later", "v"); + clientOptions.fallbackHosts[0] = "changed.example.com"; + clientOptions.transportParams[0] = new Param("other", "1"); + + // Then + assertEquals(1, copied.agents.size()); + assertTrue(copied.headers.isEmpty()); + assertEquals("a.example.com", copied.fallbackHosts[0]); + assertEquals("remainPresentFor", copied.transportParams[0].key); + } + + /** + * Fails when an option is added to {@link ClientOptions} (or its auth superclass) without being + * added to {@code copy()}: every public instance field is set to a non-default value, and the + * copy must carry an equal one. + */ + @Test + public void copy_carries_every_public_field() throws Exception { + assertCopyCarriesEveryField(new ClientOptions()); + } + + /** As above for the debug subclass, which must also keep its runtime type and every hook. */ + @Test + public void debugOptions_copy_carries_every_public_field_and_keeps_its_type() throws Exception { + DebugOptions copied = assertCopyCarriesEveryField(new DebugOptions()); + assertEquals(DebugOptions.class, copied.getClass()); + } + + @SuppressWarnings("unchecked") + private static T assertCopyCarriesEveryField(T original) throws Exception { + Object defaults = original.getClass().newInstance(); + for (Field field : original.getClass().getFields()) { + if (!Modifier.isStatic(field.getModifiers()) && !Modifier.isFinal(field.getModifiers())) { + field.set(original, nonDefaultValue(field, defaults)); + } + } + ClientOptions copied = original.copy(); + assertEquals(original.getClass(), copied.getClass()); + assertNotSame(original, copied); + for (Field field : original.getClass().getFields()) { + if (Modifier.isStatic(field.getModifiers()) || Modifier.isFinal(field.getModifiers())) { + continue; + } + assertNotNull(field.getName() + " has no test value", field.get(original)); + assertTrue("copy() does not carry " + field.getDeclaringClass().getSimpleName() + "." + field.getName(), + Objects.deepEquals(field.get(original), field.get(copied))); + } + return (T) copied; + } + + private static Object nonDefaultValue(Field field, Object defaults) throws Exception { + Class type = field.getType(); + if (type == boolean.class) return !field.getBoolean(defaults); + if (type == int.class) return 7; + if (type == long.class) return 7L; + if (type == String.class) return "value"; + if (type == String[].class) return new String[]{"value"}; + if (type == Param[].class) return new Param[]{new Param("k", "v")}; + if (type == Map.class) { + Map map = new HashMap<>(); + map.put("k", "v"); + return map; + } + if (type.isInterface()) { + return Proxy.newProxyInstance(type.getClassLoader(), new Class[]{type}, (proxy, method, args) -> null); + } + // Any other option type must have a no-arg constructor; add a case above if one does not. + java.lang.reflect.Constructor constructor = type.getDeclaredConstructor(); + constructor.setAccessible(true); + return constructor.newInstance(); } } diff --git a/lib/src/test/java/io/ably/pubsub/util/AgentHeaderCreatorTest.java b/lib/src/test/java/io/ably/pubsub/util/AgentHeaderCreatorTest.java index 853ffe8d9..4a859ccf5 100644 --- a/lib/src/test/java/io/ably/pubsub/util/AgentHeaderCreatorTest.java +++ b/lib/src/test/java/io/ably/pubsub/util/AgentHeaderCreatorTest.java @@ -85,6 +85,27 @@ public String createPlatformAgent() { assertMatchingAgentHeaders(PREDEFINED_AGENTS + " android/25 library/1.0.1", agentHeaderValue); } + @Test + public void should_emit_additional_agents_sorted_by_name_and_keep_the_platform_agent_after_them() { + // given + Map agents = new HashMap<>(); + agents.put("zeta", "3"); + agents.put("alpha", null); + agents.put("mid", "2.0"); + PlatformAgentProvider platformAgentProvider = new PlatformAgentProvider() { + @Override + public String createPlatformAgent() { + return "android/25"; + } + }; + + // when + String agentHeaderValue = AgentHeaderCreator.create(agents, platformAgentProvider); + + // then + org.junit.Assert.assertEquals(PREDEFINED_AGENTS + " alpha mid/2.0 zeta/3 android/25", agentHeaderValue); + } + private void assertMatchingAgentHeaders(String expectedAgentHeader, String actualAgentHeader) { assertPredefinedAgentsAreAtTheStart(actualAgentHeader); assertAllExpectedAgentsArePresentInActualAgents(expectedAgentHeader, actualAgentHeader); diff --git a/lib/src/test/java/io/ably/pubsub/util/SideTest.java b/lib/src/test/java/io/ably/pubsub/util/SideTest.java new file mode 100644 index 000000000..fed251bc3 --- /dev/null +++ b/lib/src/test/java/io/ably/pubsub/util/SideTest.java @@ -0,0 +1,51 @@ +package io.ably.pubsub.util; + +import io.ably.pubsub.types.ClientOptions; +import org.junit.Test; + +import java.util.HashMap; +import java.util.Map; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertFalse; +import static org.junit.Assert.assertNull; +import static org.junit.Assert.assertTrue; + +public class SideTest { + + @Test + public void identifiers_arePinnedToTheirWireLiterals() { + // Literal strings on purpose: the platform classifies traffic by these exact tokens. + assertEquals("ably-pubsub-server", Side.SERVER_AGENT_IDENTIFIER); + assertEquals("ably-pubsub-device", Side.DEVICE_AGENT_IDENTIFIER); + } + + @Test + public void injectSideAgent_removesTheOppositeSideAndKeepsOtherEntries() { + ClientOptions options = new ClientOptions(); + options.agents = new HashMap<>(); + options.agents.put("ably-pubsub-server", "x"); + options.agents.put("some-sdk", "1.2.3"); + + Side.injectSideAgent(options, Side.DEVICE_AGENT_IDENTIFIER); + + assertTrue(options.agents.containsKey("ably-pubsub-device")); + assertNull(options.agents.get("ably-pubsub-device")); + assertFalse(options.agents.containsKey("ably-pubsub-server")); + assertEquals("1.2.3", options.agents.get("some-sdk")); + } + + @Test + public void injectSideAgent_removesTheDeviceEntryOnAServerClient() { + ClientOptions options = new ClientOptions(); + Map callerAgents = new HashMap<>(); + callerAgents.put("ably-pubsub-device", null); + options.agents = callerAgents; + + Side.injectSideAgent(options, Side.SERVER_AGENT_IDENTIFIER); + + assertTrue(options.agents.containsKey("ably-pubsub-server")); + assertFalse(options.agents.containsKey("ably-pubsub-device")); + assertTrue("the caller's own map is not modified", callerAgents.containsKey("ably-pubsub-device")); + } +} diff --git a/lib/src/test/kotlin/io/ably/pubsub/uts/unit/realtime/ChannelsCollectionTest.kt b/lib/src/test/kotlin/io/ably/pubsub/uts/unit/realtime/ChannelsCollectionTest.kt index f2fb25ed5..7a0a877b9 100644 --- a/lib/src/test/kotlin/io/ably/pubsub/uts/unit/realtime/ChannelsCollectionTest.kt +++ b/lib/src/test/kotlin/io/ably/pubsub/uts/unit/realtime/ChannelsCollectionTest.kt @@ -4,6 +4,7 @@ import io.ably.pubsub.uts.infra.unit.* import io.ably.pubsub.realtime.ChannelState import io.ably.pubsub.realtime.ConnectionState import io.ably.pubsub.types.AblyException +import io.ably.pubsub.types.ErrorInfo import io.ably.pubsub.types.ProtocolMessage import io.ably.pubsub.uts.infra.awaitChannelState import io.ably.pubsub.uts.infra.awaitState @@ -126,4 +127,148 @@ class ChannelsCollectionTest { assertEquals(0, capturedDetachMessages.size) client.close() } + + /** + * SDK-local coverage of RTS4d for a state the UTS spec does not test yet; **NOT a UTS spec test**. + */ + @Test + fun `RTS4d - release removes a failed channel`() = runTest { + val channelName = "test-RTS4d-failed-${UUID.randomUUID()}" + lateinit var mockWs: MockWebSocket + mockWs = MockWebSocket { + onConnectionAttempt = { conn -> conn.respondWithSuccess(CONNECTED_MESSAGE) } + onMessageFromClient = { msg -> + if (msg.action == ProtocolMessage.Action.attach) { + // ErrorInfo constructor arg order: ErrorInfo(message, statusCode, code). + mockWs.sendToClient(ProtocolMessage(ProtocolMessage.Action.error, msg.channel).apply { + error = ErrorInfo("Attach refused", 400, 40160) + }) + } + } + } + val client = TestRealtimeClient { + autoConnect = false + install(mockWs) + } + val channel = client.channels.get(channelName) + + client.connect() + awaitState(client, ConnectionState.connected) + + channel.attach() + awaitChannelState(channel, ChannelState.failed) + + client.channels.release(channelName) + + assertFalse(client.channels.containsKey(channelName)) + client.close() + } + + /** + * SDK-local coverage of RTS4e for a state the UTS spec does not test yet; **NOT a UTS spec test**. + */ + @Test + fun `RTS4e - release of an attaching channel fails`() = runTest { + val channelName = "test-RTS4e-attaching-${UUID.randomUUID()}" + // The mock never answers the ATTACH, so the channel stays ATTACHING. + val mockWs = MockWebSocket { + onConnectionAttempt = { conn -> conn.respondWithSuccess(CONNECTED_MESSAGE) } + } + val client = TestRealtimeClient { + autoConnect = false + install(mockWs) + } + val channel = client.channels.get(channelName) + + client.connect() + awaitState(client, ConnectionState.connected) + + channel.attach() + awaitChannelState(channel, ChannelState.attaching) + + val error = assertFailsWith { client.channels.release(channelName) }.errorInfo + + assertEquals(90011, error.code) + assertEquals(400, error.statusCode) + assertEquals(ChannelState.attaching, channel.state) + assertSame(channel, client.channels.get(channelName)) + client.close() + } + + /** + * SDK-local coverage of RTS4e for a state the UTS spec does not test yet; **NOT a UTS spec test**. + */ + @Test + fun `RTS4e - release of a detaching channel fails`() = runTest { + val channelName = "test-RTS4e-detaching-${UUID.randomUUID()}" + lateinit var mockWs: MockWebSocket + // The mock answers the ATTACH but never the DETACH, so the channel stays DETACHING. + mockWs = MockWebSocket { + onConnectionAttempt = { conn -> conn.respondWithSuccess(CONNECTED_MESSAGE) } + onMessageFromClient = { msg -> + if (msg.action == ProtocolMessage.Action.attach) { + mockWs.sendToClient(ProtocolMessage(ProtocolMessage.Action.attached, msg.channel)) + } + } + } + val client = TestRealtimeClient { + autoConnect = false + install(mockWs) + } + val channel = client.channels.get(channelName) + + client.connect() + awaitState(client, ConnectionState.connected) + + channel.attach() + awaitChannelState(channel, ChannelState.attached) + channel.detach() + awaitChannelState(channel, ChannelState.detaching) + + val error = assertFailsWith { client.channels.release(channelName) }.errorInfo + + assertEquals(90011, error.code) + assertEquals(400, error.statusCode) + assertEquals(ChannelState.detaching, channel.state) + assertSame(channel, client.channels.get(channelName)) + client.close() + } + + /** + * SDK-local coverage of RTS4e for a state the UTS spec does not test yet; **NOT a UTS spec test**. + */ + @Test + fun `RTS4e - release of a suspended channel fails`() = runTest { + val channelName = "test-RTS4e-suspended-${UUID.randomUUID()}" + lateinit var mockWs: MockWebSocket + // A server-sent DETACHED while ATTACHING moves the channel to SUSPENDED (RTL13b); see deviations.md. + mockWs = MockWebSocket { + onConnectionAttempt = { conn -> conn.respondWithSuccess(CONNECTED_MESSAGE) } + onMessageFromClient = { msg -> + if (msg.action == ProtocolMessage.Action.attach) { + mockWs.sendToClient(ProtocolMessage(ProtocolMessage.Action.detached, msg.channel)) + } + } + } + val client = TestRealtimeClient { + autoConnect = false + channelRetryTimeout = 600_000 + install(mockWs) + } + val channel = client.channels.get(channelName) + + client.connect() + awaitState(client, ConnectionState.connected) + + channel.attach() + awaitChannelState(channel, ChannelState.suspended) + + val error = assertFailsWith { client.channels.release(channelName) }.errorInfo + + assertEquals(90011, error.code) + assertEquals(400, error.statusCode) + assertEquals(ChannelState.suspended, channel.state) + assertSame(channel, client.channels.get(channelName)) + client.close() + } } diff --git a/liveobjects/build.gradle.kts b/liveobjects/build.gradle.kts index 47482fe1b..33a6db482 100644 --- a/liveobjects/build.gradle.kts +++ b/liveobjects/build.gradle.kts @@ -38,6 +38,15 @@ tasks.withType().configureEach { .orElse(providers.environmentVariable("UTS_PROXY_LOCAL_PATH")) .getOrElse(""), ) + + // Which package's entry points the UTS specs construct clients through (`core` default, or + // `server`); forwarded explicitly for the same reason as uts.proxy.localPath. See :uts ClientFactories.kt. + systemProperty( + "uts.side", + providers.systemProperty("uts.side") + .orElse(providers.environmentVariable("UTS_SIDE")) + .getOrElse("core"), + ) } tasks.register("runLiveObjectsUnitTests") { diff --git a/server/src/main/java/io/ably/pubsub/server/PubSubServer.java b/server/src/main/java/io/ably/pubsub/server/PubSubServer.java index 0cfd0674c..b15cdb724 100644 --- a/server/src/main/java/io/ably/pubsub/server/PubSubServer.java +++ b/server/src/main/java/io/ably/pubsub/server/PubSubServer.java @@ -25,6 +25,13 @@ * on, rather than leaving it to be inferred. *

* If your code runs on an end-user device, use the {@code io.ably.pubsub:device} artifact instead. + *

+ * Token authentication and realtime. Clients built here declare themselves as + * servers, so authenticate them with an API key. Token authentication on a realtime connection + * works only with a token that carries the signed {@code x-ably-clientType=server} claim, which + * today means a JWT; with any other token (an {@code authCallback}, {@code token} or + * {@code tokenDetails} yielding a native Ably token) Ably rejects the connection with error + * 40167. Clients from {@code PubSubDevice} are not affected. * *

{@code
  * PubSubHttpClient http = PubSubServer.httpClientBuilder()
@@ -66,11 +73,10 @@ public static HttpClientBuilder httpClientBuilder() {
      * builder over assembling options for it.
      * 

* The builder's setters still apply on top of what is passed here, and {@link - * HttpClientBuilder#build()} stamps the side-declaring agent entry as usual. The - * options object is handed to the client rather than copied, so later changes to it are not - * isolated from the built client, and {@code build()} replaces its {@code agents} map with - * the stamped one; pass {@link ClientOptions#copy()} if either matters. A {@code null} is - * carried through to {@code build()}, which surfaces the core constructor's own + * HttpClientBuilder#build()} stamps the side-declaring agent entry as usual. {@code build()} + * snapshots the options ({@link ClientOptions#copy()}), so neither later builder calls nor + * later changes to the object passed here affect a client already built, and the passed + * object's own {@code agents} map is never modified. A {@code null} is carried through to {@code build()}, which surfaces the core constructor's own * "no options provided" error. * * @param options the options to start from. @@ -104,11 +110,10 @@ public static RealtimeClientBuilder realtimeClientBuilder() { * builder over assembling options for it. *

* The builder's setters still apply on top of what is passed here, and {@link - * RealtimeClientBuilder#build()} stamps the side-declaring agent entry as usual. The - * options object is handed to the client rather than copied, so later changes to it are not - * isolated from the built client, and {@code build()} replaces its {@code agents} map with - * the stamped one; pass {@link ClientOptions#copy()} if either matters. A {@code null} is - * carried through to {@code build()}, which surfaces the core constructor's own + * RealtimeClientBuilder#build()} stamps the side-declaring agent entry as usual. {@code build()} + * snapshots the options ({@link ClientOptions#copy()}), so neither later builder calls nor + * later changes to the object passed here affect a client already built, and the passed + * object's own {@code agents} map is never modified. A {@code null} is carried through to {@code build()}, which surfaces the core constructor's own * "no options provided" error. * * @param options the options to start from. @@ -127,7 +132,7 @@ public static RealtimeClientBuilder realtimeClientBuilder(ClientOptions options) */ public abstract static class ClientBuilder> { - /** Accumulates the calls made on this builder; handed to the client as-is by build(). */ + /** Accumulates the calls made on this builder; build() hands the client a snapshot of it. */ final ClientOptions options; ClientBuilder() { @@ -138,6 +143,11 @@ public abstract static class ClientBuilder> { this.options = options; } + /** The copy handed to the client; null stays null for the core constructor to reject. */ + static ClientOptions snapshot(ClientOptions options) { + return options == null ? null : options.copy(); + } + @SuppressWarnings("unchecked") private T self() { return (T) this; @@ -530,7 +540,7 @@ public static final class HttpClientBuilder extends ClientBuilder agents = new HashMap<>(); + agents.put("ably-pubsub-device", null); + agents.put("some-sdk", "1.2.3"); + PubSubHttpClient http = PubSubServer.httpClientBuilder().key(FAKE_KEY).agents(agents).build(); + PubSubRealtimeClient realtime = PubSubServer + .realtimeClientBuilder().key(FAKE_KEY).autoConnect(false).agents(agents).build(); + for (Map built : Arrays.asList(http.options.agents, realtime.options.agents)) { + assertFalse(built.containsKey("ably-pubsub-device")); + assertTrue(built.containsKey("ably-pubsub-server")); + assertEquals("1.2.3", built.get("some-sdk")); + } + assertTrue("the caller's map is not modified", agents.containsKey("ably-pubsub-device")); + } + + @Test + public void reusingABuilder_doesNotAffectAnEarlierClient() throws AblyException { + PubSubServer.HttpClientBuilder builder = PubSubServer.httpClientBuilder().key(FAKE_KEY); + PubSubHttpClient first = builder.build(); + + Map later = new HashMap<>(); + later.put("later-sdk", "9"); + builder.key("other.key:secret").agents(later).tls(false); + PubSubHttpClient second = builder.build(); + + assertEquals(FAKE_KEY, first.options.key); + assertTrue(first.options.tls); + assertFalse(first.options.agents.containsKey("later-sdk")); + assertEquals("other.key:secret", second.options.key); + assertTrue(second.options.agents.containsKey("later-sdk")); + } + + @Test + public void changingPassedOptionsAfterBuild_doesNotAffectTheClient() throws Exception { + ClientOptions options = new ClientOptions(FAKE_KEY); + options.autoConnect = false; + PubSubRealtimeClient client = PubSubServer.realtimeClientBuilder(options).build(); + + options.clientId = "changed"; + options.agents = new HashMap<>(); + + assertNotSame(options, client.options); + assertNull(client.options.clientId); + assertServerFlag(client.options.agents); + } + + /** + * Wire-level assertion for the realtime door: the {@code agent} query parameter of the WebSocket + * upgrade request carries the base identifier and the server flag, and not the device flag. + */ + @Test + public void webSocketConnections_carryTheServerAgentParamOnTheWire() throws Exception { + try (ServerSocket listener = new ServerSocket(0, 1, java.net.InetAddress.getByName("127.0.0.1"))) { + AtomicReference requestLine = new AtomicReference<>(); + CountDownLatch seen = new CountDownLatch(1); + Thread acceptor = new Thread(() -> { + try (Socket socket = listener.accept()) { + BufferedReader reader = new BufferedReader( + new InputStreamReader(socket.getInputStream(), StandardCharsets.UTF_8)); + requestLine.set(reader.readLine()); + } catch (Exception ignored) { + // the test fails on the missing request line + } finally { + seen.countDown(); + } + }); + acceptor.setDaemon(true); + acceptor.start(); + + PubSubRealtimeClient client = PubSubServer + .realtimeClientBuilder() + .key(FAKE_KEY) + .tls(false) + .endpoint("127.0.0.1") + .port(listener.getLocalPort()) + .build(); + try { + assertTrue("no WebSocket upgrade request seen", seen.await(15, TimeUnit.SECONDS)); + } finally { + client.close(); + } + + String line = requestLine.get(); + assertNotNull("no request line observed", line); + String agent = null; + String query = line.contains("?") ? line.substring(line.indexOf('?') + 1, line.lastIndexOf(' ')) : ""; + for (String pair : query.split("&")) { + if (pair.startsWith("agent=")) { + agent = URLDecoder.decode(pair.substring("agent=".length()), "UTF-8"); + } + } + assertNotNull("no agent query param in: " + line, agent); + assertTrue("missing core base identifier in: " + agent, agent.contains("ably-pubsub-java/")); + assertTrue("missing bare server flag in: " + agent, + Arrays.asList(agent.split(" ")).contains("ably-pubsub-server")); + assertFalse("must not carry the device flag: " + agent, agent.contains("ably-pubsub-device")); + } + } } diff --git a/uts/README.md b/uts/README.md index 82d1d2d55..43f47e0f9 100644 --- a/uts/README.md +++ b/uts/README.md @@ -55,7 +55,7 @@ UTS fixes this by separating **what to test** from **how to test it in a given l ▼ ┌──────────────────────────────┐ │ Derived tests │ ← concrete, runnable tests in the SDK's language - │ (Kotlin, in ably-java) │ spec suites: :java / :liveobjects; + │ (Kotlin, in ably-java) │ spec suites: :core / :liveobjects; └──────────────────────────────┘ shared infra + smoke examples: :uts ``` @@ -65,8 +65,8 @@ Three concepts you will see constantly: |------|---------| | **Spec point** | A tagged requirement in the features spec, e.g. `RTN16g`, `RTN22`, `RTL4f`. Test names embed these. | | **UTS spec** | A markdown file of portable pseudocode describing the setup, steps, and assertions for one feature. The *source of truth for what to test.* | -| **Derived test** | A faithful translation of a UTS spec into a real test in a specific SDK/language. These live in the owning module's test source set (`:java` for realtime/rest, `:liveobjects` for objects); the shared infra they use lives in `:uts`. | -| **Deviation** | A documented case where the SDK's actual behaviour diverges from the spec. Recorded in the owning module's `deviations.md` (`:java` and `:liveobjects` — see §12). | +| **Derived test** | A faithful translation of a UTS spec into a real test in a specific SDK/language. These live in the owning module's test source set (`:core` for realtime/rest, `:liveobjects` for objects); the shared infra they use lives in `:uts`. | +| **Deviation** | A documented case where the SDK's actual behaviour diverges from the spec. Recorded in the owning module's `deviations.md` (`:core` and `:liveobjects` — see §12). | The golden rule (from [`writing-derived-tests.md`](https://github.com/ably/specification/blob/main/uts/docs/writing-derived-tests.md)): **translate the UTS spec faithfully** — same structure, same assertions, same naming — don't optimise or skip steps. Every derived test carries a @@ -88,7 +88,7 @@ three example tests this guide walks through span all three tiers. The three examples above are `:uts`'s own **tier smoke tests** (§9–§11) — the reference shapes this guide walks through. The real, spec-derived suites live **by module**, in the module that owns the -code under test: realtime/rest under `:java` (`lib/src/test/kotlin/io/ably/pubsub/uts/…`), objects under +code under test: realtime/rest under `:core` (`lib/src/test/kotlin/io/ably/pubsub/uts/…`), objects under `:liveobjects` (`liveobjects/.../uts/…`). `:uts`'s own test tree holds only the tier smoke examples; so a feature's tests always sit with the SDK code they exercise (see §4.2 "Where every module's UTS tests live"). @@ -160,13 +160,13 @@ A big table mapping every features-spec group (`RSC`, `RTN`, `RTL`, `RTP`, …) cover it, with a per-tier summary (`unit:✓ proxy:✓`). This is the tracker for "what's done and what's missing". The reference tests this guide walks through correspond to these rows: - `RTN16` (connection recovery) → unit spec `connection_recovery_test.md` → - **`ConnectionRecoveryTest.kt`** (`:java`, `lib/src/test/kotlin/io/ably/pubsub/uts/unit/realtime/`). + **`ConnectionRecoveryTest.kt`** (`:core`, `lib/src/test/kotlin/io/ably/pubsub/uts/unit/realtime/`). - `RTL10d` (channel history) → direct-sandbox spec `realtime/integration/channel_history_test.md` → **`ChannelHistoryTest.kt`** - (`:java`, `.../integration/standard/realtime/`). + (`:core`, `.../integration/standard/realtime/`). - `RTN22` / `RTC8a` (server-initiated re-auth) → proxy spec `realtime/integration/proxy/auth_reauth.md` → **`AuthReauthTest.kt`** - (`:java`, `.../integration/proxy/realtime/`). + (`:core`, `.../integration/proxy/realtime/`). > There is also a fifth, *referenced* spec: > [`docs/proxy.md`](https://github.com/ably/specification/blob/main/uts/docs/proxy.md) @@ -189,7 +189,7 @@ teaches from. Two things this means — and a correction to how the module used to be described. First, the infra is **this module's main/production code**, not a `testFixtures` variant and not test-only: `:uts`'s main artifact *is* the infra. Second, `:uts` does **not** host the spec-derived UTS suites — those live in -their owning modules (`:java` for realtime/rest, `:liveobjects` for objects; see §4.2). `:uts` keeps +their owning modules (`:core` for realtime/rest, `:liveobjects` for objects; see §4.2). `:uts` keeps only the infra and the tier smoke examples. ### 4.1 `uts/build.gradle.kts` @@ -201,7 +201,7 @@ plugins { } java { - // Declare Java-8 outgoing variants so :java's 8-requesting configurations can consume + // Declare Java-8 outgoing variants so :core's 8-requesting configurations can consume // project(":uts"). sourceCompatibility = JavaVersion.VERSION_1_8 targetCompatibility = JavaVersion.VERSION_1_8 @@ -321,7 +321,7 @@ The infra is shared, but the actual UTS suites live **in the module that owns th | Module | UTS tests | Location | Run with | |---|---|---|---| -| `:java` | realtime (and future rest) — unit, integration, proxy | `lib/src/test/kotlin/io/ably/pubsub/uts/{unit, integration/standard, integration/proxy}/realtime/` | `:java:runUtsUnitTests` / `:java:runUtsIntegrationTests` | +| `:core` | realtime (and future rest) — unit, integration, proxy | `lib/src/test/kotlin/io/ably/pubsub/uts/{unit, integration/standard, integration/proxy}/realtime/` | `:core:runUtsUnitTests` / `:core:runUtsIntegrationTests` | | `:liveobjects` | objects — unit, integration, proxy | `liveobjects/src/test/kotlin/io/ably/pubsub/liveobjects/uts/{unit, integration, proxy}/` | `:liveobjects:runLiveObjectsUnitTests` / `:liveobjects:runLiveObjectsIntegrationTests` | | `:uts` | **none** — shared infra + one smoke test per tier | `uts/src/main/.../infra/**` + `uts/src/test/.../{unit, integration/standard, integration/proxy}/` | `:uts:runUtsUnitTests` / `:uts:runUtsIntegrationTests` | @@ -620,7 +620,7 @@ technique used by `liveobjects/.../TestUtils.kt`). See Appendix B.1. not derived from a spec) and must never trip the spec-parity tooling; it is the reference shape a future unit-tier UTS test should take. -> The real spec-derived unit suites this pattern scales to live in `:java` +> The real spec-derived unit suites this pattern scales to live in `:core` > (`lib/src/test/kotlin/io/ably/pubsub/uts/unit/realtime/`, e.g. `ConnectionRecoveryTest`) and > `:liveobjects` — see §13. @@ -681,7 +681,7 @@ One long **await-style** test that walks the SDK through the whole transport lif *Why await-style throughout?* The initial connect, the FakeClock reconnect, and the refuse branch each need a **different** answer per attempt — a single `onConnectionAttempt` callback answers every attempt -uniformly, and the two styles cannot be mixed on one mock. (This is the same reason `:java`'s +uniformly, and the two styles cannot be mixed on one mock. (This is the same reason `:core`'s `ConnectionRecoveryTest` is await-style.) *Technique on show: await-style `awaitConnectionAttempt` / `awaitNextMessageFromClient`, `sendToClient` for server frames, `events` for assertions, `FakeClock` for deterministic backoff, and a Unicode channel-name round-trip.* @@ -741,7 +741,7 @@ alone reaching quiescence. **Purpose:** the permanent acceptance test for the middle-tier infra — `SandboxApp` + `TestRealtimeClient`/`TestHttpClient` wired straight to the sandbox. No `@UTS` marker. -> The real spec-derived direct-sandbox suites this pattern scales to live in `:java` +> The real spec-derived direct-sandbox suites this pattern scales to live in `:core` > (`lib/src/test/kotlin/io/ably/pubsub/uts/integration/standard/realtime/`, e.g. `ChannelHistoryTest`) > and `:liveobjects` — see §13. @@ -823,7 +823,7 @@ REST `history()` call. `SandboxApp` + client wiring through the proxy, exercising **both** fault-injection styles. No `@UTS` marker. -> The real spec-derived proxy suites this pattern scales to live in `:java` +> The real spec-derived proxy suites this pattern scales to live in `:core` > (`lib/src/test/kotlin/io/ably/pubsub/uts/integration/proxy/realtime/`, e.g. `AuthReauthTest`) and > `:liveobjects` — see §13. @@ -907,7 +907,7 @@ assertions** as the primary verification (`getLog()` → filter by `type`/`direc Deviations live **with their tests**, not in `:uts` (whose smoke tests carry none by design). There are two catalogues: -- `lib/src/test/kotlin/io/ably/pubsub/uts/deviations.md` — the **realtime/rest** tiers, hosted in `:java`. +- `lib/src/test/kotlin/io/ably/pubsub/uts/deviations.md` — the **realtime/rest** tiers, hosted in `:core`. - `liveobjects/src/test/kotlin/io/ably/pubsub/liveobjects/uts/deviations.md` — **all three objects tiers** (unit, integration, proxy), hosted in `:liveobjects`. @@ -918,7 +918,7 @@ The mechanism (from [`writing-derived-tests.md`](https://github.com/ably/specifi gates it behind the `RUN_DEVIATIONS` env var, with a regression-guard assertion for the SDK's actual behaviour running by default. Normal runs stay green; `RUN_DEVIATIONS=1` turns the failing assertions on so the gap is reproducible and the test flips automatically once the SDK is fixed. In code (from -`:java`'s `ConnectionRecoveryTest`, RTN16f): +`:core`'s `ConnectionRecoveryTest`, RTN16f): ```kotlin if (System.getenv("RUN_DEVIATIONS") != null) { assertEquals(42L, currentRecoveryKey.msgSerial) // spec-correct: recover preserves msgSerial @@ -926,14 +926,14 @@ if (System.getenv("RUN_DEVIATIONS") != null) { assertEquals(0L, currentRecoveryKey.msgSerial) // regression guard: the SDK's actual behaviour } ``` -Run it with `RUN_DEVIATIONS=1 ./gradlew :java:runUtsUnitTests --tests "*ConnectionRecoveryTest*"` (§13). +Run it with `RUN_DEVIATIONS=1 ./gradlew :core:runUtsUnitTests --tests "*ConnectionRecoveryTest*"` (§13). Representative entries from the realtime catalogue (`lib/.../deviations.md`): | Spec point | Gist | Touches | |------------|------|---------| -| **RTN16f** | SDK resets `msgSerial` to 0 on connect even with `recover`; spec says preserve it (42). | `ConnectionRecoveryTest` (`:java`) — `assertEquals(42L,…)` gated, `assertEquals(0L,…)` default guard. | -| **RTN16g2** | Spec's fatal error 50000/500 isn't fatal to the SDK (`isFatalError()` needs code 40000–49999 or status < 500); also `send_to_client_and_close` races the FAILED transition. | `ConnectionRecoveryTest` (`:java`) — uses 40000/400 + plain `sendToClient`. | +| **RTN16f** | SDK resets `msgSerial` to 0 on connect even with `recover`; spec says preserve it (42). | `ConnectionRecoveryTest` (`:core`) — `assertEquals(42L,…)` gated, `assertEquals(0L,…)` default guard. | +| **RTN16g2** | Spec's fatal error 50000/500 isn't fatal to the SDK (`isFatalError()` needs code 40000–49999 or status < 500); also `send_to_client_and_close` races the FAILED transition. | `ConnectionRecoveryTest` (`:core`) — uses 40000/400 + plain `sendToClient`. | | **RTL13b / RTL13c** | Channel-state reattach / `channelRetryTimeout` gaps. Retained as confirmed SDK gaps; the citing channel tests aren't in the suite yet (only `ConnectionRecoveryTest` is translated). | pending the channels-module translation. | The objects catalogue additionally records the typed-SDK / language adaptations (RTTS API @@ -948,20 +948,20 @@ listener-dedup divergence — see `liveobjects/.../deviations.md`. ## 13. How to Run the Tests Each module registers package-filtered Gradle tasks. `:uts` registers `runUtsUnitTests` / -`runUtsIntegrationTests` (in `uts/build.gradle.kts`); `:java` registers the same-named tasks for its -realtime suites; `:liveobjects` registers `runLiveObjectsUnitTests` / `runLiveObjectsIntegrationTests`. +`runUtsIntegrationTests` (in `uts/build.gradle.kts`) for the infra smoke tests only; `:core` registers +the same-named tasks for the realtime conformance suites; `:liveobjects` registers `runLiveObjectsUnitTests` / `runLiveObjectsIntegrationTests`, and its UTS specs are under `io.ably.pubsub.liveobjects.uts.*`. | What | Command | |---|---| | `:uts` smoke (unit, offline) | `./gradlew :uts:runUtsUnitTests` | | `:uts` smoke (integration + proxy) | `./gradlew :uts:runUtsIntegrationTests` | -| realtime UTS unit | `./gradlew :java:runUtsUnitTests` | -| realtime UTS integration + proxy | `./gradlew :java:runUtsIntegrationTests` | +| realtime UTS unit | `./gradlew :core:runUtsUnitTests` | +| realtime UTS integration + proxy | `./gradlew :core:runUtsIntegrationTests` | | objects UTS unit | `./gradlew :liveobjects:runLiveObjectsUnitTests` | | objects UTS integration + proxy | `./gradlew :liveobjects:runLiveObjectsIntegrationTests` | Each `runUts*` / `runLiveObjects*` task is package-filtered (`io.ably.pubsub.uts.unit.*` / -`io.ably.pubsub.uts.integration.*` for `:uts` and `:java`; `io.ably.pubsub.liveobjects.uts.*` for +`io.ably.pubsub.uts.integration.*` for `:uts` and `:core`; `io.ably.pubsub.liveobjects.uts.*` for `:liveobjects`). The `…IntegrationTests` tasks cover **both** the direct-sandbox (`integration/standard/`) and proxy (`integration/proxy/`) tiers — proxy tests additionally download/launch the uts-proxy. @@ -970,10 +970,10 @@ download/launch the uts-proxy. # All :uts smoke tests (every tier), or one class: ./gradlew :uts:test ./gradlew :uts:runUtsUnitTests --tests "io.ably.pubsub.uts.unit.UnitInfraSmokeTest" -./gradlew :java:runUtsIntegrationTests --tests "io.ably.pubsub.uts.integration.proxy.realtime.AuthReauthTest" +./gradlew :core:runUtsIntegrationTests --tests "io.ably.pubsub.uts.integration.proxy.realtime.AuthReauthTest" # Turn on the spec-correct (currently failing) deviation assertions (§12): -RUN_DEVIATIONS=1 ./gradlew :java:runUtsUnitTests --tests "*ConnectionRecoveryTest*" +RUN_DEVIATIONS=1 ./gradlew :core:runUtsUnitTests --tests "*ConnectionRecoveryTest*" # Run proxy tests against a locally built proxy instead of a GitHub release: ./gradlew :uts:runUtsIntegrationTests -Duts.proxy.localPath=/path/to/uts-proxy # or .tar.gz @@ -981,17 +981,18 @@ RUN_DEVIATIONS=1 ./gradlew :java:runUtsUnitTests --tests "*ConnectionRecoveryTes ``` > **Unqualified task names collide — intentionally.** `./gradlew runUtsUnitTests` (no module prefix) -> runs the task in **both** `:uts` and `:java` (Gradle matches the task name across every project that +> runs the task in **both** `:uts` and `:core` (Gradle matches the task name across every project that > declares it). That's intended and harmless — both unit tiers are offline — and mirrors the existing -> `runUnitTests` precedent (registered in both `:java` and `:pubsub-adapter`). Qualify with `:uts:` / -> `:java:` to target one. The `RUN_DEVIATIONS`, `-Duts.proxy.localPath` and `UTS_PROXY_LOCAL_PATH` +> `runUnitTests` precedent (registered in both `:core` and `:pubsub-adapter`). Qualify with `:uts:` / +> `:core:` to target one. The `RUN_DEVIATIONS`, `-Duts.proxy.localPath` and `UTS_PROXY_LOCAL_PATH` > knobs are valid for **all three** modules. **Where CI runs them:** `check.yml` runs -`… runUnitTests runLiveObjectsUnitTests :java:runUtsUnitTests :uts:runUtsUnitTests` as the PR gate +`… runUnitTests runLiveObjectsUnitTests :core:runUtsUnitTests :uts:runUtsUnitTests` as the PR gate (all offline unit tiers); `integration-test.yml`'s `check-uts` job runs -`:java:runUtsIntegrationTests :uts:runUtsIntegrationTests`, and its `check-liveobjects` job runs -`runLiveObjectsIntegrationTests`. +`:core:runUtsIntegrationTests :uts:runUtsIntegrationTests`, and its `check-liveobjects` job runs +`runLiveObjectsIntegrationTests`. Both workflows run a second `-Duts.side=server` leg (see +[Per-side package modes](#per-side-package-modes)). ### Per-side package modes @@ -1000,18 +1001,32 @@ in `infra/unit/ClientFactories.kt`), selected by the `uts.side` system property `UTS_SIDE` environment variable): ```bash -./gradlew :uts:runUtsUnitTests # core (default): the core construction seam, no side declared -./gradlew :uts:runUtsUnitTests -Duts.side=server # io.ably.pubsub:server — the PubSubServer builders +./gradlew :core:runUtsUnitTests :uts:runUtsUnitTests # core (default): the core construction seam, no side declared +./gradlew :core:runUtsUnitTests :uts:runUtsUnitTests -Duts.side=server # io.ably.pubsub:server — the PubSubServer builders +./gradlew :core:runUtsIntegrationTests :uts:runUtsIntegrationTests -Duts.side=server # the same, against the sandbox and proxy ``` +The conformance specs live in `:core` (and in `:liveobjects` where applicable), not in `:uts`, which +holds only `SideModesTest` and the infra smoke tests. `:core`'s `runUts*` tasks forward `uts.side` / +`UTS_SIDE` to the forked test JVM, so running `:uts:runUtsUnitTests -Duts.side=server` alone would +exercise only the smoke tests and prove nothing about conformance. Always run the `:core` task +alongside it for a side leg. + The server builders only stamp the side-declaring agent entry (a versionless flag, per ably/ably-common#361) and pass everything else through — `DebugOptions` included, whose `copy()` override keeps the mock hooks — so every mode must pass identically. `SideModesTest` asserts each mode's stamp so a broken seam cannot silently degrade the server leg into a duplicate core run. -CI runs both modes (`check.yml` and `integration-test.yml`). +CI runs both modes. The exact server legs are: + +```bash +# check.yml +./gradlew :core:runUtsUnitTests :uts:runUtsUnitTests :liveobjects:runLiveObjectsUnitTests -Duts.side=server +# integration-test.yml (--tests narrows only the :liveobjects task) +./gradlew :core:runUtsIntegrationTests :uts:runUtsIntegrationTests :liveobjects:runLiveObjectsIntegrationTests --tests 'io.ably.pubsub.liveobjects.uts.*' -Duts.side=server +``` -Unlike ably-js's UTS there is no `device` mode: `io.ably.pubsub:device` is an Android artifact, -so its door cannot run on the JVM this suite uses. Its stamping contract is covered by the +Unlike ably-js's UTS there is no `device` mode: `io.ably.pubsub:device` is not wired +into this suite's construction seam. Its stamping contract is covered by the instrumentation tests in the `device` module (`emulate.yml`). **Token auth on the server leg.** Realtime rejects a token-authenticated connection that @@ -1093,7 +1108,7 @@ is talking to a real socket; in fact every byte is intercepted by the mock and s │ │ client.connect() ▲ awaitState(client, connected) │ │ ▼ │ │ │ ┌───────────┐ webSocketEngineFactory ┌──────────────────────────┐ │ - │ │ PubSubRealtimeClient (SDK :java) │──────────▶ │ MockWebSocketEngineFactory │ (implements SDK SPI) │ + │ │ PubSubRealtimeClient (SDK :core) │──────────▶ │ MockWebSocketEngineFactory │ (implements SDK SPI) │ │ │ ConnectionManager, etc. │ └─────────────┬────────────┘ │ │ └───────────┬────────────────┘ │ create() │ │ │ send(frame) ───────────────────────────▶ │ │ @@ -1200,7 +1215,7 @@ implicit. > from. The infra under `infra/unit/` and `infra/integration/` is built out beyond what the smokes > exercise (full HTTP mock, all four rule builders, REST proxy wiring, etc.), anticipating the broader > UTS coverage catalogued in [`completion-status.md`](https://github.com/ably/specification/blob/main/uts/docs/completion-status.md). The real spec-derived suites that consume this -> infra live in `:java` (`lib/src/test/kotlin/io/ably/pubsub/uts/…`) and `:liveobjects` +> infra live in `:core` (`lib/src/test/kotlin/io/ably/pubsub/uts/…`) and `:liveobjects` > (`liveobjects/.../uts/…`) — see §13. --- @@ -1220,4 +1235,4 @@ implicit. | Integration helpers | `uts/src/main/.../uts/infra/integration/*` (+ `…/integration/proxy/*`) | | Async helpers | `uts/src/main/.../uts/infra/Utils.kt` (awaits), `…/infra/unit/Utils.kt` (ConnectionDetails builder) | | The three tier smoke tests | `uts/src/test/.../uts/unit/UnitInfraSmokeTest.kt`, `…/uts/integration/standard/IntegrationInfraSmokeTest.kt`, `…/uts/integration/proxy/ProxyInfraSmokeTest.kt` | -| Deviations | `lib/src/test/kotlin/io/ably/pubsub/uts/deviations.md` (realtime/rest, `:java`), `liveobjects/src/test/kotlin/io/ably/pubsub/liveobjects/uts/deviations.md` (objects, `:liveobjects`) | +| Deviations | `lib/src/test/kotlin/io/ably/pubsub/uts/deviations.md` (realtime/rest, `:core`), `liveobjects/src/test/kotlin/io/ably/pubsub/liveobjects/uts/deviations.md` (objects, `:liveobjects`) | diff --git a/uts/src/main/kotlin/io/ably/pubsub/uts/infra/unit/ClientFactories.kt b/uts/src/main/kotlin/io/ably/pubsub/uts/infra/unit/ClientFactories.kt index 242be969c..c3dcff9fb 100644 --- a/uts/src/main/kotlin/io/ably/pubsub/uts/infra/unit/ClientFactories.kt +++ b/uts/src/main/kotlin/io/ably/pubsub/uts/infra/unit/ClientFactories.kt @@ -22,19 +22,19 @@ class ClientOptionsBuilder : DebugOptions("appId.keyId:keySecret") { /** * Which package's entry points the suite constructs clients through, selected by the - * `uts.side` system property (uts/build.gradle.kts forwards it to the test JVM): + * `uts.side` system property (the :uts, :core and :liveobjects test tasks forward it to the test JVM): * * - `core` (default): the core construction seam ([RealtimeClientFactory] / [HttpClientFactory]), * which declares no side. * - `server`: `io.ably.pubsub:server` — both client kinds via its side-stamping builders. * - * There is no `device` mode, unlike ably-js's UTS: `io.ably.pubsub:device` is an Android - * artifact, so its door cannot run on the JVM this suite uses; its stamping contract is - * covered by the instrumentation tests in the device module instead. + * There is no `device` mode, unlike ably-js's UTS: `io.ably.pubsub:device` is not wired into + * this suite's construction seam. Its stamping contract is covered by the device module's own + * tests (`jvmTest` and the instrumentation tests in emulate.yml). * * The builders only stamp the side-declaring agent entry and pass every other option - * through — [DebugOptions] included: its `copy()` override keeps the mock hooks the suite - * installs — so conformance must be identical whichever door constructed the client. + * through (`build()` snapshots the options via `copy()`, and [DebugOptions]'s `copy()` + * override keeps the mock hooks the suite installs), so conformance must be identical whichever door constructed the client. * `SideModesTest` asserts each mode's stamp, so a broken seam cannot silently degrade the * server CI leg into a duplicate of the core leg. */