diff --git a/README.md b/README.md index d399462..8e9b761 100644 --- a/README.md +++ b/README.md @@ -1,23 +1,24 @@ # Java SDK for [Visual Regression Tracker](https://github.com/Visual-Regression-Tracker/Visual-Regression-Tracker) -[![Codacy Badge](https://app.codacy.com/project/badge/Grade/79dcd96f2be04992bc1059cad60e2e04)](https://www.codacy.com/gh/Visual-Regression-Tracker/sdk-java?utm_source=github.com&utm_medium=referral&utm_content=Visual-Regression-Tracker/sdk-java&utm_campaign=Badge_Grade) -Requires Java 11 or later. +Track screenshots from Java tests in a Visual Regression Tracker server. Requires Java 25 or later. Built and tested against VRT server 6.0; image URLs need server 5.0.10 or later. -## Gradle +## Install + +The SDK is published on [JitPack](https://jitpack.io/#Visual-Regression-Tracker/sdk-java). The version is a git tag; see [releases](https://github.com/Visual-Regression-Tracker/sdk-java/releases). + +### Gradle ```groovy repositories { maven { url 'https://jitpack.io' } } -``` -```groovy dependencies { - implementation 'com.github.Visual-Regression-Tracker:sdk-java:5.5.0' + testImplementation 'com.github.Visual-Regression-Tracker:sdk-java:6.0.0' } ``` -## Maven +### Maven ```xml @@ -26,164 +27,165 @@ dependencies { https://jitpack.io -``` -```xml com.github.Visual-Regression-Tracker sdk-java - 5.5.0 + 6.0.0 + test ``` -[Available versions](https://github.com/Visual-Regression-Tracker/sdk-java/releases) -More info about https://jitpack.io/ +The SDK logs through SLF4J 2. Add a provider such as `logback-classic` to see its output. -## Usage +### Upgrading to 6.0 -### Configuration -In order to initialize VisualRegressionTracker, following options should be defined: +- Java 25 or later is required. +- SLF4J 1.7 bindings are not picked up; use an SLF4J 2 provider (logback 1.3+, `log4j-slf4j2-impl`). +- The API is unchanged. A call that passes a literal `null` or a bare Mockito `any()` as the image is now ambiguous between the `String`, `byte[]` and `Path` overloads of `track`; use `any(String.class)`. -* `apiUrl` (**Required**) - URL where backend is running. _Example_: "http://localhost:4200" -* `project` (**Required**) - Project name or ID. _Example_: "003f5fcf-6c5f-4f1f-a99f-82a697711382" -* `apiKey` (**Required**) - User apiKey. _Example_: "F5Z2H0H2SNMXZVHX0EA4YQM1MGDD" -* `branchName` (Optional) - Current git branch. _Example_: "develop". If not set, the build gets the project's main branch and tracked images use the build's branch -* `enableSoftAssert` (Optional) - Log errors instead of exceptions. Default value is false -* `ciBuildId` (Optional) - id of the build in CI system -* `httpTimeoutInSeconds` (Optional) - how long to wait for each server response, in seconds. Default value is 360 seconds, because the server compares images while handling the request. Connecting has its own 10 second limit +## Configuration - There are a few ways to provide those options +| Option | Environment variable | Required | Description | +|---|---|---|---| +| `apiUrl` | `VRT_APIURL` | yes | URL of the VRT API, for example `http://localhost:4200`; trailing slashes are ignored | +| `apiKey` | `VRT_APIKEY` | yes | API key of a VRT user | +| `project` | `VRT_PROJECT` | yes | Project name or ID | +| `branchName` | `VRT_BRANCHNAME` | no | Branch of the build. If not set, the build gets the project's main branch | +| `sourceBranch` | `VRT_SOURCEBRANCH` | no | Branch whose baselines a test without a baseline on `branchName` is compared against. If not set, the project's main branch | +| `ciBuildId` | `VRT_CIBUILDID` | no | ID of the CI build. Starting a build with an ID that already exists continues that build | +| `enableSoftAssert` | `VRT_ENABLESOFTASSERT` | no | Log `new` and `unresolved` results instead of throwing. Default `false` | +| `httpTimeoutInSeconds` | `VRT_HTTPTIMEOUTINSECONDS` | no | How long to wait for each response. Default `360`, because the server compares images while it handles the request; `0` means no limit. Connecting has its own 10 second limit | -
+Each option is taken from the builder (or constructor) first, then the environment variable, then `vrt.json` in the working directory (or the file passed to `configFile`). -Create config with builder +### Builder ```java VisualRegressionTrackerConfig config = VisualRegressionTrackerConfig.builder() - .apiUrl("http://localhost:4200") - .apiKey("F5Z2H0H2SNMXZVHX0EA4YQM1MGDD") - .project("003f5fcf-6c5f-4f1f-a99f-82a697711382") - .enableSoftAssert(true) - .branchName("develop") - .build(); + .apiUrl("http://localhost:4200") + .apiKey("F5Z2H0H2SNMXZVHX0EA4YQM1MGDD") + .project("Default project") + .branchName("feature/login") + .sourceBranch("develop") + .enableSoftAssert(true) + .build(); ``` -
- -
- -Set environment variables +### Environment variables -``` +```bash export VRT_APIURL=http://localhost:4200 export VRT_APIKEY=F5Z2H0H2SNMXZVHX0EA4YQM1MGDD -export VRT_PROJECT=003f5fcf-6c5f-4f1f-a99f-82a697711382 -export VRT_BRANCHNAME=develop -export VRT_ENABLESOFTASSERT=true +export VRT_PROJECT="Default project" +export VRT_BRANCHNAME=feature/login +export VRT_SOURCEBRANCH=develop export VRT_CIBUILDID=40bdba4 -export VRT_HTTPTIMEOUTINSECONDS=15 - +export VRT_ENABLESOFTASSERT=true ``` -
- -
- -Create vrt.json file in the working directory +### `vrt.json` ```json { "apiUrl": "http://localhost:4200", - "project": "003f5fcf-6c5f-4f1f-a99f-82a697711382", "apiKey": "F5Z2H0H2SNMXZVHX0EA4YQM1MGDD", - "branchName": "develop", - "enableSoftAssert": false, + "project": "Default project", + "branchName": "feature/login", + "sourceBranch": "develop", "ciBuildId": "40bdba4", - "httpTimeoutInSeconds": 15 + "enableSoftAssert": true } - ``` -
- -> [!NOTE] -> Final values, that will be used by VisualRegressionTracker, will be resolved as following: -> 1. If explicitly set while creating `VisualRegressionTrackerConfig` - use this value -> 2. Use value from environment variable if it exists -> 3. Try to get it from the configuration file - -
+Keep the API key out of git: commit a `vrt.json.example` and ignore `vrt.json`. - -### Create an instance of `VisualRegressionTracker` +## Usage ```java -VisualRegressionTracker visualRegressionTracker = new VisualRegressionTracker(config); +VisualRegressionTracker vrt = new VisualRegressionTracker(config); +// or new VisualRegressionTracker() to read the environment and vrt.json only + +vrt.start(); +try { + // Selenium: a file, bytes or base64 + Path screenshot = ((TakesScreenshot) driver).getScreenshotAs(OutputType.FILE).toPath(); + vrt.track("Login page", screenshot, TestRunOptions.builder() + .browser("Chrome") + .os("Linux") + .viewport("1280x720") + .build()); +} finally { + vrt.stop(); +} ``` -or +`start()` creates a build; every `track` call adds a test run to it; `stop()` finishes the build. -```java -VisualRegressionTracker visualRegressionTracker = new VisualRegressionTracker(); -``` +### Image -> [!TIP] -> If config is not provided explicitly, it will be created based on the environment variables or configuration file. Please see [Configuration](README.md#configuration) section +| Call | Upload | +|---|---| +| `track(name, Path image)` | multipart, file name kept | +| `track(name, byte[] image)` | multipart | +| `track(name, String imageBase64)` | base64 in JSON | -
+Each also takes a `TestRunOptions` as the third argument. Prefer a file or bytes: base64 in JSON is bound by the server's `BODY_PARSER_JSON_LIMIT` (5 MB in the server's shipped `.env`, Express's 100 kB when unset), multipart by the much larger `MULTIPART_FILE_SIZE_LIMIT` (100 MB by default). -### Start `VisualRegressionTracker` +### `TestRunOptions` -```java -visualRegressionTracker.start(); -``` +| Option | Description | +|---|---| +| `os`, `browser`, `viewport`, `device`, `customTags` | Part of the test's identity: a baseline is matched on name and all of these | +| `diffTollerancePercent` | Difference, in percent, still treated as a match (the spelling is the server's) | +| `ignoreAreas` | List of `IgnoreAreas` (`x`, `y`, `width`, `height`) left out of the comparison | +| `comment` | Comment shown on the test run | +| `merge` | Approving this test run in the UI updates the main branch's baseline instead of `branchName`'s | -At that point VisualRegressionTracker will try to create a new build for provided project. All of the subsequent tracked screenshots are going to be included in that build. +## Result and statuses -
+`track` returns a `TestRunResult`: -### Take a screenshot as String in Base64 format +- `getTestRunResponse()`: `getId()`, `getStatus()`, `getUrl()` (the test run in the VRT UI), `getDiffPercent()` and the rest of the server's answer; +- `getImageUrl()`, `getBaselineUrl()`, `getDiffUrl()`: the images, served through `/images/`; `null` when there is no such image. -```java -// Selenium example -String screenshotBase64 = ((TakesScreenshot) driver).getScreenshotAs(OutputType.BASE64); -``` +| Status | `track` | +|---|---| +| `OK`, `APPROVED`, `AUTO_APPROVED`, `FAILED` | returns | +| `NEW` (no baseline yet) | throws `TestRunException` "No baseline: ", or logs it with `enableSoftAssert` | +| `UNRESOLVED` (difference found) | throws `TestRunException` "Difference found: ", or logs it with `enableSoftAssert` | +| `UNKNOWN` (a status this SDK does not know) | logs a warning and returns | -
+## Errors -### Track image +When the server rejects a request, the SDK throws `TestRunException`. `getStatusCode()` holds the HTTP status and the message reads `Request failed with status : `. Calling `track` or `stop` before `start` also throws it, with no status code. -Default options +| Status | Meaning | +|---|---| +| 401, 403 | Wrong `apiKey`, or the user may not use the project | +| 404 | Project or build not found | +| 413 | Request too large. For base64 the message suggests multipart; for multipart it names `MULTIPART_FILE_SIZE_LIMIT` | +| 422 | Image has more pixels than the server's `MAX_IMAGE_PIXELS` | +| 503 | Server busy. With `Retry-After` the SDK waits and retries up to 3 times (at most 60 s per wait) before throwing | -```java -visualRegressionTracker.track( - "Name for test", - screenshotBase64 -); -``` +Network failures and timeouts surface as `IOException` (`HttpTimeoutException` for a timeout). -With specific options +## Releasing -```java -visualRegressionTracker.track( - "Name for test", - screenshotBase64, - TestRunOptions.builder() - .browser("Chrome") - .os("Windows") - .viewport("1200x800") - .diffTollerancePercent(3.0f) - .build() -); -``` +JitPack builds the SDK from a git tag; there is nothing to upload. -`track` throws `TestRunException` when there is no baseline (`NEW`) or a difference is found (`UNRESOLVED`), unless `enableSoftAssert` is on. A status the SDK does not know comes back as `UNKNOWN`, which is logged as a warning and never fails `track`. When the server rejects a request, `TestRunException.getStatusCode()` holds the HTTP status, and the message reads `Request failed with status : `. +1. Make sure `master` is green. +2. Tag the commit with the version, without a `v`: `git tag && git push origin `. +3. Create a GitHub release for the tag with the release notes. +4. Open `https://jitpack.io/#Visual-Regression-Tracker/sdk-java/` to start the build, and check that its log ends with the artifact list. +JitPack runs `./gradlew clean -Pgroup=com.github.Visual-Regression-Tracker -Pversion= -xtest assemble publishToMavenLocal` on the JDK from `jitpack.yml`, so the artifact version is always the tag. `version` in `gradle.properties` is only the default for local builds. CI runs the same command on Java 25 for every pull request. -### Stop `VisualRegressionTracker` +## Links -```java -visualRegressionTracker.stop(); -``` +- [Java examples](https://github.com/Visual-Regression-Tracker/examples-java) +- [Other SDKs and agents](https://github.com/Visual-Regression-Tracker) + +## License -Should be called once current build should be considered as completed. +[Apache-2.0](LICENSE) diff --git a/build.gradle b/build.gradle index 6423709..9df5113 100644 --- a/build.gradle +++ b/build.gradle @@ -57,6 +57,16 @@ publishing { publications { maven(MavenPublication) { from components.java + pom { + name = 'Visual Regression Tracker Java SDK' + url = 'https://github.com/Visual-Regression-Tracker/sdk-java' + licenses { + license { + name = 'Apache-2.0' + url = 'https://www.apache.org/licenses/LICENSE-2.0' + } + } + } } } }