Skip to content

Add Visual Regression Testing capability #447

@giulong

Description

@giulong

We'd like the user to be able to run visual regression testing. When configured:

  1. the first run is used to generate snapshots (golden images)
  2. the subsequent runs will compare the screenshots taken during each with the snapshots.

The snapshots should be taken in the src/test/resources folder by default, so to be versioned. Screenshots should be either taken:

  • programmatically (user takes screenshot explicitly)
  • automatically upon user's actions (clicking buttons, typing text etc)

The configuration snippet should look something like this:

visualRegression:
  failFast: false # Whether to fail immediately when the first visual regression is found, rather than running the entire test
  snapshots:
    folder: src/test/resources/visual-regression # Where to save the screenshot references
    override: false # Whether to override the snapshots references already generated
  checks: # Checks configuration
    count: 0 # Number of checks to perform before considering a snapshot valid
    interval: .5 # Interval between checks in seconds
    maxRetries: 3 # Max retries before throwing an exception

Tasks:

  • check multiple times before saving reference image
  • provide a parameter to force the snapshots update
  • provide a parameter to fail fast
  • work with autoBefore and autoAfter screenshots, relying on the EventsWebDriverListener or a dedicated WebDriverListener
  • inline visual regression images in html report
  • support @TestFactory
  • provide an exception with a message counting the total number of regressions

Metadata

Metadata

Assignees

Labels

enhancementNew feature or request

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions