TechPrufer
Get started
Microsoft
Microsoft logoMicrosoft · Feature trail

Visual Testing in CI/CD Pipelines

7 posts · Mar 2012 – Nov 2020

What this trail covers

This thread tracks the evolution of visual testing in CI/CD pipelines, detailing configuration for Azure Pipelines, GitHub Actions, and container-based pipelines. It covers integrating visual testing tasks, setting up environment variables, and using tools like Codefresh and JFrog Pipelines. The focus is on enabling visual tests within containerized environments, including the setup of Selenium Hub and nodes, and addressing challenges like local Chrome access and WebDriver configurations. The series also touches on HTTP checks and capturing build environment variables.

All Microsoft trails
  1. Things to consider when running visual tests in CI/CD pipelines: Container Pipeline Edition (Part 3)This post, the third in a series, details running visual tests in container-based CI/CD pipelines. It explains the benefits of containerized tasks, the challenges of not having local Chrome, and how to use Docker Compose with Selenium Hub and nodes. It provides specific YAML examples for Codefresh, including defining service containers (Selenium Hub, Chrome node, Firefox node), performing HTTP checks, setting Applitools environment variables, and running visual tests with Maven. It also offers guidance for Jenkins and JFrog Pipelines, and provides a sample project for local Docker Compose experimentation.
  2. Things to consider when running visual tests in CI/CD pipelines: Azure Devops & GitHub Actions (Part 2)This post provides concrete YAML examples for integrating visual testing into Azure Pipelines and GitHub Actions. It details the necessary tasks (HTTP Check, Capture Build Environment Variables, Run Visual Tests) and demonstrates how to configure environment variables like APPLITOOLS_API_KEY, RUNWHERE, and TEST_START_PAGE for both platforms. For GitHub Actions, it shows how to use `::set-env` to persist environment variables and includes a step for setting the Java version.
  3. Things to consider when running visual tests in CI/CD pipelines: Getting Started (Part 1)This post details the setup of a build server environment for running visual tests in CI/CD pipelines, specifically focusing on headless Chrome configuration on Ubuntu. It provides instructions for installing Chromium and optionally Java 13 and Maven. A Java example demonstrates how to capture environment variables like `RUNWHERE`, `TEST_START_PAGE`, `APPLITOOLS_BATCH_ID`, and `APPLITOOLS_API_KEY` to configure tests for local, pipeline, or container-based environments. The post also includes a sample Maven test execution command and a GitHub repository link for full code context.
  4. Announcing easy to use browser-based exploratory testing for Visual Studio Team ServicesIntroduced a browser-based exploratory testing extension for Visual Studio Team Services. This extension allows users to document exploratory tests, capture and annotate screenshots, file bugs, and test smart devices. It integrates with Perfecto Mobile for device testing. The extension is installed via the Visual Studio Marketplace and connects to a VSTS account. Features include image annotation tools, cross-browser tab and instance support, and device testing integration.
  5. Using Internet Explorer 11 and not able to record a web performance test successfully?Provides a workaround for recording web performance tests with Internet Explorer 11 by disabling 'Enhanced Protection Mode' and ensuring the 'Microsoft Web Test recorder 10.0' add-on is enabled. Also offers an alternative solution involving registry key deletion if the primary workaround fails.
  7. Using Microsoft Visual Studio 11 Beta Feedback Client tool with Windows 8 Metro Style applicationsThis post details the use of the Visual Studio 11 Beta Feedback Client tool with Windows 8 Metro style applications, specifically for remote testing scenarios. It explains how to connect to Windows 8 Metro style applications for testing using Remote Desktop Connection (MSTSC.exe) or the simulator integrated into Visual Studio 11 Beta. It also provides links to further documentation on running Windows 8 Metro style applications and using the feedback client.
  8. Getting Started with Exploratory TestingThis post details the use of Microsoft Test Manager for exploratory testing, enabling simultaneous test execution and design. It highlights features for capturing screenshots, adding comments, attaching log files, creating bugs with pre-populated repro steps, and generating test cases from exploratory sessions. The post emphasizes the automatic linking of created work items to parent backlog items and the creation of regression test suites.
  9. Visual Studio 11 Beta: Manual Testing of Windows Metro Style AppsThis post details enhancements to Microsoft Test Manager for testing Windows Metro style apps on remote devices. Key features include: specifying remote devices for testing, available data diagnostic adapters (Event Log, System Info, Action Log), installation of Metro style apps from Test Manager onto remote devices, and the Image Action Log which captures detailed steps with text, control thumbnails, and full screenshots. The post also discusses action log quality metrics (completeness and readability) and how they are achieved, and the support for exploratory testing on remote devices. Prerequisites for manual testing are also outlined.