TechPrufer
Get started
Microsoft
Microsoft logoMicrosoft · Feature trail

Azure Load Testing

53 posts · Mar 2012 – Mar 2023

What this trail covers

Azure Load Testing provides a fully managed service for generating high-scale load and identifying application performance bottlenecks. This includes support for creating custom load test plugins to extend functionality, such as dynamically adjusting user load during tests. The service integrates with Azure DevOps for CI/CD pipelines and offers various reporting and analysis capabilities. This post introduces the ability to specify where load test traffic is generated from, allowing users to col

All Microsoft trails
  1. Integrate Azure Load Testing into Azure DevOpsThis post details how to integrate Azure Load Testing into Azure DevOps pipelines, enabling load testing as part of CI/CD toolchains. It includes a link to a YouTube tutorial for configuration.
  2. Cloud-based load testing service end of lifeThis post announces the deprecation of the cloud-based load testing service in Azure DevOps, scheduled for shutdown on March 31st, 2020. It also details the deprecation of load test functionality in Visual Studio 2019 and the associated Test Controller/Test Agent for on-premises load testing. The post provides a timeline for these deprecations and suggests alternatives such as Azure Load Testing, Apache JMeter, and commercial tools. It also outlines how existing load tests can be migrated or run using self-hosted agents.
  3. Cloud-load testing service is hyper-scale ready: lessons from generating 1M concurrent user loadThis post details infrastructure changes to the Cloud Load Testing (CLT) service that enable it to generate up to 1 million concurrent user load. It highlights improvements in agent efficiency (2.5x more virtual users per core) and reduced resource acquisition time (over 50%). The post also delves into test settings, including the 'webtest connection model' (Connection Per User vs. Connection Pool) and provides a formula for calculating connection pool size based on dependent requests and server URI hosts. It discusses agent capacity planning and the use of performance counters like TCPv4 for hyper-scale testing.
  4. Run cloud-based load tests using your own machines (a.k.a. Bring your own subscription)This post introduces the 'Bring your own subscription' (BYOS) capability for Azure Load Testing, allowing users to provision and use their own machines (on-premises or Azure VMs) as load agents. It details two primary scenarios: more control over agent machines (e.g., installing custom software) and testing private apps behind firewalls. For Azure VMs, it highlights the use of ARM templates (simple dynamic/static IP, and VNet-based) for simplified provisioning. It also clarifies the differences between the cloud-load agent and the older Test Controller/Test Agent model, emphasizing the cloud-load agent's lack of a separate controller, direct integration with the CLT service for results, and HTTP(S) communication.
  5. Testing private/intranet applications using Cloud-based load testingThis post introduces solutions for load testing private/intranet applications that are not publicly accessible. It details five scenarios: using ARM templates to deploy load agents in a VNet, using ARM templates to deploy load agents with static IPs, using cloud load agents on customer infrastructure via a PowerShell script, and using Test Controller/Test Agents for on-premises testing. It provides links to ARM templates and GitHub repositories for deploying load agents in existing or new VNets, and explains how to queue runs using the VSTS portal or Visual Studio Enterprise IDE. It also covers managing self-provisioned agents with a PowerShell script and discusses VM sizing and agent group registration.
  7. Use cloud load agents on your infrastructureThis post introduces the capability to use user-provisioned machines (physical or VMs) as load agents for the Cloud-based Load Testing service. It provides a PowerShell script (ManageVSTSCloudLoadAgent.ps1) to configure these machines, requiring TeamServicesAccountName and a PATToken with 'Load Test (read and write)' scope. The script can configure agents, list agent groups and agents, and delete agents. It also details how to queue runs using these self-provisioned agents via the VSTS portal or Visual Studio Enterprise IDE by setting context parameters like 'UseStaticLoadAgents' and 'StaticAgentsGroupName'. The post clarifies that runs on user-owned agents are not charged, but Azure VM costs apply. It also addresses FAQs regarding communication, charging, machine usage, shutdown, proxy settings, and log collection.
  8. Speed up cloud-load test execution by retaining resources for quick consecutive runsIntroduced a feature to retain load agent resources for a configurable duration after a load test run finishes. This allows for quicker consecutive test iterations by reusing provisioned Azure resources, reducing the time spent on resource provisioning which can take up to 20 minutes. The feature is accessible via the 'Resources Retention' setting in Visual Studio 2015 Update 3 active run settings, or by adding a 'ResourcesRetentionTimeInMinutes' context parameter in older versions. Resources can be retained for a maximum of 240 minutes. A small surcharge of 5 VUMs per core per hour is applied for this feature. The feature is currently only available for load tests triggered using Visual Studio and will be brought to the web portal for other test types like Apache JMeter soon.
  9. Troubleshooting an HTTP archive-based load testThis post introduces the capability to create and run load tests using HTTP archive (.har) files within the Azure cloud-based load testing service in VSTS. It provides troubleshooting guidance for request failures by detailing the use of the 'Diagnostics' and 'Logs' tabs, and explains how to export test results in HAR format for viewing in tools like Fiddler. It also addresses the handling of dynamic parameters in requests and the process of exporting tests to Visual Studio for more advanced fixes.
  10. Feature Preview: Creating load tests using HTTP archiveThis post introduces a new capability within Azure Load Testing (then VSTS portal) to create load tests by importing HTTP archive (.har) files generated from browser developer tools or Fiddler. This allows users to simulate more complex user scenarios than simple URL-based tests without needing Visual Studio Enterprise. The process involves recording traffic, saving it as a .har file, and then importing it into the VSTS Load test hub. The system automatically extracts URLs, HTTP methods, headers, and parameters, and attempts to correlate dynamic information like ASP.NET view state. Data-driving and comprehensive dynamic information correlation are noted as future backlog items.
  11. Trying out the cloud load testing service using Visual Studio Enterprise trialThis post illustrates how to use Visual Studio Enterprise trial to learn about load test capabilities in the VS IDE, and how to leverage the free 20,000 VUM monthly quota of a VSTS account to see cloud-based load testing in action. It details the steps for downloading and installing the Enterprise trial, signing in to activate the trial, creating a VSTS account, and downloading a sample load test project to complete a walkthrough. It also addresses FAQs regarding basic access in VSTS, alternative load testing options for non-Visual Studio users (Apache JMeter), and options for obtaining an Enterprise license.
  12. Performance Testing with App Service Continuous DeploymentThis post details the integration of performance testing with continuous deployment for Azure App Services. It outlines the steps to configure continuous deployment with a chosen source control and then set up performance tests as a post-deployment command. Users can log in to the Azure portal, navigate to App Services, configure continuous deployment, and then set up performance tests. The post shows how to execute these tests after pushing new changes and how to view the test results. It also provides instructions for adding performance tests to existing continuous deployments.
  13. Cloud-load testing: rich reports in the webThis post introduces rich reporting capabilities in the web for cloud-load tests, allowing users to analyze runs, compare results, and share them with their team. Reports include summary views with key performance indicators, charts for performance and throughput, and diagnostic information for test errors. It also highlights the 'Top 5 slowest requests' and 'Top 5 slowest transactions' for quick identification of problem areas. The comparison feature allows users to compare current runs against previous baselines.
  14. Running Apache JMeter based load tests in the cloud - how toThis post announces the public availability of running Apache JMeter based load tests in Visual Studio Team Services (now Azure DevOps). It provides a step-by-step guide for users to create JMeter tests, upload them along with supporting files, configure the number of agents for scaling load, set the load duration, and select load generation locations. It also details how to view live reports during test execution, access diagnostics for errors, and download results and logs post-test. The post also includes an FAQ addressing JMeter version support, sampler support, property configuration, custom listener support, VUM calculation, and other cloud-based load testing options.
  15. Adding Performance/load test to Azure Web and Mobile App Continuous DeploymentThis post details the integration of performance/load testing into Azure App Service Continuous Deployment (CD) workflows. It explains how to configure Azure App Service settings with VSTS PAT tokens and URLs to trigger performance tests automatically after each deployment. It also references existing functionality for performance testing within App Service Plans and provides links to relevant documentation for continuous deployment and VSTS account creation.
  16. Cloud Load Test Build Task informationThis post introduces the Cloud Load Test Build Task for Visual Studio Team Services, enabling the execution of cloud-based load tests as part of CI/CD pipelines. It details the task's arguments, including connection setup, test file paths, and threshold violation configuration. It also outlines the demands on the build agent (MSBuild, Azure PowerShell) and provides links to further resources on load testing and build task creation.
  17. Creating Load Test Plugins for Cloud-based Load TestThis post details how to create custom load test plugins for Azure Load Testing (and on-premises load testing) by implementing the ILoadTestPlugin interface. It provides a step-by-step guide with code examples for creating a plugin that randomly increments user load during a test, demonstrating the extensibility of the load testing framework for custom scenarios and load profiles.
  18. Scheduling Load Test ExecutionThis post details how to schedule load test execution within Azure DevOps pipelines. For cloud-based load tests, it describes creating a new build definition, setting up a generic service endpoint using a PAT or alternate credentials, and configuring 'Quick Load Test Task' and 'Load Test Task'. For on-premises load tests, it explains using the command-line executable 'mstest.exe' with various options like /TestContainer, /runconfig, and /resultsfile, and suggests using Windows Task Scheduler for automation.
  19. Private Preview: Running Apache JMeter load tests in the cloudThis post announces the private preview of Cloud-based Load Testing, allowing users to run Apache JMeter based tests using the Azure cloud. It highlights the ability to generate load from 10 different locations globally, scale load easily, and view live results and summary reports. It also mentions the option to download results for offline analysis. The post also briefly touches on other capabilities like simple URL load tests and Visual Studio based load tests.
  20. Announcing Public Preview for Performance/Load testing of Azure WebappThis post announces the public preview of a performance testing feature for Azure Web Apps, integrated directly into the Azure Portal. It allows users to associate a Visual Studio Online account to create and maintain a history of performance tests. Users can specify the target Web App URL, choose the load generation location, and view results including request pass/fail rates and average response times. The post also details usage limits during the preview and pricing for general availability.
  21. Load Testing from different geographies with Cloud-based load testingThis post details a method for distributing load across different geographic locations in Azure Load Testing, even without a direct UI multi-select feature for locations. It explains how to achieve this by creating multiple copies of a load test, each configured with a specific geographic location and user count, and then executing these tests either manually or via the command line. It also references a related post on using mstest.exe for command-line execution and XAML builds.
  22. Cloud Load Test Support in mstest.exe command line and XAML BuildsThis post introduces the integration of cloud load testing capabilities with mstest.exe command line and XAML builds in TFS 2015. It details how to configure test settings files with account URLs and alternate credentials (both unencrypted and encrypted) to execute load tests against the cloud load test service. It also explains how to integrate these settings into XAML build definitions to trigger cloud load tests as part of the build process.
  23. Announcing support for Goal-based Load Pattern in Cloud-based Load TestsIntroduced support for goal-based load patterns in Cloud-based Load Tests with Visual Studio Online. This pattern adjusts user load based on performance counter thresholds (agent or load test counters) rather than periodic user load adjustments. The post details the properties for configuring goal-based load patterns, including performance counter categories, target ranges, and user count limits, and provides screenshots of the configuration interface. It also mentions ongoing work to support server/app counters.
  24. Collecting Advanced Agent Diagnostics Data in Cloud-based Load TestThis post introduces the ability to collect advanced agent diagnostics data directly from the Visual Studio UI for cloud-based load tests. Previously, debugging cloud agents was difficult due to inaccessibility. This new feature allows users to queue load tests, access a 'Test Attachments' section in the 'Details' tab, and view/download diagnostic files generated by unit tests (redirected to the 'Results Directory') or trace statements. It also highlights the 'Traces.log' file containing all trace logs from unit tests and mentions a UI limit of 2000 logs.
  25. Generate load test traffic from any Azure Datacenter around the worldIntroduced the ability to specify the location from where load test traffic is generated, allowing selection from 10 Azure Datacenter locations. This feature can be set in the load test wizard or as a property in the load test run settings. It is also available from Visual Studio Online. Performing geo-targeted load tests helps simulate load with minimal latency by choosing a datacenter closest to the deployment.
  26. Load Test Enhancements in Release Candidate of Visual Studio 2015 Enterprise editionIntroduces the ability to specify the geographic origin of load test traffic, a highly requested feature. This can be set during test creation or in the load test run settings.
  27. Generating excel reports for load test runs done in between 03/19-03/31This post addresses a bug in the Excel report generation for Azure Load Testing runs performed between March 19, 2015, and April 1, 2015. The issue stemmed from a failure to update cumulative values for performance counter samples during recent changes to the results storage system. A SQL script is provided as a workaround to recalculate these cumulative values directly in the customer's results database, enabling correct Excel report generation for affected runs. The fix for the service is stated to be coming soon.
  28. Announcing 100 core support for VSO Cloud-based Load testing and more!This post announces an increase in the core support for Visual Studio Online (VSO) Cloud-based Load Testing from 20 cores to 100 cores for a single load test run. This allows for generating up to 5 times the concurrent users compared to the previous limit. Users can specify the desired number of cores (up to 100) in the 'Run Settings' properties sheet of their load test file. It also links to related posts about load testing applications behind firewalls and using advanced diagnostics for debugging.
  29. Load testing Applications behind Firewall using Cloud-based Load Testing ServiceThis post addresses the challenge of load testing applications deployed internally behind a firewall. It introduces an approach that allows cloud-based load testing agents to securely reach these private applications. A link to a separate blog post provides the detailed methodology for this specific scenario.
  30. Initiate a Load Test from Hosted Build work-aroundThis post details a workaround for initiating load tests from hosted builds in Visual Studio Online (VSO) using a post-build script and the Cloud Load Testing REST API. It outlines prerequisites (VSO account, alternate credentials, Visual Studio Ultimate 2013), provides a 10-step process involving packaging a REST API sample and a launch script, configuring build definitions, and monitoring the load test execution. The workaround addresses limitations of directly triggering load tests from hosted agents.
  31. Using advanced diagnostics to debug issues in Cloud Load Test.Introduces advanced agent diagnostics for Cloud Load Test (CLT) to facilitate debugging. This feature allows users to collect logs (e.g., trace statements, packet captures via Wireshark) and other files from test agents and download them from an Azure Blob. It details how to use trace statements in unit tests, load test plugins, and web test plugins, and how to copy files to a designated 'Result Directory' on the agent. The post also explains how to access these collected files from blob storage using REST APIs and provides code examples for downloading them. It notes a 100MB size limit per agent for collected data.
  32. Using Setup and Cleanup Script in Cloud Load TestThis post details how to use setup and cleanup scripts within Visual Studio Ultimate's test settings to configure the environment of agents running cloud-based load tests. It provides specific examples for installing certificates (using CertMgr.exe and certutil.exe) and software (like Network Monitor or WebDeploy) on these agents. It also explains how to add these scripts and necessary files (like certificates or installers) as deployment items within the test settings.
  33. New features added to simplified browser-based Load Test authoringThis post announces new features for the simplified browser-based load testing experience: 1. Real-time view of application performance during a run, showing five different performance counters. 2. Tiles indicating how well the app performed under load compared to industry standards. It also references previous capabilities and encourages feedback.
  34. Getting started with Cloud-based Load Test REST APIsThis post provides a simplified example of using the Cloud-based Load Test REST APIs to interact with Visual Studio Online. It demonstrates how to enable alternate credentials, make authenticated HTTP GET requests to retrieve test drop information using HttpClient in both ASP.NET and a console application, and configures the necessary App.Config settings. It also highlights common error scenarios like HTTP 401.
  35. Cloud-based Load Testing from Europe is now possibleThis post announces the availability of cloud-based load testing from Europe via Visual Studio Online (VSO) accounts created in Europe. The load testing infrastructure will be spawned in the Azure West EU Datacenter, and load test data will reside there. This addresses EU data sovereignty concerns and allows for load generation from multiple datacenters, including the EU.
  36. Cloud-based Load Testing REST APIs are hereIntroduced REST APIs for Cloud-based Load Testing (CLT) with Visual Studio Online. These APIs allow users to start/stop load test runs, retrieve load test results (KPIs, performance, throughput, application metrics), get messages and exceptions during runs, retrieve counter instances and samples, and list past load test runs. The post also details API usage requirements, including enabling alternate credentials, specifying the correct service endpoint (e.g., `https://abc.vsclt.visualstudio.com/`), including the `api-version` parameter, and outlines the steps for creating and running load tests using these APIs, including uploading test files to Azure Blob storage.
  37. Announcing a simplified browser-based authoring and configuration experience for Load TestingIntroduced a simplified browser-based authoring and configuration experience for Azure Load Testing, allowing users to create and run load tests with just three options: target URL, number of users, and duration. This reduces the barrier to entry for performance testing. The post also highlights the continued support for the full authoring experience in Visual Studio Ultimate for more complex scenarios and mentions the integration with Visual Studio Online for viewing and comparing load test reports.
  38. Visual Studio Cloud Load Testing & Error codesThis post introduces error codes for Visual Studio Cloud Load Testing to improve customer conversations and issue reporting. It lists specific error codes (VS1550001-VS1550061) and their corresponding error messages, detailing various failure modes such as timeouts, agent unresponsiveness, provisioning errors, and issues with Application Insights integration. It also outlines limitations specific to running load tests in Visual Studio Online, such as maximum agent counts, unsupported test types, and network types.
  39. Data-Driven Load Tests using Visual Studio Online and SQL AzureThis post details how to perform data-driven load tests using Visual Studio Online and SQL Azure. It explains the importance of adequate stress testing with representative data, the cost barriers to traditional load testing, and how cloud services like Visual Studio Online and Azure SQL Database can mitigate these issues. The article provides a step-by-step guide for setting up data in Azure SQL Database, creating data-driven web tests in Visual Studio, and integrating these tests into load tests for cloud execution. It covers scenarios like simulating dynamic URLs, filling forms with different data, calling web services with varying parameters, and simulating AJAX calls, with a specific focus on dynamic URL generation for e-commerce or CMS sites.
  40. Load Testing with Visual Studio OnlineThis post announces the integration of Visual Studio Online load testing with Application Insights for enhanced performance monitoring during load runs. It also highlights the free tier of 20,000 user minutes per month for Visual Studio Online accounts and provides links to documentation, guides, and pricing information.
  41. Error importing the result file after a load test runThis post addresses a specific error encountered during load test runs in Visual Studio: 'There was an error importing the result file after a load test run.' The root cause identified is the load test repository's SQL Express database filling up. The post explains the default location of this database and its size limits (4GB or 10GB depending on SQL Express version). It provides a solution by recommending the execution of the `loadtestresultsrepository.sql` command to manage the repository.
  42. Binding a Cloud Load Test to a data sourceThis post details how to bind a Web Performance Test to a data source (CSV or XML) for use in Cloud Load Tests within Visual Studio Ultimate. It covers creating the web test, cleaning up recordings, adding and configuring data sources, data binding parameters, setting run counts in local.testsettings, and creating and running a load test, specifically highlighting the 'Run Test using Visual Studio Online' option for cloud execution. Caveats regarding data source traversal settings in cloud tests are noted.
  43. Get Application Performance data during load runs with Visual Studio OnlineThis post details the integration of Azure Load Testing with Application Insights to collect and analyze application performance data during load runs. It outlines the steps for configuring this integration, including verifying cloud-based load testing, ensuring the latest Visual Studio updates, installing Application Insights on the target machine, connecting the Visual Studio IDE to Visual Studio Online, and adding application monitoring to load tests. It highlights that pre-selected counters like CPU, Memory, PerformanceEventsPerSec, and ExceptionsPerSecond are available, along with the option to select NetworkIO and DiskIO. The post also explains how to perform root cause analysis using the Application Insights link.
  44. Cloud Load testing with Visual Studio Online - Launching General AvailabilityThis post announces the General Availability of cloud load testing using Visual Studio Online. It provides links to download Visual Studio Ultimate 2013 RTM and Update 3 RC, a getting started guide and video, and a troubleshooting guide. It highlights the integration with Application Insights for monitoring during load runs and provides case studies and examples of load testing. It also details free usage limits (15,000 virtual user minutes per month) and applicable charges for exceeding these limits.
  45. Using Visual Studio Online to Load Test the NORAD tracks Santa websiteThis post details the use of Visual Studio Online (VSO) Load Testing Service to test the capacity of the NORAD Tracks Santa website's Timer Service. It explains how to create web tests by recording requests (using WCF Test Client and Fiddler for web services), and how to add them to a load test. The post emphasizes simulating multiple 'real life' users with fewer virtual users by removing think times, which is crucial for cost-effectiveness in VSO Load Testing (charged by Virtual User Minute). It also covers adjusting the number of test agent cores for high resource utilization tests and the process of running tests in the cloud with a single click. The results showed the system could handle a maximum of 75 virtual users and nearly 5,000 RPS with an average CPU of over 75% on a single instance. The analysis of results led to informed decisions about instance allocation and performance improvements, increasing throughput and reducing CPU utilization, ultimately impacting cost savings.
  46. Load testing with Visual Studio Online - Launching Commercial PreviewThis post announces the commercial preview of cloud-based load testing using Visual Studio Online (formerly Team Foundation Service). It provides links to download Visual Studio Ultimate 2013 RTM, a getting started guide and video for load testing with Visual Studio Online, and a troubleshooting guide. It also highlights case studies and details free usage limits (15,000 virtual user minutes per month) and charges during the preview period ($0.001 per virtual user minute).
  47. Known issues with load testing in Visual Studio 2013This post details known issues and workarounds for load testing in Visual Studio 2013, specifically addressing problems encountered when using cloud-based load tests with Team Foundation Service (hanging Visual Studio, failing result imports) and issues with recording web performance tests after upgrading to the RTM version. It provides specific registry key deletions and links to external articles as workarounds.
  48. Load Testing Siebel Applications using Visual StudioThis post details the process and challenges of load testing Siebel applications using Visual Studio. It covers recording tests using Visual Studio's web recorder and Fiddler, authoring load tests, and specifically addresses common challenges and solutions for playback failures related to Siebel's dynamic nature, including issues with SWERowId, date formats, and SWEC/SWETS parameters. It also provides best practices and additional tips for understanding Siebel architecture and dynamic parameters.
  49. Load testing with Team Foundation Service - Launching Preview and Early Adoption ProgramThis post announces the commercial preview of cloud-based load testing with Visual Studio Online, now available to all Visual Studio Online users. It provides links to download Visual Studio Ultimate 2013 Preview, a getting started guide, troubleshooting guide, forum, and a Tech-Ed session recording. It also details usage limits (15,000 virtual user minutes per account per month during preview) and mentions an Early Adoption Program for users requiring more capacity.
  50. Content Index for Visual Studio Web Tests and Load TestsThis post serves as a content index for Visual Studio Web Tests and Load Tests, consolidating information from various sources and versions (VS2010, VS2012). It highlights the deprecation of load testing functionality in Visual Studio and the retirement of the Azure DevOps cloud-based load testing service, directing users to Azure Load Testing as the successor. It also provides links to resources for web performance and load testing, including SharePoint applications, WCF services, and general guidance.
  51. Getting Started with Load Testing in Visual Studio 2012This post details how to get started with load testing in Visual Studio 2012. It explains the process of creating a new 'Web Performance and Load Test Project', adding load tests, and addresses confusion around running these tests with the new Test Explorer. It also provides a solution for the 'Could not connect to load test results database' error by detailing how to execute the `loadtestresultsrepository.sql` script to set up the SQL Server database for storing load test results.
  52. Visual Studio 11 Beta - Changes in Web Performance Test / Load Test WorkflowThis post details changes in the Web Performance Test / Load Test workflow in Visual Studio 11 Beta, highlighting a new, separate test project type for Web Performance Test / Load Test.