Using the Azure Load Testing service

What is load testing, and why is it important?

You have designed, planned, and built your application, and it's now operating in your environment. Next up is understanding how the application behaves during expected, and perhaps unexpected, load.

Load testing is generally considered a practice of testing the expected usage of your application(s) when multiple users are accessing it simultaneously.

For example, you designed a new web app to serve customers or end-users with specific tasks. Perhaps it's an order form, booking system, an entire SaaS product suite, or something else. How do you know that it can handle simultaneous user load as it scales up and your service grows, or if you have traffic spikes during specific events?

In these situations, a good plan for load testing can help you identify bottlenecks before the end-users discover them.

I often see Azure solutions deployed without regard for reliance, reliability, operational excellence, and scalability. In many cases, applications are deployed and seem to work, but the issues often come when there is an increased load.

I've identified common pitfalls in many solutions, specific to Azure solutions in this case, but relevant regardless of tech stack.

• The system doesn't automatically scale as demand increases, leading to CPU and memory exhaustion.
• The system doesn't handle retry-actions with exponential back-off, causing it to drop important calls or events, eventually causing application crashes or unexpected behavior.
• Connectivity to dependent services like the Azure Key Vault isn't done according to best practices, leading to throttling and exceptions.
• Service leaks are injected across multiple user sessions, eventually causing inconsistencies and crashes.

There are many more common pitfalls when designing your solutions, but this list should hint at some of the common real-world challenges I have faced.

With proper load testing, you can identify several of these bottlenecks and ensure the system scales accordingly and can handle the load - and also ensure that dependent services are correctly being used and do not cause dependency failures as the system scales up.

There are a few things that load testing helps with, and the above.

• Test your scalability. Can you handle an increase in traffic? How quickly?
• Test how your application performs under stress. Do your application and the dependent services work as expected?
• Test your signals and monitoring! The signals, logs, and monitoring data will significantly increase when the load increases. Can you still make sense of this information, or do you need to adjust how you log and monitor your system?
• Verify that you get your alerts. Can you get the alerts you expect for unexpected outages, dependency failures, scalability issues, resource exhaustion (CPU/Memory/Disk IO, SNAT port exhaustion, and more)?
• Does your load balancer work? Is traffic successfully balanced between instances and regions?
• Measure your server-side metrics. See how the services you use to support your solution behave and any flags being raised during load.
• Measure your client-side metrics. Track response times, the number of requests happening per second, how many users are being loaded onto the system - this helps in correlating the data with what happens on the service side.

I often see "fire and forget" deployments, demo-code, and solutions that work well on a development machine or even in a QA system with minimal users and load. When the solutions hit prime-time and get battle-tested, the results usually look different.

While this is not an exhaustive list of things you should check, I hope it paints a picture of the importance of load testing your applications.

What is the Azure Load Testing service?

I have set the scene. You know why load testing is essential and how it can help you understand how your system behaves under stress and heavy load.

Let's take a moment to explore the Azure Load Testing service and how it helps us achieve a more reliable solution.

If you want to tune in and listen to a podcast about this service:
Episode 115 - We took the Azure Load Testing service for a spin
- Ctrl+Alt+Azure Podcast

Microsoft published an excellent overview to understand how the Azure Load Testing service works.

Azure Load Testing service functionality - from Microsoft Docs

The critical points for why I'm currently evaluating the Azure Load Testing service:

• Ease of use.
• Take any JMeter script and plug it into the load test.
• Measure both the client-side and server-side metrics during load tests.
• Configurable with virtual users and simulated load.
• Integrate with Azure Pipelines or GitHub.
• Can use managed identities in Azure.
• Supports many Azure resource types for server-side monitoring.

The documentation is very elaborate, and therefore I don't need to repeat what's already been stated there.

Let's set it up

The documentation around creating Azure Load Testing service accounts and setting up tests is elaborate. I'll just run through the basics to show how easy this is to get started with - then you can take it further in your organization and turn the volume up.

Create the Azure Load Testing service

You now have a new service called "Azure Load Testing" from the Azure Portal that you can provision into your Azure resource groups.

Azure Load Testing, as seen in the Azure Portal.

Per usual, define your resources and create them in any available locations.

Create a new Azure Load Testing resource in your given resource group and location from the Azure Portal.

Provisioning is done - that's the easiest part. Next, when the service is created, your need to explicitly configure some new permissions.

Grant the appropriate Load Test role to your user(s)

To work with and create new Load Tests with the Azure Load Test service, you need to assign at least one of the Load Test roles to your users or groups.

If you have not been granted the Load Test roles, you may see the following message:

You are not authorized to use this resource. You need to have the Load Test Owner, Load Test Contributor, or Load Test Reader role. To assign Azure roles, you need to have Microsoft.Authorization/roleAssignments/write permissions such as User Access Administrator or Owner. In case the role was granted recently, please refresh the page and try again. Learn more

Go to your new Azure Load Testing service and select "Access Control (IAM)", and then add any of the desired "Load Test" roles.

There are currently three available roles, and the names are self-explanatory.

• Load Test Reader
• Load Test Contributor
• Load Test Owner

Since I want to perform all operations, settings, and further configurations in my Azure Load Testing instance, I will assign the Owner role to myself.

The Azure Load Testing service requires one of the Load Test roles to be assigned to a group or user.

Quickly reviewing the permissions on the resource should now show that you have one of the Load Test roles assigned.

The Azure Load Test service has roles assigned to some users.

Create a new test in the Azure Load Testing service

To set up a new load test, you have one main requirement. You need a JMeter test file.

For instructions on how to use JMeter, check this out:

• https://jmeter.apache.org/usermanual/jmeter_proxy_step_by_step.html

I will not talk about how to create JMeter files, as there are many available resources for that purpose. Instead, let's review how to make the test and eventually see the results.

Click "Create test" to start configuring a new load test.

Create a new test in the Azure Load Testing service.

Follow the instructions per the wizard, and upload your recorded JMeter script, which will be in the .jmx format.

Upload an Azure Load Testing service script based on JMeter (jmx script).

Some notes about secrets and sensitive parameters

In the next step of the wizard, you can add parameters. Parameters can, for example, include Environment variables or Secrets if needed for your script to execute. For example, username/passwords, tokens, API keys required for a web request, etc.

Using secrets in JMX, use the GetSecret function to pull out the value of the secret when the test is executing. You need to perform that on the Jmeter side when designing your scripts.

There is also a built-in integration to Azure Key Vault to parameterize load tests with secrets from the Key Vault.

• Read more: Conduct configurable load tests with secrets and environment variables (Microsoft Docs)

Moving on, configure the load. The load setting determines how many test engines to spin up for the test. The engines multiplied with the number of threads defined in the JMX script equals the number of total threads - a great way to scale up the load testing.

Define the number of engine instances for the Azure Load Testing service.

Next, you need to determine and define the test criteria. These are usually metrics that decide if the test fails or not.

Define test criteria for the Azure Load Testing service.

Connect monitoring

When you reach the "Monitoring" step, it becomes more interesting again. We can connect any back-end service in Azure from this page, like Application Insights or App diagnostics, or metrics from Azure Storage Accounts.

The Azure Load Testing service has a great USP (Unique Selling Point) in that it directly integrates with many of the supporting services to our web apps. Using the integrations to monitor these services during load gives us a better understanding of how the overall system is impacted - not just the web application metrics.

Connect monitoring services to the Azure Load Testing service.

Connect the test to any of the monitoring resources.

• Read more: Monitor server-side application metrics for load testing (Microsoft Docs)

Review and create

Creating the load test will bring you to the review and eventually generate the test. New tests usually start to run within a few moments, and then you can start seeing the results as the test progresses and puts your system under load.

The Azure Load Testing service is creating new tests.

Review the results

When the load tests have finished, we can see the test's metrics, including errors and performance - and all the server-side metrics from the connected services.

The connected monitoring is the true benefit of using Azure Load Testing. I have used many load testing platforms in the past, but I never got immediate insights from all my dependencies.