What stands behind a quality application or service?

For developers, that is probably making sure that the application or software that is delivered works as intended. To achieve that, there are a few issues that must be tested and handled before the release:

• Is the system failing in any circumstances? What is the reason?
• Is it fast?
• Can it work under pressure? (I.e. does it work well even when the load is extensive?)
• Are there any security concerns?
• Is the response time satisfactory? What about CPU usage? Are throughput and latency working good enough during all occasions? (average vs. heavy load)

Some of those problems can be identified early in the development, while performing unit and integration tests during CI/CD process. But most of the time, this kind of testing will not find bottlenecks and unexpected behaviors that happen during high levels of traffic. Before we can be sure that our app is free of those, we must test it under load. Load testing can help figure out all the capabilities (and incapabilities) of a certain application or its part, no matter if it is serverless or server based.

Let us jump into practical knowledge and show how to load test your application or service, while keeping the cost of load testing low. Also, for the purpose of demonstration, we will be using Amazon Web Services as our cloud platform.

What tool should you use?

So how can we load test at scale without having a high-end testing infrastructure? To save the day (and your wallet), here comes Artillery.io. It is an open-source tool for testing APIs. Artillery makes it easy to test your services for performance and functionality quickly, easily and without need to maintain any servers or testing infrastructure. If we run Artillery in cloud, we can then bypass limitations of our own resources.

Artillery.io

Artillery is a modern, powerful, and easy to use performance testing toolkit. It is designed for testing backend systems, such as API services or application backend, regardless of what protocol it speaks or what language in which it is written. Artillery supports HTTP, WebSocket, and Socket.io out of the box, and can be configured to support other protocols as well.

Artillery is typically used across teams responsible for delivery, testing, and operating production backend systems. It is used in various scenarios, which include:

– Running ad-hoc load tests against individual APIs or microservices as part of a development process to explore their performance characteristics, and make performance optimizations if needed

– Running tests against staging/feature environments as part of a CI/CD pipeline

– Large-scale load testing before production release, or to prepare systems for heavy traffic

Artillery test for ecommerce backend

Artillery consists of Artillery Code – a free, open source tool, which can be used to write and run load and functional tests, and Artillery Pro – a commercial package for enterprise teams which adds additional features. In this post we will show free version of Artillery. To eliminate any constraints of our local network and computing power, we can run Artillery on AWS Cloud9 instance.

AWS Cloud9

AWS Cloud9 is a cloud-based integrated development environment (IDE) that offers code editing, running, and debugging with just a browser. It includes a code editor, debugger, and terminal. Also, it is prepackaged with essential tools for popular programming languages like Python and JavaScript. AWS Cloud9 comes with a terminal that includes privileges to the managed Amazon EC2 instance which is hosting your development environment and authenticated AWS Command Line Interface. This makes it easy to quickly run commands and directly access AWS services. If the backend is hosted on AWS, we can combine Artillery and Cloud9 for cheap and effective load testing tool.

To create Cloud9 instance:

1. Log into your AWS account and go to the Cloud9 console

2. Click the “Create environment” button

3. Name your instance (you can add description too if you want)

4. Configure environment settings – for this tutorial following is used:

• Environment type: direct access
• Instance type: t2.micro
• Platform: Amazon Linux
• Cost-saving setting: After 30 minutes (default)

5. Review name and setting, then click the “Create environment” button.

If everything went well, you should see a loading screen. When the setup is done, Cloud9 instance should be ready for usage, and your screen should look something like this:

AWS Cloud9 welcome screen

Load testing serverless endpoint

For this post, we created a serverless endpoint that will be load tested using Artillery. This endpoint will accept username and password as an input, and it will write that data into DynamoDB database. Although not very useful, this example will show you how to test endpoints quickly and easily.

The Artillery configuration file test.yaml is configured to create ten requests per second over 2-minute duration. This equates to 600 requests per minute, which is probably higher than endpoint like this would get. Our payload will be read from users.csv file, with each row representing one user.

test.yaml

We execute the Artillery test with the command artillery run test.yaml. Our test concludes with the following results:

Over 1200 requests, the median response timer is 55.5 ms. The p95 response time shows that 95% of all responses are served within 87 ms. The slowest response of 239.4 ms is caused by AWS Lambda cold starts.

Since this process inserts data into DynamloDB table, we can also see how many write capacity units (WCUs) are consumed by the test. Go to DynamoDB console, select your table, and click under Metrics tab. There you should see several different graphs. If you select Write capacity one, you should see results like this:

Under heavy load, DynamoDB increases WCU

Conclusion

In this blog post, we explored cheap and easy ways to perform load testing on applications. Artillery presents itself as a versatile and cheap tool for load testing backend systems. It is free, fast, and easy to configure and include in existing testing suite solutions. The proverb “time is money” fits here perfectly, with Artillery saving both!