I’m writing this to post on uTest, a testing outsourcing platform. I have wanted to get a testing job for a while, but lacking anything on my resume that makes me look like a tester (rather than a programmer or a general dilettante), seems to get me nowhere. So, uTest seems like as good an avenue as any to learn formal testing. It’s not as magical as I thought it would be, so far… but that’s the case with most IT tasks, right?
This article is a riff on …Successful API Testing, by Kankaria. I’ve coded up APIs, and the process lends itself to test automation, so I’ll go over how I do it.
Use and abuse a JUnit-like testing framework.
Every language has one, and it’s intended for unit testing functions or methods, but I will rig it up to make API calls to endpoints. The easy way is to use a www client library, like Python’s Request, Perl’s WWW:Mechanize, PHP’s Guzzle, or (my preference) PHP’s CURL extension. Then, instead of making it part of the unit tests, make it a separate suite of tests. It’s not one of the unit tests.
Go ahead and insert data.
A typical test involves testing not only that the API call was made, but that it succeeded, and affected the environment. So you insert via the API, and then check that the remote system has been altered, by either inspecting the database directly or by requesting the data back from the API. I prefer to inspect the database directly. This isn’t a regular “unit test”, but a kind of integration test that’s written using a unit testing framework.
Write the spec, and write the test based on it.
API’s are published interfaces, and require documentation. You should design the API, and write the docs, before you write the code. This is the common admonition for writing many kinds of programs, and most programmers don’t follow the rule, due to deadlines. With API’s, however, it’s a lot more important, because it’s the way other programmers are going to understand your API. The documentation is really your product; it’s also your sales pitch. No docs = no sales.
Abuse the API by going beyond your docs.
All APIs should list restrictions: they should have and enforce restrictions. It’s completely correct to say that an input has a limited length, and then fail when the limit is exceeded. It’s completely correct to enforce data types, and character set limits. This is a network API, not a code library.
Moreover, flexibility is a security problem: if it’s not some kind of buffer overrun, or an SQL injection attack, it’s going to lead to a Denial of Service attack. This is a network API, shared by many computers across a network, not a code library that executes on the same CPU that called the library. It’s basic stuff, and pretty obvious, but, sometimes, when it’s late at night, you accidentally design your REST API as if it’s a code library.
Write tests that abuse the API by passing in bad data, too-long strings, random noise, and a sequence of numbers.
Learn a DevOps tool like Vagrant.
It’s important to be able to bring up a “clean copy” of the application. REST APIs are stateful services, typically. You have the POST verb, and it causes changes to state. You have the PUT and DELETE and PATCH verbs that alter objects already in the service. Not only that, but the trend is to use REST APIs for all state information, retaining only a temporary set of objects on the client. The REST API is replacing the “save” button.
Give this fact, you need to be able to wipe out the state before running tests. You need to automate the process of firing up a virtual machine or container, and then installing the latest copy of the software. This means learning a tool like Vagrant and Ansible, or Puppet.
You don’t need to wipe out the system for every test run – that would be painful. You should just do it every day or every week, to make sure the system can be built from nothing, and run tests successfully.