Intermittent Django REST Framework glitch examined with Apache Benchmark

While programming in the rest framework, I hit this error, intermittently, but over and over, along with others:

Could not resolve URL for hyperlinked relationship using view name “parseuser-detail”. You may have failed to include the related model in your API, or incorrectly configured the `lookup_field` attribute on this field.

Being a noob, I hit the books (documentation) again, and again, and thought I had it right, or close. (Don’t start out creating Django models for non-Django-managed tables. It’s pretty hard. It’s better to learn the vanilla Django way first, if possible.)

It seemed to be related to running mod_wsgi and re-entrancy. Tweaking that didn’t help, but it was something to learn.

On a guess, I chose the json output format, and it looked like the error went away. So, maybe it was in the rendering to HTML. So, I brought out Apache Benchmark, and hit the server with a hundred requests:

 ab -n 100 -c 4 "http://api.ggg.lo/1/report/4/?format=json"
 ab -n 100 -c 4 "http://api.ggg.lo/1/report/4/?format=api"
 ab -n 100 -c 4 "http://api.ggg.lo/1/report/4/?format=json"
 ab -n 100 -c 8 "http://api.ggg.lo/1/report/4/?format=api"
 ab -n 100 -c 1 "http://api.ggg.lo/1/report/4/?format=api"
 ab -n 100 -c 1 "http://api.ggg.lo/1/report/4/?format=json"

Boom. There it was. All the runs with JSON as the format succeeded, and were also pretty quick. All the runs with API as the format had many errors, running from under 10% up to 30%.

The errors decreased when concurrency (the -c option) was set to a larger number, and maxed out when -c was 1.

So the programmer-browseable API is a nice to have feature, but it triggers some bugs. Fortunately, JSON is the format used by front-end apps. So, we can ignore this problem (for now). (Note that this was not using the daemon process feature of mod_wsgi.)

[Edit: I went and ran the program through runserver, and it raised some errors. After these were corrected, the error rate declined… but didn’t get to 0. Running the ab against the runserver instance, however, returned no errors. So there’s a problem with the integration between Apache WSGI and the framework.]

Bitten Again

I got hit with this bug again, but this time around found at least one problem.

The first element of the queryset had an _object_id of ”. This wasn’t NULL, but an empty string. This caused a problem. The quick fix was to filter out null and empty keys from the queryset:

queryset = ParseUser.objects.exclude(object_id__isnull=True).exclude(object_id='')

I haven’t explored the issue, but the bad record was consistently the first element of the queryset, and it consistently raised the error, even when the format was JSON.