Fun with AppEngine Task Queues

So AppEngine Task Queues are fun and I say that without really digging deeply in the matter. As part of my project, I incorporated them because I figured my app may take a while to crunch through the data. You have a 60 second limit to normally respond to a user's request, which is probably sufficient enough for my needs, but just in case, I decided to use a Task Queue. That increases the crunching time limit to 10 minutes, which should totally be fine for my needs. If you need more than 10 minutes, I would suggest breaking your crunching into smaller tasks.

One of the issues I ran into was that my task would not execute. I set up the code in its own CodeIgniter controller and I was able to directly access it with my web browser, but when I added the URL to the task queue, it would just sit there. It would try to execute, but nothing would happen. I found my first clue in my AppEngine logs, which read something like this:

INFO     2013-11-06 02:52:10,700 module.py:608] default: "POST /worker/process_set HTTP/1.1" 302 -WARNING  2013-11-06 02:52:10,700 taskqueue_stub.py:1980] Task task4 failed to execute. This task will retry in 0.800 seconds

A task has to return a status between 200 and 299 (inclusive), so that 302 was the clue. 302 is the code for Found and usually is followed by an HTTP redirect Location header. I tried several different things, but what finally worked was declaring the worker path explicitly in my app.yaml (vs. letting the default handler take care of it). This is something I would have to do eventually to lock down the URL from any external requests.