Uploading in GAE is different

Chugging right along and hit a speed bump, figured it out and got it working. Coming from the CodeIgniter File Uploading Class, the Google AppEngine (GAE) approach is totally different. You pretty much have to throw away what you know & use there. Direct file uploads to your handler will not work. You have to use the GAE API to create a dynamic upload URL to put as your form's action. You miss out on CI's ability to limit file types and image dimensions (you can limit file sizes), but GAE adds cloud storage and the ability to receive very large files (up to 100 TB). GAE passes back the $_FILES super global to a handler you specify when you create the URL.

So the first step is to add the API inclusion in your CI controller. Then in your handler that creates the upload form, you call createUploadUrl and pass that into your template for the form action parameter. Then you create another handler (which name you used in the aforementioned createUploadUrl method) and you can verify the upload from the $_FILES parameter and then do whatever you need.

Here's what my controller ended up looking like:

The Google documentation on the topic was a really good reference. Note that the user has 10 minutes until that dynamic URL expires.

CodeIgniter routing with AppEngine

A few more steps forward on my Deduper project. I designed out the URL's and got the rudimentary homepage and create set form going. Also got it integrated with my local MySQL install. The one thing I learned lately is that with CodeIgniter in Google AppEngine, you have to configure the index_page (in config/config.php) to be blank so index.php doesn't get injected into the various URL's it creates. My app.yaml directs all requests already to index.php, so the extra index.php confuses CodeIgniter's routing.

AppEngine is different

So last night, I basically got things setup and learned a few lessons. The first one is that the AppEngine environment is not your typical PHP web environment. I was trying to build out a login part, using the lessons I learned from OAuth and Google's own PHP API and it wouldn't work because curl isn't available in the environment. The PHP AppEngine environment has its own fetch content by URL, but that means the Google OAuth library wouldn't work and I would need to look elsewhere. Fortunately, AppEngine has a User library that you can tap into. Even more, you can specify in the app.yaml file to protect certain places of your website and it will take care of that for you, which simplifies a lot of what I dealt with previously.

Anyway, the main lesson here is that things are different and what I have used in the past may not work. I was able to get CodeIgniter working in the environment, but I figure I may run into other problems when I try to use some CodeIgniter libraries (i.e. the file Uploader class) and it runs into the environment limitations.

Google Cloud Developer Challenge (Accepted)

I know I'm late to the challenge, but I just heard about the Google Cloud Developer Challenge and it's an itch I wanted to scratch - getting a project on the cloud. My goal is to write a simple web application that takes in multiple spreadsheets and asks the user if they're interested in finding duplicated or unique records. Pretty simple, but it's something we do a lot at work and I imagine others might be interested in this, too.

I applied for and received a $2,000 credit, so I plan on putting it to good use. Let's see if I can get this done!

I'm going to use PHP and CodeIgniter and I'd love to get Google Drive integration to work w/ the spreadsheets.

Adding OAuth2 to a CodeIgniter Project

Lately, I've been adding OAuth2 (via Google) to some of our internal administrative web applications and I've been pretty happy with them. I especially love how this gets me out of worrying about setting up usernames & passwords, password reset mechanisms and liability for a reused credential. OAuth2 just has a bucket full of win, once you get your head around it & get used to it.

I had a hard time, though, getting my head around it. Sure, there are lots of technical docs & specs, but nothing to really step you through it. So I thought I'd share my approach here, in the case it might help others incorporate this awesome system. I'll admit that my approach is still going through refinements and I definitely welcome input, so let me know if you have ideas or suggestions.

First, I'm using CodeIgniter and Google. If you want to use Twitter, Facebook or LinkedIn, this post is not for you. If you're not using CodeIgniter, this post is probably not for you (though you may be able to give it a read and see if it helps with your PHP project).

The first step is to download Google APIs Client Library for PHP. I save it in the libraries folder underneath CodeIgniter's application folder. Then I log into the Google API Console system and either create or use an existing project.

Then I create a new controller called login, which has 5 methods:

• relogin: Basically, this simply shows a login page with a button to "Login with Google." This page also has a revoke button, which Google suggests. 
• not_approved: This is another simple method which shows a screen that tells the user that while they're logged in, their account hasn't been setup with permission to do the specific task they're trying to do.
• logout: Destroys the website's cookie and redirects them to the relogin method URL.
• revoke: Sets a session variable that the user wants to revoke access and redirects the user to the login method URL.
• index: This is where the bulk of the login logic lies. Include the Google PHP libraries, setup the client and then go through a series of checking if a code get query parameter was passed in or if the access token is already in the session and redirects back to itself to authenticate the user.

I want to explain a few parameters of setting up the Google Client. The ID, Secret and API Key are standard and you get those from the Google API Console. But the other three parameters are things I've learned from experience. setRedirectUri + CodeIgniter's site_url() parameter makes the code very portable as you move from server to server. Just make sure that you setup each server & URI as a valid redirect in your project settings. I use array( 'openid', 'email', 'profile' ) for the setScopes parameter, which gives me some great information to tuck away in my user table. Other scopes are available should you want to do more Google-y services on behalf of your users. Lastly (and most importantly!) is the setApprovalPrompt to auto. If you don't, the default behavior prompts the user to verify your app every time they need to login. Using auto provides a much better user experience, in my opinion - not sure why it isn't the default behavior.

Then I create a few views that the login controller will be using: login and blacklist. The blacklist view just has a simple message saying that the administrator needs to approve their account. I add an email link so the user can prod me if they've been waiting a while.

Then I create a user model that is tied to a table I create that basically holds the information that comes back from the token. In my model, I have a nice utility function that either updates the existing user (based on their Google Profile ID #) or inserts them and returns the primary key ID #. My login controller uses this after the user has approved my app to insert or update the user information and then get the ID # and save that in the session.

Finally, I update my primary application controller and add a __construct method (if one doesn't already exist) and then add code to look for a user ID # in the session and if it's either not there or if the token has expired, I destroy the session and redirect the user to the login's index method URL. If it is there and the token is still valid, I check to see if the user has permission. Just because they have a Google Account doesn't mean you necessarily want them to do things.  My table has an extra column for a simple is_admin bit that defaults to 0. I manually set that to 1 for approved administrators. If they don't have permission, then I redirect them to the login's not_approved method URL.

So anyway, there you have it! I hope that helps and again, please feel free to provide feedback in the comments below. Thanks!

Database dates

Before I started using Drupal, I recorded timestamps as MySQL timestamps, where were easily readable, but that's where the advantages end. Drupal opened my eyes to using the epoch value, which allows easy sorting, comparison and other fun math functions. Further, MySQL and whatever coding language you're using has support for converting epochs to human-readable strings in all kinds of formats.

It isn't without its own disadvantages, though. It's hard to read when you're mired in the MySQL console, though I imagine at some point, I'll know that 1400000000 is May 2014 (guess I know that now) and perhaps it'll become like looking at Matrix code.

Anyway, MySQL has various integer types and when you're designing your database table, you could have a field be either int, int unsigned, bigint or bigint unsigned.

First, it makes no sense to use the default signed unless you're dealing with dates before (January 1,) 1970.

Second, we may be looking at another Y2K problem, depending on how the date fields are setup.

Data Type	Max #	"Y2K" deadline
int	2147483647	Tue, 19 Jan 2038 03:14:07 GMT
int unsigned	4294967295	Sun, 07 Feb 2106 06:28:15 GMT
bigint	9223372036854775807	Sometime (waaaay) after 2286
bigint unsigned	18446744073709551615	Sometime (waaaaaay) after 2286

It looks like Drupal 6's default setup is a signed int, so I'll have to upgrade it sometime before 2038.

Update: I figured out the bigint Y2K dates... 292 billion years from now for the signed version, so I think whatever code you write will be long gone by then.

AdSense mystery

So I'm having an AdSense mystery with my work sites and I wanted a large space to lay it all out and hopefully get some ideas/pointers on what I'm doing wrong from anyone in the AdSense community.

We have a site for our industry publication, QSR magazine with a mobile version/theme. The first half of this year, I simply had an AdSense spot at the top of our page and it was clipping right along with pretty decent AdSense revenue. In trying to quantify that, I'm confused with AdSense reports with two different products: AdSense for Content and AdSense for Mobile Content. If I just look at July, though, I just see AdSense for Content for our mobile site, so let's just isolate that:

• 39,426 page views, 162 clicks, $120.47 earned

Ok, so my bright idea was: Hey, this is based on one banner ad. What if we update our mobile site to have 4 leaderboards on each page? We could multiple our revenue! I also wanted to consolidate our banner management with our other system (DFP), so I updated our site templates to use the new DFP ad code and created the 4 slots in DFP. This would also allow our advertisers to advertise directly on our mobile site, too.

But what I'm seeing is a dramatic reduction in CPC. We have yet to have direct advertisers, so all impressions are being delivered through AdSense in DoubleClick, but the (revenue) numbers aren't what I was hoping. We made the switch on August 31st, so if I compared November to those July #'s, here's what I'm seeing:

• Mobile Leaderboard #1: 25,794 impressions, 122 clicks, $20.37 earned
• Mobile Leaderboard #2: 24,826 impressiosn, 92 clicks, $11.22 earned
• Mobile Leaderboard #3: 20,714 impressions, 56 clicks, $4.37 earned
• Mobile Leaderboard #4: 22,671 impressions, 9 clicks, $0.58 earned

So my question is why am I seeing such a huge reduction when using DFP vs. AdSense directly?