My Favorite 2013 Videos

I submit my favorite 15 videos from 2013. Not necessarily that they were created in 2013, but that I became aware of them in 2013. Why 15? Why not? Let's go!

#15: Flying Eagle POV

It's one thing to imagine this or to see a CGI rendering from the LOTR movies, but it's another thing altogether to see it for real.

#14: Dream of the 90's - Portlandia on IFC

Ok, so this is not safe for work or safe for kids, but as a teenager of the 90's, I can relate to a lot of this. I just wish I watched this before going to DrupalCon in Portland back in May ... it would have explained a lot.

#13: Zachary Quinto vs. Leonard Nimoy

Battle of the Spocks. A commercial, but to hear Nimoy sing the Ballad of Bilbo Baggins again is priceless.

#12: Most Expensive Starbucks Drink

This should probably be stricken from the list for its hedonistic and consumer values, but if you were wondering how much you could spend on a Starbucks drink, look no further.

#11: Two Monkeys Demand Equal Pay

Good to see that humans aren't the only selfish species.

#10: Feynman on trains

My wife is ahead of me on appreciating Feynman and how he can make previously unapproachable subjects not only approachable, but understandable to the point where you could teach it. I enjoyed this talk on trains and had no idea they worked like this. Never really thought about it before, but now I get it.

#9: Chompy the Shark

Yeah, I couldn't paddle fast enough and never (ever) go into that body of water again.

#8: Billy Joel singing with a student accompanist

It took some stones to request this, but the guy has chops and the end result is pure magic.

#7: Speed Painter on Anderson Cooper

I had to watch this multiple times.

#6: Volvo Trucks Stunt with Jean-Claude Van Damme

Crazy stunt. Of course, Chuck Norris had to top that.

#5: Latte for SharShar

SNL has really stepped up their game this year with some great comedy, including this great parody skit on the new Starbucks Verismo system. This video is no longer available on Hulu, but it looks like you can watch it on Vimeo.

#4: Almost Pizza

Another SNL gem. This is available on YouTube ... for now. I miss Bill Hader & Kristin Wiig.

#3: What Does the Fox Say?

I'm sure I lose some hipster points for this, but it is so great and bonus points for being something I can play (& dance) with my kids.

#2: New Hotspurs Manager

When NBC got the rights to the Premier League, they made a promo video with Jason Sudeikis.

#1: Jaws recut

I will never look at Jaws the same way again and if anyone ever brings up the Jaws movie around me, I will start laughing (at least on the inside). Sooo good.

I created a playlist of the 13 videos available on YouTube for your viewing pleasure.

Well, that was fun

So I completed work on my Google Cloud Developer Challenge project and it's live and submitted for consideration. I even put together a quick video walkthrough. Now it's time to sit back, cross my fingers and hope for the best.

The last few days have been a mad scramble, as I was done earlier, but when it came to deploying the project on the real live AppEngine, I ran into a series of production issues. The big one I posted about on StackOverflow, but after not getting any help there, I stumbled across this brilliant AppEngine/CodeIgniter boilerplate on GitHub, so I restarted the project from that and copied my controllers, models and views over there. That worked, so then it was small polish things.

+Ian Barber wrote a great post about improving the credential flow, so that inspired me to come up with a logo, which I did with the Drawing app in Google Drive. I also took out the Drive scope since I wasn't yet using Drive, although I think I will at some point, so the users can pull in files from Drive as well as save their results there.

Then when it time to submit the app, I realized the domain had to be in the format gcdc2013-X.appspot.com. D'oh! Since you cannot rename the instance name, I then had to go and create a new application, configure it, setup billing, etc for the new (& proper) name.

So now we wait & see.

Great news, even! [appengine project]

Not sure why I didn't stumble upon this until now, but I'm glad that I waited to develop the user aspect to my project. The PHP AppEngine folks have an unofficial blog with some neat tips & tricks, including this dandy to get the Google PHP API client library working inside of your AppEngine app. This is huge because with the library, you can take advantage of some of Google's modern systems.

So now instead of using AppEngine's native User class, I can now use my OAuth login approach. The old (2007!) User class was ok, but you aren't able to do much with the resulting user, beyond getting their nickname and ID #. With OAuth, you can get their profile picture to slap into your header, ask them to integrate with their other services (which you can use the library for, too).

My guess is that Google's working towards getting the library integrated into the runtime, but there's probably some dust that needs to settle somewhere.

Fun with AppEngine Task Queues

So AppEngine Task Queues are fun and I say that without really digging deeply in the matter. As part of my project, I incorporated them because I figured my app may take a while to crunch through the data. You have a 60 second limit to normally respond to a user's request, which is probably sufficient enough for my needs, but just in case, I decided to use a Task Queue. That increases the crunching time limit to 10 minutes, which should totally be fine for my needs. If you need more than 10 minutes, I would suggest breaking your crunching into smaller tasks.

One of the issues I ran into was that my task would not execute. I set up the code in its own CodeIgniter controller and I was able to directly access it with my web browser, but when I added the URL to the task queue, it would just sit there. It would try to execute, but nothing would happen. I found my first clue in my AppEngine logs, which read something like this:

INFO     2013-11-06 02:52:10,700 module.py:608] default: "POST /worker/process_set HTTP/1.1" 302 -WARNING  2013-11-06 02:52:10,700 taskqueue_stub.py:1980] Task task4 failed to execute. This task will retry in 0.800 seconds

A task has to return a status between 200 and 299 (inclusive), so that 302 was the clue. 302 is the code for Found and usually is followed by an HTTP redirect Location header. I tried several different things, but what finally worked was declaring the worker path explicitly in my app.yaml (vs. letting the default handler take care of it). This is something I would have to do eventually to lock down the URL from any external requests.

Uploading in GAE is different

Chugging right along and hit a speed bump, figured it out and got it working. Coming from the CodeIgniter File Uploading Class, the Google AppEngine (GAE) approach is totally different. You pretty much have to throw away what you know & use there. Direct file uploads to your handler will not work. You have to use the GAE API to create a dynamic upload URL to put as your form's action. You miss out on CI's ability to limit file types and image dimensions (you can limit file sizes), but GAE adds cloud storage and the ability to receive very large files (up to 100 TB). GAE passes back the $_FILES super global to a handler you specify when you create the URL.

So the first step is to add the API inclusion in your CI controller. Then in your handler that creates the upload form, you call createUploadUrl and pass that into your template for the form action parameter. Then you create another handler (which name you used in the aforementioned createUploadUrl method) and you can verify the upload from the $_FILES parameter and then do whatever you need.

Here's what my controller ended up looking like:

The Google documentation on the topic was a really good reference. Note that the user has 10 minutes until that dynamic URL expires.

CodeIgniter routing with AppEngine

A few more steps forward on my Deduper project. I designed out the URL's and got the rudimentary homepage and create set form going. Also got it integrated with my local MySQL install. The one thing I learned lately is that with CodeIgniter in Google AppEngine, you have to configure the index_page (in config/config.php) to be blank so index.php doesn't get injected into the various URL's it creates. My app.yaml directs all requests already to index.php, so the extra index.php confuses CodeIgniter's routing.

AppEngine is different

So last night, I basically got things setup and learned a few lessons. The first one is that the AppEngine environment is not your typical PHP web environment. I was trying to build out a login part, using the lessons I learned from OAuth and Google's own PHP API and it wouldn't work because curl isn't available in the environment. The PHP AppEngine environment has its own fetch content by URL, but that means the Google OAuth library wouldn't work and I would need to look elsewhere. Fortunately, AppEngine has a User library that you can tap into. Even more, you can specify in the app.yaml file to protect certain places of your website and it will take care of that for you, which simplifies a lot of what I dealt with previously.

Anyway, the main lesson here is that things are different and what I have used in the past may not work. I was able to get CodeIgniter working in the environment, but I figure I may run into other problems when I try to use some CodeIgniter libraries (i.e. the file Uploader class) and it runs into the environment limitations.

Google Cloud Developer Challenge (Accepted)

I know I'm late to the challenge, but I just heard about the Google Cloud Developer Challenge and it's an itch I wanted to scratch - getting a project on the cloud. My goal is to write a simple web application that takes in multiple spreadsheets and asks the user if they're interested in finding duplicated or unique records. Pretty simple, but it's something we do a lot at work and I imagine others might be interested in this, too.

I applied for and received a $2,000 credit, so I plan on putting it to good use. Let's see if I can get this done!

I'm going to use PHP and CodeIgniter and I'd love to get Google Drive integration to work w/ the spreadsheets.

Adding OAuth2 to a CodeIgniter Project

Lately, I've been adding OAuth2 (via Google) to some of our internal administrative web applications and I've been pretty happy with them. I especially love how this gets me out of worrying about setting up usernames & passwords, password reset mechanisms and liability for a reused credential. OAuth2 just has a bucket full of win, once you get your head around it & get used to it.

I had a hard time, though, getting my head around it. Sure, there are lots of technical docs & specs, but nothing to really step you through it. So I thought I'd share my approach here, in the case it might help others incorporate this awesome system. I'll admit that my approach is still going through refinements and I definitely welcome input, so let me know if you have ideas or suggestions.

First, I'm using CodeIgniter and Google. If you want to use Twitter, Facebook or LinkedIn, this post is not for you. If you're not using CodeIgniter, this post is probably not for you (though you may be able to give it a read and see if it helps with your PHP project).

The first step is to download Google APIs Client Library for PHP. I save it in the libraries folder underneath CodeIgniter's application folder. Then I log into the Google API Console system and either create or use an existing project.

Then I create a new controller called login, which has 5 methods:

• relogin: Basically, this simply shows a login page with a button to "Login with Google." This page also has a revoke button, which Google suggests. 
• not_approved: This is another simple method which shows a screen that tells the user that while they're logged in, their account hasn't been setup with permission to do the specific task they're trying to do.
• logout: Destroys the website's cookie and redirects them to the relogin method URL.
• revoke: Sets a session variable that the user wants to revoke access and redirects the user to the login method URL.
• index: This is where the bulk of the login logic lies. Include the Google PHP libraries, setup the client and then go through a series of checking if a code get query parameter was passed in or if the access token is already in the session and redirects back to itself to authenticate the user.

I want to explain a few parameters of setting up the Google Client. The ID, Secret and API Key are standard and you get those from the Google API Console. But the other three parameters are things I've learned from experience. setRedirectUri + CodeIgniter's site_url() parameter makes the code very portable as you move from server to server. Just make sure that you setup each server & URI as a valid redirect in your project settings. I use array( 'openid', 'email', 'profile' ) for the setScopes parameter, which gives me some great information to tuck away in my user table. Other scopes are available should you want to do more Google-y services on behalf of your users. Lastly (and most importantly!) is the setApprovalPrompt to auto. If you don't, the default behavior prompts the user to verify your app every time they need to login. Using auto provides a much better user experience, in my opinion - not sure why it isn't the default behavior.

Then I create a few views that the login controller will be using: login and blacklist. The blacklist view just has a simple message saying that the administrator needs to approve their account. I add an email link so the user can prod me if they've been waiting a while.

Then I create a user model that is tied to a table I create that basically holds the information that comes back from the token. In my model, I have a nice utility function that either updates the existing user (based on their Google Profile ID #) or inserts them and returns the primary key ID #. My login controller uses this after the user has approved my app to insert or update the user information and then get the ID # and save that in the session.

Finally, I update my primary application controller and add a __construct method (if one doesn't already exist) and then add code to look for a user ID # in the session and if it's either not there or if the token has expired, I destroy the session and redirect the user to the login's index method URL. If it is there and the token is still valid, I check to see if the user has permission. Just because they have a Google Account doesn't mean you necessarily want them to do things.  My table has an extra column for a simple is_admin bit that defaults to 0. I manually set that to 1 for approved administrators. If they don't have permission, then I redirect them to the login's not_approved method URL.

So anyway, there you have it! I hope that helps and again, please feel free to provide feedback in the comments below. Thanks!

Database dates

Before I started using Drupal, I recorded timestamps as MySQL timestamps, where were easily readable, but that's where the advantages end. Drupal opened my eyes to using the epoch value, which allows easy sorting, comparison and other fun math functions. Further, MySQL and whatever coding language you're using has support for converting epochs to human-readable strings in all kinds of formats.

It isn't without its own disadvantages, though. It's hard to read when you're mired in the MySQL console, though I imagine at some point, I'll know that 1400000000 is May 2014 (guess I know that now) and perhaps it'll become like looking at Matrix code.

Anyway, MySQL has various integer types and when you're designing your database table, you could have a field be either int, int unsigned, bigint or bigint unsigned.

First, it makes no sense to use the default signed unless you're dealing with dates before (January 1,) 1970.

Second, we may be looking at another Y2K problem, depending on how the date fields are setup.

Data Type	Max #	"Y2K" deadline
int	2147483647	Tue, 19 Jan 2038 03:14:07 GMT
int unsigned	4294967295	Sun, 07 Feb 2106 06:28:15 GMT
bigint	9223372036854775807	Sometime (waaaay) after 2286
bigint unsigned	18446744073709551615	Sometime (waaaaaay) after 2286

It looks like Drupal 6's default setup is a signed int, so I'll have to upgrade it sometime before 2038.

Update: I figured out the bigint Y2K dates... 292 billion years from now for the signed version, so I think whatever code you write will be long gone by then.