lloyd.io is the personal website of Lloyd Hilaiel, a software engineer who works for Team Ozlo and lives in Denver.

All the stuff you'll find here is available under a CC BY-SA 3.0 license (use it and change it, just don't lie about who wrote it). Icons on this site are commercially available from steedicons.com. Fonts used are available in Google's Web Font directory, and I'm using Ubuntu and Lekton. Finally, Jekyll is used for site rendering.

Finally, Atul, Pascal, and Stephen inspired the site's design. And in case you're interested, this site's code is available on github.

NPAPI, the HTML5 File object, and the glory of UNIX
2009-09-04 00:00:00 -0700

Earlier today I was impressing my wife with some unix foo by automatically swapping FIRST LAST —> LAST, FIRST formatted data while sorting and finding duplicated entries (ok, so she was only mildly impressed). The shell command looked a little like this:

[lth@lappro ~] $ ./first_to_last.rb < new_names.txt | sort -f | uniq > back_to_you_bob.txt

Not rocket surgery, certainly, but still a great example of combining several small tools to achieve an end. This simple act re-filled me with awe on the beauty of design with lots of little processes. Processes enforce proper protocols and data hiding (unless you’re talking about cross process windows or some other such tomfoolery), they guarantee 100% resource reclamation, they afford a robust system where if one guy falls down the party can still pick up and continue.

But I’m like 2 years late here to this re-realization for the web: we already all understand the beauty of lots of little processes. The Chrome browser started by moving plugins (and just about every other distinct subtask) into it’s own process. IE8 joined the fray by running each tab in a separate process. Safari 4.0.3 on Snow Leopard running 64bit puts npapi plugins on the other side of a process break. The fine folks at mozilla are talking through it and will inevitably follow suit.

So this is awesome. Now our browsers don’t crash as much, instead they tell us “The Yahoo! BrowserPlus plugin has crashed” (for instance, usually it’s the Adobe Flash plugin, right?).. They tell us “the page is unresponsive”. Blame is appropriately assigned so we, as users, can complain to the right people. Fewer human hours are dwindled and more broken pages are fixed. Followed by an era of increased world peace and obesity. Great.

The File is the Thing

In that code snippet above, it’s really the file abstraction that deserves all the glory. Once we’ve got all these processes, how do they communicate? Well, first and foremost they need to be able to move data back and forth. The simplest way to do this is in an opaque blob. A file if you will. A container that is capable of holding data that can be moved back and for between loosely coupled components of the software system.

If you take a look at innovation on the web today, we’ve got plenty of separate processes, we’ve got HTML5 goodness running in browser, we’ve got Gears running in a combination of plugins and extensions, we’ve got BrowserPlus running in an NPAPI plugin (or activex, but they always have thought different), we’ve got adobe flash in another plugin, and if you’re retro there’s a little java soup in there too (right, Silverlight too, don’t forget! this is important!).

Now in most cases, all of these runtimes/tools are running in different processes. And we have two choices: first they could be extremely loosely coupled and only share a control channel that’s brokered by the browser (as the world is today), or they could be more deeply integrated. It could be possible to build a web application using multiple client technologies that pass data back and forth. Imagine using HTML5 drag and drop to attain file handles, passing those into, say, BrowserPlus to process or alter the file contents, then taking the result and uploading it peer to peer using, say flash?

A simple file abstraction shared by all of these platforms is the key to interoperability.

Why do we care?

Why do we care if the interoperate? Simply because we need to nurture experiementation, and if there is no clean way for extensions to hook into browser supported drag and drop, then we’ll see monstrous hacks arise to make them interoperate so that non browser companies can continue participating in the innovation (without building yet another browser, so yesterday).

HTML5 and the File

Some very cool happs are goin on over at the w3c regarding files, uploading, and drag and drop. Excellent work in a pretty package.

I especially like the way the file abstraction works, and deeply suggest similar security policies to what’s present in BrowserPlus. Specifically the notion that untrusted javascript may not construct a file, but may freely pass it around between privileged apis that can do stuff with it. A key idea here being we leverage the user gesture of dropping or (multi) selecting, once that’s done they’ve implicitly authorized the page to do stuff with that file. Yeah, I know, people also like going phishing.

The One (File) Two (Stream) Punch

So is every browser going to implement powerful client-side movie editing? Or is this a bit bigger than what we want to build in the browser? Let’s assume I can name a feature, X, that’s interesting in a browser, but more than we could ever expect microsoft, google, apple, mozilla, and opera to go and implement. The web is the platform, right? We still want to be able to build this in open web technologies (uh, the user drops the movie on the page).

I suggest a simple fix on the npapi side. Here’s the patch:

--- npruntime.h.orig    2009-09-04 17:47:45.000000000 -0600
+++ npruntime.h 2009-09-04 17:50:20.000000000 -0600
@@ -128,6 +128,7 @@
     NPVariantType_Int32,
     NPVariantType_Double,
     NPVariantType_String,
+    NPVariantType_Path,
     NPVariantType_Object
 } NPVariantType;

@@ -160,12 +161,14 @@
 #define NPVARIANT_IS_INT32(_v)   ((_v).type == NPVariantType_Int32)
 #define NPVARIANT_IS_DOUBLE(_v)  ((_v).type == NPVariantType_Double)
 #define NPVARIANT_IS_STRING(_v)  ((_v).type == NPVariantType_String)
+#define NPVARIANT_IS_PATH(_v)  ((_v).type == NPVariantType_Path)
 #define NPVARIANT_IS_OBJECT(_v)  ((_v).type == NPVariantType_Object)

 #define NPVARIANT_TO_BOOLEAN(_v) ((_v).value.boolValue)
 #define NPVARIANT_TO_INT32(_v)   ((_v).value.intValue)
 #define NPVARIANT_TO_DOUBLE(_v)  ((_v).value.doubleValue)
 #define NPVARIANT_TO_STRING(_v)  ((_v).value.stringValue)
+#define NPVARIANT_TO_PATH(_v)    ((_v).value.stringValue)
 #define NPVARIANT_TO_OBJECT(_v)  ((_v).value.objectValue)

 #define NP_BEGIN_MACRO  do {
@@ -190,6 +193,7 @@
         Boolean                         NPVariantType_Bool
         Number                          NPVariantType_Double or NPVariantType_Int32
         String                          NPVariantType_String
+        File                            NPVariantType_Path
         Object                          NPVariantType_Object

         C (NPVariant with type:)   to   JavaScript
@@ -199,6 +203,7 @@
         NPVariantType_Int32             Number
         NPVariantType_Double            Number
         NPVariantType_String            String
+        NPVariantType_Path              File
         NPVariantType_Object            Object
 */

So the key is these magic File objects in javascript are translated to full paths as they are sent into plugins. This leaves all the selection, drag and drop, and upload up to the browser, and empowers plugins the ability to seemlessly integrate with this brave new world.

Next we can talk about streams, which buy us an efficient way to build a playground where everyone’s workin' together.

simple, yeah? lloyd