Later this week I'll be moderating a chat entitled "Implementing the open web" at gluecon in my home town, Denver.

So in preparation for this panel, the logical first step seems to be to establish a clear and concrete definition for the "open web"...

Stealing Brad's work

Aha, this is easy, I'll flip back to one of my favorite responses to this question, written by Brad Neuberg last April. I feel that Brad has nicely covered the bases, and tend to agree with most of his points. Some things that I think are good enough to repeat include "Freedom of Social Forms":

The Open Web should support extreme gift economies, such as open source and Wikis, all the way to traditional free market entities, such as Amazon.com and Google. I call this Freedom of Social Forms; the tent is big enough to support many forms of social and economic organization, including ones we haven't imagined yet.

We must leave our philosophies around making a buck at the door. People are free to experiment with different business models all they like, from advertising to free tools with commercial consulting available, to whatever you can think up… People are free to build commercial and proprietary "derivative" work on top of the open web, and the platform is in no way geared to one model or another. The key here is that the "open web" attempts to serve all without bias, and in order to properly do so cannot in itself be owned nor governed by any small set of companies nor individuals, nor can it involve any technologies that infringe on the freedom of applications built on top of it.

Next on the list of restatables includes what Brad calls Openness (ok, a criticism, Openness is vauge, perhaps we should call it the Specification or Documentation requirement):

Openness – Whether the protocols used are de facto or de-jure, they should either be documented with open specifications or open code. Any entity should be able to implement these standards or use this code to hook into the system, without penalty of patents, copyright of standards, etc.

Specifications OR code. Hear that? Quite forward thinking given the present state of SQL in the browser. At present, the code is the specification. I think this is extremely messy — Having "standards" where there is no specification — however who ever claimed that evolution was pretty? Look at a very nice bit of work that I've come across lately, JSONQuery – A means of performing sophisticated queries against JSON documents. But here's the odd part, The best "specification" I can find is that blog post. No real specifications, but a couple different implementations. So we could say Brad's forcing a mess on us here by allowing the code to be the specification – But my claim is that as technologies gain popularity, the BNF will get written – And you can't really write a specification until you've kicked the tires and actually seen it in action. And a specification needs a champion anyway, I think JSONRequest is a fine piece of work, but it's still just sitting there.

Open Questions

Transparency. Brad argues the should here, that we should be able to inspect the web at all levels from protocols, to documents, to layout, to code. SSL is inscrutable over the wire, and minified javascript is also. Further, it could be argued that this goes against our previous claim, that the open web is a platform which allows for Freedom of Social Forms, now we turn around and say that whoever you are writing a web application for profit, you should take pains to lay out your site so that it's scrutable for others. Perhaps Brad was taking a jab at Flash, don't get me wrong, I dislike the clumsy way that flash fits into the web as much as the next guy (there's the discoverability problems, the search problem, absence of good free tools, yeah yeah.) But, I think that this argument applies more to the sites written on top of the technology stack that is the open web, rather than the open web itself. So as much as I like it, I don't know how much this belongs in the core definition.

Missing Bullets

Here's where I get into some difficult terrain where my judgement may be clouded by my dayjob, but I shall forge ahead despite that. This point comes when you cross several of Brad's ideas about what the open web is:

1. It enables "Third Party Innovation – without asking the powers that be for permission"
2. "Decentralization – …the web, an open system that anyone can plug into and create information at the end-points…"
3. "Third-Party Integration – At all layers of the system third-parties should be able to hook into the system, whether creating web browsers, web servers, web services, etc."
4. "Hackability – It should be easy to lash together and script the different portions of this web."

All of these points somewhat redundantly point to a basic idea. Consider web applications (aka sites): Anyone should be able to write a site using well documented and accessible technologies and tools (today: HTTP, javascript, CSS, php or rails). This person should be able to consume aggregate and perhaps enhance information from multiple sources (today: leveraging APIs serving structured data over HTTP). They should be free to make money off this site in any way that they like (today sell ads from google or yahoo, charge your users, offer advanced paid features, whatever).

This is fine and makes sense the web platform itself must be application agnostic and free for all to apply, but let's dig into the "Third Party Integration" point. Anyone should be able to write a web browser. Really? Think about that. Writing a web browser involves a lot of work, and if you regularly read ajaxian it feels like more and more every day. Javascript evaluation? Document rendering? CSS parsing? Caching? Option management? Localization? A web browser is not a trivial piece of software, and it feels like with innovations in the open web we're looking to make them exponentially more complex… We have already have started putting more and more in the browser, richer video rendering, 3d vector graphics, a full blown database… Yikes.

So no, with the way this is going, not anyone can go out and build a web browser. And in fact the people building them right now can't really keep up. But stay with me, the fix is simple. I think we can just take a look at what the bright folks at Mozilla have already done:

First let's look at Geode – a GeoLocation extension for firefox. A user installing this ext. would be able to easily share their location with javascript at a level of granularity that they specify. Now check out the weave extension , also from mozilla labs. This one most recently attacks the problem of identity management in the browser.

Both of these projects are clearly working to forward the open web. Both satisfy all of the criteria set forward above by Brad, and both are innovations that could push our web forward. But this isn't the point. The point is take a look at how Mozilla initially realized these features? Not inside but outside of the browser, as extensions.

Here's the problem, viscerally. The latest Firefox 3.5 beta has GeoLocation built-in:

However, we were still left with a chicken-or-egg problem: unless lots and lots of users installed an add-on, websites wouldn't have a significant audience for which to develop location-enabled services; and without lots of useful web content using the feature, users had no reason to install an add-on.

Herein lies the problem folks. This is the WRONG DIRECTION. We shouldn't be moving more shit into the browser, we should be pulling it out! Why? For the good of innovation. It goes like this:

A select few companies have the resources to build a browser, while many folks in the community have vested interest in getting distinct new features into the browser (aka, the open web). However the current situation requires that browser vendors are the ones doing the innovating (because if it's not built into the browser, we "wouldn't have a significant audience for which to develop" using it). Further, the moment something gets into more than one browser it gains a momentum, a dangerous momentum. We're not free to change our minds. Have a look at SQL in the browser, and search around for alternatives to SQL client side storage. There's a uncertainty here around which way is the right way, but it may be too late: we already have SQLlite implementations in several major modern browsers.

The Core Problem

The core problem is simple. There is no real way to get people excited about a new javascript API other than building it into a browser. And even then, you're going to have to wait for other browser vendors to come along that implement the feature in the same way. This nebulous thing that we now call the open web is dangerously slow in comparison to other runtimes that don't require inter-company participation.

The core problem is that we're experimenting inside the browser because we don't have any really viable ways to experiement outside of the "core" browser (yes, client side SQL is still an experiment as far as I'm concerned). This problem is fueled by the following facts:

1. There is no reasonable extension API that runs across all browsers.
2. This extension API that doesn't exist doesn't support extremely simple component installation, update, and revocation.
3. This extension API that doesn't exist isn't rich enough to allow something like SQLLite, or a JSON database, or identity mangement to be wholly implemented inside of it.

Thus finally we have enough to correctly patch the definition of the open web: The open web should allow decentralized extensions to it's own fabric. That is any rendering engine that is capable of rendering the open web must be augmentable with additional versioned APIs in a way that is seemless to the end user.

What did you say the open web was?

Oh, right. That's where we started, we need a clear and concrete definition of the open web. Well shucks, let's try it, borrowing heavily from Brad:

The open web is a set of technologies that enable the production of network accessible applications that are highly accessible to users everywhere. These specific technologies may change over time but are governed by the following properties:

• Documented – Every technology that is part of the open web is documented by either a rigorous standard or a reference implementation. This documentation is freely available.
• "Free" – Every technology comprising the open web may be freely used (no need to pay royalties, worry about infringing on patents, etc)
• Application Agnostic – The technologies included in the open web enable equally enable free or proprietary extensions or applications.
• Simple to Use – The open web is a platform designed for applications targeted at an average person without significant technical knowledge – an overall principle is that none of the technologies that are part of the open web should preclude a presentation that achieves this goal of accessibility.
• Extensible – The core technologies of the open web should be extensible at many different levels. That is it should be possible in a decentralized fashion and without any approval to write web applications, web services that serve information to applications, new web browsers that can render applications, or extensions that add new features to existing browsers.

In short, the Open Web to me is an idea. It is a set of technologies that are well documented, cost nothing, and allow for the creation of applications that are accessible to a wide audience. The open web is an extensible platform that allows equally for the creation of free or commercial applications. Finally the Open Web is designed to evolve, embracing at every level, the notions of extension and evolution.