From Extensibility to Evolvability Once upon a time, HTTP was simple – what happened?
Overview of Talk • What makes an extension? • And why extend in the first place? • In the good old days, HTTP was simple • Why did it get complex? • HTTP is already quite extensible • Why isn’t this enough? • What is Evolvability? • How can we support it?
Why Extensions? • Why not just a fixed set of features? • Would be a lot easier • Doesn’t work because… • Extensions are fundamental to the Web • The Web is an information space • Not a single service • Every one lives in the same information space • How you live there is up to you • Change of Focus • From creating infrastructure to creating services using that infrastructure
What is an Extension? • Qualification of a normal HTTP message • authentication, payment, etc. • New HTTP syntax and encoding • Sticky headers, binary encoding etc. • Extensions to the “HTTP application” • WebDAV, etc. • HTTP-like protocols for new applications • RTSP, IMAP etc. • Leaving HTTP and TCP entirely • Multicast etc.
Once Upon a Time... • HTTP started out as a simple hypertext protocol • Send GET request - get back a document • Hypertext was what you asked for and what you got • There was no information about the document you retrieved • Especially no content-type • This were the early days of HTTP/0.9
Then HTTP/1.0 came along • No more only text based documents • Needed type information to distinguish images from text • MIME provided a mechanism for describing protocol messages • Was adopted for describing HTTP messages • A major cross road on the evolutionary path
And Much More Followed… • Large number of added features • Byte ranges • Cookies • Content negotiation • Authentication and proxy authentication • Proxying, caching, and gateways • Hit metering • Persistent connections and pipelining • … • Pattern has been copied in RTSP, SIP etc.
Thanks to HTTP’s Architecture • All these extensions could fit into HTTP • Using the basic HTTP building blocks… • Request Method (one pr message) • GET, HEAD, POST, DELETE, etc. • Response Status Code (one pr message) • 1xx, 2xx, 3xx, 4xx, 5xx, etc. • Header Fields (many pr message) • Protocol headers • Upgrade, User-Agent, Accept, Date, Server, etc. • Entity Headers • Content-Length, Content-Encoding, etc.
The Good Part… • What are the lessons so far? • The good side of all these extensions: • HTTP could actually support them • The extensions provide very useful features • Several of them have become widespread • Even across trust domains! • Thanks to the IETF WG • HTTP is really quite extensible
And the Bad Part… • All these extensions had to be part of the base protocol • Have blurred the (already implicit) line between base protocol and extensions in HTTP • Cost has been a very complex protocol • HTTP/1.1 has been under way a long time • Not just result of complex extensions but how they interact • Adding new features gets harder and harder • Reason: No way of replacing existing features • Cost of backwards compatibility keeps increasing • Will never be able to replace existing features
Cost pr feature increases over time Extensibility: Evolvability: Extensibility vs Evolvability
Interoperability vs. Evolvability • Cost of evolvability: • As evolvability goes up, ease of deployment goes down
Supporting Evolvability • How are Extensions deployed in the Web? • Often by extending existing applications • Spreading from in the small to the large over time • This means that: • Applications have different capabilities at all times • This requires that: • Applications supporting a particular extension should be able to employ this with no prior agreement; • Applications can require that the other party either understand and abide by the new protocol or abort the operation; • Has to work with existing HTTP
The Four Rules of Evolvability • Can be accomplished by expressing these four parameters as part of the protocol: • What it is (identify extension) • A URI identifies the extension in the global URI space • May be resolvable (interface description, code, etc.) • How to deal with it (optional or mandatory) • Can recipient ignore it or not? • Who should deal with it (hop-by-hop or end-2-end) • Should this go to the proxy or the user-agent, origin server • When to deal with it (ordering) • First do this; then do this etc.
HTTP Extension Framework • Mechanism for lowering cost of extensions • Works with existing HTTP • Stable specification ready to be used • Fulfills the 4 rules through support for • Decentralized deployment based on • URIs for identification • No risk of collisions between methods, status codes, or header fields • Evolution through mandatory declarations • You can’t ignore this extension • Separation of end-2-end and hop-by-hop extensions • Allows • Ordering of extensions
Example of Mandatory Ext. M-POST http://www.buy.com/payment HTTP/1.1 Man: "http://www.ecommerce.org/ext/payment" … <parameters describing your payment> HTTP/1.1 200 OK Ext: Date: Sun, 25 Oct 1998 08:12:31 GMT …
Example of Optional Ext. GET http://www.somesite.com/some.doc HTTP/1.1 Opt: "http://www.webalizer.org/ext/hitcount" … HTTP/1.1 200 OK Opt: "http://www.webalizer.org/ext/hitcount"; ns=12 12-hitcount: 24 Cache-Control: no-cache="12-hitcount,Opt" Date: Sun, 25 Oct 1998 08:12:31 GMT Expires: Sun, 25 Oct 1998 08:12:31 GMT …
Extension Declaration Params • What it is (identify extension) • The URI of the extension points to a definition of that extension • This can be a formal interface description, a piece of code, a textual description, or nothing at all (may not be resolvable) • How to deal with it • optional or mandatory • Who should deal with it • hop-by-hop or end-2-end • When to deal with it • ordering
Extension Declarations • Use the header field space for declaring extensions • This means that declarations can • Float between Methods • Float between Status Codes • Can be applied to any HTTP request/response • Few exceptions like 304 (Not Modified)
Server Response • When a server sees an extended message it can respond with • 510 (Not extended) • Message did not contain the information I needed. This can either be parameters or extensions • Client can then try again if it wants to • 2xx (OK in some form) • Request succeeded and extension was dealt with • Any other HTTP code • In case it is covered by HTTP already
Client Response • When a client sees an extended response, it can • Ignore optional extension declarations or deal with them if it wants to • The server can only send mandatory extensions in a response if it knows that the client has some way of dealing with it • For example if it has asked for a mandatory extension. • The client can then retry the request with extensions included, or go away and forget about it