Pipeline might not be quite the right term. And maybe middleware isn't (PJE disagrees with how I use the term). And maybe filter isn't the right term. But eh. The WSGI Onion, if you will. I think it's applicable here.
Some kinds of content-type changing middleware are pretty uncontroversial, I think. For instance, there's a middleware somewhere out there that translates XHTML to HTML for clients that don't accept XHTML. The output is basically isomorphic to the original content, but the type is different. Maybe truly isomorphic content isn't that interesting or useful, but at least that case seems reasonable and possibly useful (if I ever in my life found myself caring about XHTML it might seem more useful). That actually isn't best as *just* an egress transformation, because it would be nice to add XHTML to the Accept header on the way in. If the Accept header in the wild wasn't full of complete bullshit, maybe this example would be more than academic even.
Realistically, I find that ingress and egress filters aren't very distinct. At least, any output-transforming filter seems to eventually grow *some* ingress transformation, though the opposite isn't always true. I don't see a reason to distinguish between the two; keeping them together makes the HTTP message and interaction (large) correct and complete at each step. (The exception being if middleware adds mutable structures to the environment, some bad stuff can happen.) Having a singular notion of middleware, that can be either input and/or output modifying, but where it always goes together, avoids a *lot* of errors in the configuration and composition of the system.
The thing you lose using WSGI for this, is that you can only communicate in HTTP terms, with some small extensions in the request. You don't have shared data structures. Your transformations have to be pretty completely defined in terms of HTTP. If you transform XML, all you get is XML -- there is no underlying data structure you can refer to. Personally I find that a benefit; sharing data structures across applications SUCKS. You end up slowly recreating Acquisition, and we all know how that turns out. Using WSGI gives you an inspectable and simple structure, that is very easy to test, and where reuse is highly valued (since you can express the API in terms of a document format or conventions built on existing document formats, as opposed to code).
Now maybe rewriting semantic XML into HTML is too much. I don't know; XSLT seems like it was *built* for this kind of design pattern. XSLT also kind of sucks, so maybe that's part of the problem. Maybe you just can't get enough information in there; though I would *hope* that you don't restrict yourself to a one-to-one match of an HTML page and an XML document. And maybe it makes sense to actually compose the application as a truly separate application at a different URL, that just happens to feed off XML documents elsewhere in the system. You have to worry about the notion of "here" a bit, but it's not too horrible if you have detectable links.
So... wandering about, that's my feelings. As to actually *writing* the middleware, WebOb's req.get_response() method makes it *tons* easier, basically using a WSGI 2.0 model at the loss of some of the overly complex features of WSGI 1.0 (it works with any WSGI 1.0 application, but will sometimes exhaust streams of data even if it's not necessary).
isomorphic!
Posted bychrismat
2007-11-20 12:40 AM
That's the word I've been searching for, thank you. Armed with that word, I'll restate. I'd prefer that each step in the "main" WSGI egress pipeline generated a content body that had a structurally isomorphic relationship to the content body generated by the application itself. At least if it wasn't something that caught an exception and turned it into a response (e.g. httpexceptions). That said, I don't a-priori object to non-structurally-isomorphic transforms in the WSGI pipeline, I just prefer to keep them to a minimum in my own pipelines.
I probably shouldn't have mentioned the separate ingress and egress pipelines, it's not very important.
Some kinds of content-type changing middleware are pretty uncontroversial, I think. For instance, there's a middleware somewhere out there that translates XHTML to HTML for clients that don't accept XHTML. The output is basically isomorphic to the original content, but the type is different. Maybe truly isomorphic content isn't that interesting or useful, but at least that case seems reasonable and possibly useful (if I ever in my life found myself caring about XHTML it might seem more useful). That actually isn't best as *just* an egress transformation, because it would be nice to add XHTML to the Accept header on the way in. If the Accept header in the wild wasn't full of complete bullshit, maybe this example would be more than academic even.
Realistically, I find that ingress and egress filters aren't very distinct. At least, any output-transforming filter seems to eventually grow *some* ingress transformation, though the opposite isn't always true. I don't see a reason to distinguish between the two; keeping them together makes the HTTP message and interaction (large) correct and complete at each step. (The exception being if middleware adds mutable structures to the environment, some bad stuff can happen.) Having a singular notion of middleware, that can be either input and/or output modifying, but where it always goes together, avoids a *lot* of errors in the configuration and composition of the system.
The thing you lose using WSGI for this, is that you can only communicate in HTTP terms, with some small extensions in the request. You don't have shared data structures. Your transformations have to be pretty completely defined in terms of HTTP. If you transform XML, all you get is XML -- there is no underlying data structure you can refer to. Personally I find that a benefit; sharing data structures across applications SUCKS. You end up slowly recreating Acquisition, and we all know how that turns out. Using WSGI gives you an inspectable and simple structure, that is very easy to test, and where reuse is highly valued (since you can express the API in terms of a document format or conventions built on existing document formats, as opposed to code).
Now maybe rewriting semantic XML into HTML is too much. I don't know; XSLT seems like it was *built* for this kind of design pattern. XSLT also kind of sucks, so maybe that's part of the problem. Maybe you just can't get enough information in there; though I would *hope* that you don't restrict yourself to a one-to-one match of an HTML page and an XML document. And maybe it makes sense to actually compose the application as a truly separate application at a different URL, that just happens to feed off XML documents elsewhere in the system. You have to worry about the notion of "here" a bit, but it's not too horrible if you have detectable links.
So... wandering about, that's my feelings. As to actually *writing* the middleware, WebOb's req.get_response() method makes it *tons* easier, basically using a WSGI 2.0 model at the loss of some of the overly complex features of WSGI 1.0 (it works with any WSGI 1.0 application, but will sometimes exhaust streams of data even if it's not necessary).
I probably shouldn't have mentioned the separate ingress and egress pipelines, it's not very important.
Replies to this comment