In mid-September, I requested a grant from the Python Software Foundation to port WebOb to Python 3. The proposal was as follows:

   Porting WebOb To Python 3
   =========================

   Proposal submitted to the Python Software Foundation

   September 16, 2011

   Abstract
   --------

   WebOb ( http://docs.webob.org/en/latest/index.html ) is a library used
   to make creating Python web applications easier.  It contains request
   and response abstractions which automate tasks such as setting
   response headers, setting cookies, decoding headers and cookies,
   generating ETags, and working with query strings.  It is based on the
   WSGI protocol.

   WebOb was created by Ian Bicking, and is now maintained primarily by
   Sergey Schetinin.

   WebOb is currently used in the following major systems:

   - Pylons
     (https://docs.pylonsproject.org/projects/pylons_framework/dev/)

   - Pyramid (https://docs.pylonsproject.org/docs/pyramid.html)

   - Google App Engine (http://code.google.com/appengine/)

   - Bobo (http://bobo.digicool.com/)

   - Restish (http://ish.io/projects/show/restish)

   - TurboGears 2 (http://turbogears.org/2.0/docs/)

   It is also used by other less-well-known frameworks and in many WSGI
   middleware implementations.

   Currently, WebOb is limited to running on the following platforms:
   CPython 2.5, CPython 2.6, CPython 2.7, PyPy 1.5 and 1.6, and Jython
   2.5.2.  This proposal requests funding to port WebOb to Python 3.  I
   propose to do this work.  After the work is completed, a version of
   WebOb will be available which is compatible with both Python 2 and
   Python 3.

   Why Funding Is Needed
   ---------------------

   The funding will service two high-level requirements:

   - If Python 3 is to be maximally useful to web developers, they need
     tools that make working with WSGI easier.  Working with "bare" WSGI
     as an application development platform is error-prone and difficult.
     The lack of a shared request/response abstraction on Python 3 leads
     to duplication of effort and dissatisfaction with Python as a web
     application development platform.  Existing Python 3-compatible web
     frameworks (Bottle, CherryPy) each implement their own
     request/response abstractions which are not divisible (or at least
     not separated) from their respective framework.  Continuation of
     this pattern will lead to further fragmentation of the Python web
     development community as time goes on.

   - If Python 3 is to be relevant to existing Python WebOb-based web
     framework authors, a Python 3-compatible WebOb must be made
     available to them.  Existing web frameworks which depend on WebOb
     cannot be ported to Python 3 without being able to depend upon a
     Python 3-compatible WebOb implementation.

   WebOb currently does not run on Python 3.  Creating a version that
   will run on Python 3 will satisfy the above requirements.

   The person or group who successfully ports WebOb to Python 3 must have
   the following skills: a deep understanding of HTTP and related
   specifications, a deep understanding of WSGI, mastery of Python 2,
   mastery of Python 3, mastery of testing tools, and a deep
   understanding of WebOb design goals.  There aren't very many people
   with the prior qualifications who aren't already engaged writing
   Python web applications full-time on Python 2.  They can't use Python
   3 without having a suitable toolchain available, and therefore they
   don't see much benefit in using it.  Making WebOb available on Python
   3 is also a rather boring and unrewarding task.  As a result, it has
   been difficult to convince anyone to to take the time from their
   schedule to provide an initial Python 3 port.  It's a bit of a
   chicken-or-egg scenario.

   Funding a port of WebOb to Python 3 will break the chicken-or-egg
   cycle, and make Python 3 a viable platform to the people who use
   WebOb, and, eventually, people who use frameworks based on WebOb.

   I submit this proposal mainly on the advice (via Twitter) of Jesse
   Noller, who replied to my public reports of trying to port WebOb on my
   own time with a suggestion that I apply for a grant
   (http://twitter.com/#!/chrismcdonough/status/108292018182172673).

   Strategy
   --------

   The strategy will be to have a single version of WebOb that will run
   on both Python 2 and Python 3 until such time as Python 2
   compatibility can be retired.

   A Python 2-compatible version of WebOb must continue to exist for the
   foreseeable future, as a large number of existing projects depend upon
   the combination of Python 2 and WebOb.  Both bugfix and feature
   releases of WebOb will need to be made indefinitely for the Python 2
   platform.

   A fork of WebOb to make it run on Python 3 that abandons Python 2 is
   untenable.  There is no one willing maintain a Python 3-only version
   of WebOb.  Likewise, there is no one willing to synchronize changes
   made to separate Python 2 and Python 3 versions of WebOb, and having
   the two versions fall out of sync is undesirable for forward and
   backward compatibility purposes.

   Therefore, we propose to use a single codebase to support both Python
   2 and Python 3.  This codebase will work under CPython 2.5, 2.6, 2.7,
   and 3.2, PyPy 1.5 and 1.6, and Jython 2.5.2.

   Since WebOb is very heavily based around I/O (taking data from the web
   and presenting it to the developer in a form he can work with),
   automated porting strategies such as 2to3 don't help very much; lots
   of code that does encoding and decoding between bytes and text needs
   to be changed "by hand".  Likewise, details of the new WSGI
   specification detailed in PEP 3333 require attention to detail that
   fall outside the scope of any automated translation.  2to3 automated
   translation is also costly in terms of library installation wall-time.
   Therefore, the WebOb codebase will "straddle" Python 2 and Python 3
   using a shared subset of both languages and will not rely on any
   automated translation tools.

   WebOb has no non-stdlib dependencies that aren't used solely for
   testing purposes.  We will rewrite any existing tests that have
   Python-2-only testing requirements to use Python 3-compatible
   analogues.

   Prior Work
   ----------

   In March of 2011, the Pylons Project sprinted at PyCon with the goal
   of providing 100% unit test statement coverage to WebOb.  This goal
   was reached and laid the groundwork for being able to port to Python 3
   by making it possible to have some level of confidence that Python
   3-compatibility-related changes do not break existing code.

   In the summer of 2011, Google sponsored student Joe Dallago to help
   port WebOb to Python 3 via their Google Summer of Code program.  This
   project was mentored by people from the Pylons Project.  Joe did the
   best he could, but the task was too complex to be completed
   successfully within the timeframe of GSOC.  However, his work at
   https://bitbucket.org/jayd3e/webob-1.1-py3 will serve as a basis (or
   at least as a guideline and reference) to a funded porting effort.

   Schedule and Budget
   -------------------

   The amount of work required to perform the port as described above in
   "Strategy" is estimated to be 80 hours. The work can be broken down
   like this:

   - Remove all existing deprecated methods which promise results as
     binary data (``str_GET``, ``str_POST``, ``str_params``).

   - Redesign existing "MultiDict" strategy to not encode/decode at
     lookup time, but instead to decode to text at request.POST /
     request.GET / request.params dictionary creation time.

   - Redesign existing cookie encoding/decoding strategy to work in terms
     of text instead of binary.

   - Redesign existing request and response descriptor implementations
     which take environ data and return binary data to expect and return
     text instead.

   - Get all existing WebOb tests passing on Python 3.

   - Functionally test on Python 2 and 3 under wsgiref, the CherryPy WSGI
     server, and mod_wsgi.  On Python 2, functionally test under
     paste.httpserver.

   - Run "in the wild" middleware, frameworks, and applications under the
     new codebase on Python 2, to ensure we haven't broken anything (use
     of non-API-but-probably-should-be-API methods/classes/functions).

   - Ensure 100% test statement coverage on both Python 2 and Python 3.

   - Update documentation to detail differences between usage of WebOb on
     Python 2 and Python 3; also note any behavior differences from prior
     releases.

   - Work with Sergey Schetenin to merge the code into the WebOb mainline
     repository.  We have already discussed the need to support Python 3,
     so this will be a code review process, not a process of defending
     the need for a Python 3-compatible version.

   Coding work will be done on a public Bitbucket fork of the main WebOb
   repository.

   The work will begin on October 1 and will finish on October 31.  I am
   requesting a total of of $3000.00 for this effort, to be paid on or
   before November 30.

   Risks
   -----

   There is not much "big design" thought required to do this work; just
   lots of "little design" requirements.  Most of the truly hard legwork
   has already been done by the Python core team (wsgiref), by Phillip
   Eby in PEP 3333, by Joe Dallago in his GSOC efforts, and by Graham
   Dumpleton and Bob Brewer in mod_wsgi and the CherryPy WSGI servers,
   respectively.  There is little risk of getting the design wrong, as
   the WebOb design already anticipates most requirements of encoding and
   decoding web data from binary to text.

   The only real risk is to existing users of WebOb under Python 2.  We
   need to make sure it continues to work for them and isn't backwards
   incompatible for indefensible reasons.  I'll attempt to make sure of
   this by running "in the wild" middleware under the new implementation,
   and fixing things as appropriate.

   Qualifications
   --------------

   I am the primary author of the Pyramid web framework, which uses WebOb
   as a base.  I have contributed regularly to WebOb, and I have
   collaborated with its maintainers many times.  I have served as a
   Google Summer of Code mentor this year to a student working on WebOb.
   I am the original author of Python PEP 444 ("Web3", a WSGI replacement
   protocol), although I am no longer its maintainer.  I have been
   developing Python web applications for 11 years.

   Conclusion
   ----------

   For a relatively small amount of funding, I believe we can make Python
   3 a more pleasant web development platform, recognizable to people who
   have a history developing web applications under Python 2.  New users
   will also benefit by having a reasonable request/response abstraction
   rather than dealing with "bare" WSGI or being tied to a particular
   framework implementation when they use Python 3.

I'm happy to report that this proposal was accepted by the PSF, and that we've now released a Python 3 compatible version of WebOb to PyPI .

Here's a report-out on the concrete aspects of the proposal:

• Remove all existing deprecated methods which promise results as binary data (``str_GET``, ``str_POST``, ``str_params``).

  DONE

• Redesign existing "MultiDict" strategy to not encode/decode at lookup time, but instead to decode to text at request.POST / request.GET / request.params dictionary creation time.

  DONE

• Redesign existing cookie encoding/decoding strategy to work in terms of text instead of binary.

  DONE

• Redesign existing request and response descriptor implementations which take environ data and return binary data to expect and return text instead.

  DONE

• Get all existing WebOb tests passing on Python 3.

  DONE

• Functionally test on Python 2 and 3 under wsgiref, the CherryPy WSGI server, and mod_wsgi. On Python 2, functionally test under paste.httpserver.

  DONE: the codebase works under wsgiref and mod_wsgi; the CherryPy WSGI server is not yet PEP 3333 compliant so it could not be tested.

• Run "in the wild" middleware, frameworks, and applications under the new codebase on Python 2, to ensure we haven't broken anything (use of non-API-but-probably-should-be-API methods/classes/functions).

  DONE: The Pyramid, Pylons, Restish, and ToscaWidgets 2 test suites were ensured to run using the new code (not all tests passed but stakeholders with failing tests were advised of potential backwards incompatibilies).

• Ensure 100% test statement coverage on both Python 2 and Python 3.

  DONE: We have 100% test coverage under Python2, but not under Python 3. Some of the code executes solely on Python 2 and some solely on Python 3 and no existing coverage tool allows us to conditionalize coverage like this.

• Update documentation to detail differences between usage of WebOb on Python 2 and Python 3; also note any behavior differences from prior releases.

  DONE: WebOb docs adjusted thanks to Sergey.

• Work with Sergey Schetenin to merge the code into the WebOb mainline repository. We have already discussed the need to support Python 3, so this will be a code review process, not a process of defending the need for a Python 3-compatible version.

  DONE WebOb moved to the Pylons Project on GitHub, and release made.

The WebOb changelog has the gory details about changes and backwards incompatibilities. You can see the (monster) diff at https://github.com/Pylons/webob/compare/1.1-branch...master

Thanks much to the PSF board, Sergey Schetenin, and Joe Gallago.