Building blocks for world domination

Building blocks for world domination

I've been throwing around an idea for a few days that might be useful for the python ecosystem.

It's really easy to praise the bazaar way of doing things. Because of it, i can find four different libraries for manipulating pathnames in Python. There's the built in os.path which comes with every python distribution. There's pathtool which provides a nice api for walking a file system, which can replace the function 'walk' that's ordinarily provided by the Python Standard Library. There's Unipath which replaces paths as strings with paths as objects, to avoid a certain class of type based bugs. It also overrides standard behavior with what the author presumes is more logical behavior. There's path.py which provides the same but is 2 years old, and as far as i can tell, dead upstream. There's a fork of path.py which is in paver, which provides an interesting feature available to Python 2.5 and up, 'pathd'. This is a context manager that works in a different directory than the surrounding code. (No, i won't translate that for non Python users.) I've also had to do something similar in devshell, for the obviously similar reasons.

The end result is that there are so many good ideas out there that we really don't know what to do with them. An intermediate Python developer looking to put together a large application doesn't quite know what to do with all these different options. Does he pick path.py which appropriately follows the os.path behavior? Is Unipath better with it's object oriented behavior? Should he use the fork in Paver and require Paver to be installed with the application? Is doing this kind of research even a good use of time that would be better off spent just writing the tools in a util module on the side? Without prior knowledge or experience, this can muddy up the ecosystem.

This is of course not unique to manipulating pathnames. I'm finding that i'm looking for a wide range of reusable components in my Python work. It is hard for me to decide if i want to build my own parts or work with prior ones. Browsing the Python Cheese shop can be a daunting experience sometimes, although great when looking for brilliant ideas.

I think there is alot of room for a reusable component library. The goal of this library would to pick the best of the breed components and collect them all in one repository. As new ideas spring up in other libraries, we could cherry pick them and port them to the upstreams where necessary. It would exist outside other large projects like Zope and Twisted which provide other large building blocks for certain domains of problems.

Potential types of building blocks could include:

• Pathname manipulation
  
• Configuration file and runtime options management
  
• A unified set of functional programming inspired functions (I see way too many implementations of them)
  
• Implementations of more advanced functionality than provided by the Python Standard Library
  
• Add your own itch here.
  

I think the end goal would be to provide a platform of libraries that are blessed by the community at large, but don't necessarily have to be included in a default Python distribution. It'll hopefully let systems programmers and application developers do their jobs with one place to find all the information they need.

Anyone interested?