I can't believe it

I came in on schedule. In rewriting EKG, i made a few estimates and handed them off to Max as a set of deliverables. Unfortunately, reality got in the way and the man hour schedule got pushed off course a bit because of real world hours, namely taking care of school work and other things. I also had to stick in a bunch of haskell work in, as we've finally gotten some amazing momentum going there.

On the way, i decided to get fancy too. I realized that we would need to have a framework in EKG that could give us room to grow in looking at the things we analyze. I took some of the lessons i learnt doing the framework for Devshell and applied them roughly to EKG. Because of this, i ran into some technical issues handling multiple inheritance in Python. Let's just say that multiple inheritance, while it has it's uses, can end up really bad. Several reworkings later, i was a full week of real time off schedule.

This design decisions has paid off immensely. As a result, it took me 4 man-hours tonight to put together the basics of the reporting element. It's not quite as sophisticated as the version 2 grapher, nor does it do kevin bacon graphs yet, but it at least has the number one asked for feature, deltas. People, and by people, i mean the persons that pay me, want to see a numerical difference per month, per quarter, and per year, and all this is per domain.

I also took it one step further. You can even get the deltas of a particular month or quarter over multiple years. It's actually just a general purpose algorithm that can do any delta query on any collection of counts.

When I made the original estimates, i estimated around 4-8 hours for recreating the grapher tool. According to my timer, i've spent 4 man-hours on it tonight. It's closer to 6 or 8 because of constant fidgeting and moving around; i've been very ADD tonight for some reason.

Next up for EKG
Currently i'm spitting out HTML with string fudging. There are some nice libraries for generating HTML, i need to package one i like for Fedora and include it. I could also use templates, but i'm sick of doing things the MVC way, and i want to experiment more with Object-Proxy development.

Michael DeHaan's found a pretty cool javascript library that has some pretty widgets for tracking changes, and doing pretty pie graphs. Problem is, is that it will need some nice TLC to make it useable, and then pacakged as well. Things should get interesting.

I should also do a couple of blog posts about some of the cool tricks i put into EKG to make it work. Maybe we should think of a Fedora related Python code snippet database too.

1 flames:

Jef Spaleta zei

I need to understand what you are doing to make EKG pythonic.

If you are making a making a more robust analysis framework, is it going to be flexible enough to also look at our client access log data?

I just refactored how we were making global mirrormanager maps so that it could be more flexible and I could do more with it, like histogram plots and what not. But I'm still using matplotlib for plotting since its what I know best. I was going to call the project EEG as a nod to EKG.

But if the new pythonic EKG is going to be a framework for analysis...we should find some time to talk and see if we can get that log parsing and map making into EKG.

-jef