Python just got cooler for me

Being the Functional Programming fan i am, the following goes against every best functional programming practice. Fortunately, it's still good code :).

I'm trying to get some code with a few ugly 'global' objects in python to be reduced to something a bit more modular. Sometimes code is clearer than words:

>>> def foo():
... print foo.a
...
>>> foo.a = 1
>>> foo()
1
>>> foo.a
1
>>> foo.a = 2
>>> foo()
2

In otherwords, I have some value I need persistent between function calls. However, there's only one function that needs that value. I can store it in the function itself, and have it available.

The only unfortunate tragedy is that code like this still fails epically:

>>> 0.15 * 6
0.89999999999999991

Keeping Private config files Private in Git

Problem: You are working on a project with several other people, each one of you using radically different test environments. The project uses a configuration file that each developer, and eventually each user has to customize. You also want to keep a 'standard' general purpose config file with sane defaults in the source repository.

This is very commmon when developing web applications. For example, a web app that sends out mail notifications may need to use your private SMTP server. You may not feel like telling the whole world where to find it once they hack into your private network.

Solution: Just don't commit that file to the source repository of course :)

Better Solution: People are clumsy and sometimes forget to ignore it. Furthermore, changes to the config file upstream will make practically every new patch conflict with your own working tree. This becomes hell to manage, unless you are truly awesome. Or you use Git. Git is truly awesome.

Furthermore, by using these sane development practices, you'll see other benefits from them. Git makes some very advanced tricks easy, and will make development easier for everyone on your team.

One thing i recommend for git newbies is to use a graphical browser to just view all the changes each step of the way. I like Qgit personally. This exercise is done through the command line, but for some people it also helps to be able to visualize what is going on.

For starters, this requires a bit of preconfiguration. When coding on FAS, i have two main branches i work with, 'master' and 'loupz'. 'master..loupz' (git-ese for all the patches that are in loupz, but not in master) contains one single patch. This patch fixes the configuration files to use my own environment. It contains a few passwords and some information about my local network topology that i just don't want to share. It looks like this:

yankee@koan ~/Projekten/FAS2 $ git log master..loupz
commit cc4990566df6cfd67707892d81c826b898aff5cd
Author: Yaakov M. Nemoy
Date: Tue Jul 22 15:23:16 2008 +0200

My modded fas.cfg

This commit should never be seen in public. If it does, yell at me.


Once this is set up, it's time to start hacking. In putting together this example, i forgot to 'pre-setup' my hacking session, so i will use git-stash to skip over the 'edit' steps.

yankee@koan ~/Projekten/FAS2 $ git stash
Saved working directory and index state "WIP on loupz: 048419f... My modded fas.cfg"
(To restore them type "git stash apply")
HEAD is now at 048419f My modded fas.cfg


The first thing to do is to create a new branch for all the future development. Since we were hacking on fasclient today, i named it accordingly. This new branch has to branch from loupz, and not master, because loupz has the working configuration.

yankee@koan ~/Projekten/FAS2 $ git checkout -b fasclient loupz
Switched to a new branch "fasclient"

Then I do my hacking, or in this case, just run git-stash apply.

yankee@koan ~/Projekten/FAS2 $ git stash apply
# On branch fasclient
# Changed but not updated:
# (use "git add ..." to update what will be committed)
#
# modified: fas/model/fasmodel.py
# modified: fas/user.py
#
no changes added to commit (use "git add" and/or "git commit -a")

Then it's time to add code from the working tree to the index, the staging area, so we can commit them to the repository. I like git-add --interactive, because it gives me an extra level of verification as to exactly what is being committed.

yankee@koan ~/Projekten/FAS2 $ git add --interactive
staged unstaged path
1: unchanged +11/-0 fas/model/fasmodel.py
2: unchanged +19/-0 fas/user.py

*** Commands ***
1: [s]tatus 2: [u]pdate 3: [r]evert 4: [a]dd untracked
5: [p]atch 6: [d]iff 7: [q]uit 8: [h]elp
What now> 5

Snip!

The rest of this part is like watching sausage being made ;) . Next we commit our changes.

yankee@koan ~/Projekten/FAS2 $ git commit
Created commit 53dc510: An alternative way to get the data to fasclient as efficiently as possible.
2 files changed, 29 insertions(+), 0 deletions(-)


If you're following along in Qgit, you'll see that the commit history has become a bit forked between stash, loupz, fasclient, etc... Since we don't need the stuff in the stash anymore, let's just clear it.

yankee@koan ~/Projekten/FAS2 $ git stash clear

So now there should be a string of patches on top of loupz that have been tested with the private configuration file. We know the code works. If we were to call git-push now, the troublesome patch would be included in the push. Here's the fun part. We're going to use the awesomeness of git to rewrite the commit history. This is called rebasing. In this instance, we're going to rebase loupz..fasclient, namely all the patches that are in the fasclient branch, but not in loupz, and rebase them onto master. The command is simple:

yankee@koan ~/Projekten/FAS2 $ git rebase --onto master loupz fasclient
Already on "fasclient"
First, rewinding head to replay your work on top of it...
HEAD is now at 6c583ab Adding some legal voodoo and mumbo jumbo.
Applying An alternative way to get the data to fasclient as efficiently as possible.
warning: squelched 2 whitespace errors
warning: 7 lines add whitespace errors.

To double check this, check git-log

yankee@koan ~/Projekten/FAS2 $ git log
commit 71f41ede6350b481bba358ed612a6cc93fad7a78
Author: Yaakov M. Nemoy
Date: Sun Sep 28 22:40:25 2008 -0400

An alternative way to get the data to fasclient as efficiently as possibl

Ricky and I are trying to figure this out, this commit is one option.

commit 6c583ab7a95f28f3871360a34f233a6454a46e53
Author: Yaakov M. Nemoy
Date: Sun Sep 14 20:22:30 2008 -0400

Adding some legal voodoo and mumbo jumbo.

commit 4adc06b3f4dfd7cfd2adf14a1d990afe1bddbe15
Author: Yaakov M. Nemoy
Date: Sun Sep 14 20:17:45 2008 -0400

Adds help and some documentation to the user


Graphically, you should see that there are now two diverging branches, fasclient and loupz. We now need to merge master to fasclient. This is simple.

yankee@koan ~/Projekten/FAS2 $ git checkout master
Switched to branch "master"
yankee@koan ~/Projekten/FAS2 $ git merge fasclient
Updating 6c583ab..71f41ed
Fast forward
fas/model/fasmodel.py | 10 ++++++++++
fas/user.py | 19 +++++++++++++++++++
2 files changed, 29 insertions(+), 0 deletions(-)
yankee@koan ~/Projekten/FAS2 $ git branch -d fasclient
Deleted branch fasclient.

Note: we also deleted fasclient because it's no longer needed.

Time to push upstream.

yankee@koan ~/Projekten/FAS2 $ git push
To ssh://ynemoy@git.fedorahosted.org/git/fas.git
! [rejected] master -> master (non-fast forward)
error: failed to push some refs to 'ssh://ynemoy@git.fedorahosted.org/git/fas.git'


It turns out that our local repository was not completely up to date. This is not a big deal, because rebase can help us solve this problem as well. We only need to use a more simple incantation of git-rebase. We simply want to take all the patches that are in our local 'master' branch, but are not on the origin/master branch upstream, and have them tacked on the end.

yankee@koan ~/Projekten/FAS2 $ git rebase origin/master
Current branch master is up to date.

Oops, we need to fetch first ;)

yankee@koan ~/Projekten/FAS2 $ git fetch
remote: Counting objects: 61, done.
remote: Compressing objects: 100% (46/46), done.
remote: Total 46 (delta 31), reused 0 (delta 0)
Unpacking objects: 100% (46/46), done.
From ssh://ynemoy@git.fedorahosted.org/git/fas
6c583ab..cc20015 master -> origin/master
yankee@koan ~/Projekten/FAS2 $ git rebase origin/master
First, rewinding head to replay your work on top of it...
HEAD is now at cc20015 Initial fas_client method.
Applying An alternative way to get the data to fasclient as efficiently as possible.
warning: squelched 2 whitespace errors
warning: 7 lines add whitespace errors.

Finally, we can just push all these changes upstream.

yankee@koan ~/Projekten/FAS2 $ git push
Counting objects: 11, done.
Compressing objects: 100% (6/6), done.
Writing objects: 100% (6/6), 1.19 KiB, done.
Total 6 (delta 4), reused 0 (delta 0)
To ssh://ynemoy@git.fedorahosted.org/git/fas.git
cc20015..3df1588 master -> master

If you view this graphically, you will see that the branch with the private configuration is horribly out of date. We need rebase one more time. But this time, we need to check out the branch first.

yankee@koan ~/Projekten/FAS2 $ git checkout loupz
Switched to branch "loupz"
yankee@koan ~/Projekten/FAS2 $ git rebase master
First, rewinding head to replay your work on top of it...
HEAD is now at 3df1588 An alternative way to get the data to fasclient as efficiently as possible.
Applying My modded fas.cfg

And that does it!

This method provides three benefits. First, it makes it very easy to maintain a private configuration without conflicting with other developers on your team. Secondly, it makes it very easy to ensure that there are no 'floating bits' that are uncommitted on your working tree. The only thing that isn't handled by Git are the bits you are actively working on. You can keep each branch separate and organised. Thirdly, once you have to merge with upstream, you can do so cleanly, without a nagging 'merge' commit. This will help you maintain a clean and professional looking log to development.

To make the last point very clear, try to find two git repositories, one with 'merge' commits, and one without, and view it in Qgit. You will notice the repo with 'merge' commits will fork frequently any time two developers work simultaneously. It can sometimes be hard to follow. In contrast, look at the other repo. Development goes in a single linear line. It only forks when a developer or team has to work on a radically different feature. It is much clearer why there are forks in this second repo.

How to run the jack audio connection seamlessly in gnome on Fedora

On the coattails of Lennarts excellent post about the status of the Linux audio landscape, I would like to give the more adventurous Fedora user some tips on how to experiment with some of these other technologies.

Problem: The user would like to run 'foo' server in conjunction with Pulseaudio in either gnome or kde without too much 'tinkering'. Tinkering means changing absolutely no files outside of /home without the aid of yum or some otherwise simple and reproducible command. Furthermore, these changes should work reliably across setups.

Solution:

1) So in this case, the 'foo' server is jack. There are a number of programs I would like to work with that work with Jack instead of Pulse. Jack is essentially lower level than Pulse, so this solution is geared towards things that work underneath Pulse, thus tools that have to be running before Pulse is. Since jack is the defacto audio daemon for Freeeee and a favourite by the developers, Fedora users might want to become more familiar with it.

2) To the best of my knowledge, Gnome starts Pulseaudio by calling up the legacy ESD. Since Pulse wraps ESD nicely, it starts up instead using a systemwide default configuration. KDE on the other hand has a directory for systemwide and user specific start up scripts to be run. Pulse is included as a systemwide script. If the user is a member of pulse-rt, then it automatically uses realtime mode settings. Otherwise it resorts to defaults. We're assuming you want realtime settings.


# usermod -G pulse-rt


Nicely enough, we shouldn't need to do anything more for jack to run in realtime mode as well.

3) Although we can hack the KDE systemwide directories to start jackd as well, or just change the user specific ones, it provides no guarantees that jack can be started before pulse. Furthemore, this is not a possible solution for gnome.

When the user logins from gdm or kdm, the display manager searches for some .dekstop files that specify all the possible desktop environments available to the user. These .desktop files have links to different programs, such as 'gnome-session' or 'startkde'. These programs start up the comprehensive desktop environment. The user can also specify their own script or program to be run on the startup of the X server. This script is stored in a per-user basis at ~/.xsession . Although this is considered a 'legacy' method, it is quite commonly used in more minimalist setups. In order to enable this 'legacy' feature, run:


# yum install xorg-x11-xinit-session


Then create your ~/.xsession script.

$ vi ~/.xsession


and fill it with the following example


#! /bin/sh

jackd --realtime -d alsa &
gnome-session


and finally make it executable


$ chmod +x ~/.xsession


Notice that in the .xsession file, the command for jackd has an ampersand '&' after it. This means that the program is run 'in the background'. gnome-session does not have the ampersand. When gnome-session is finished, then the X server dies. If you replace it with any other program, remember, once that program dies, the X server dies with it. (Try it with xterm or firefox, and see what happens when you close it down.)

You can of course experiment further with .xsession and put any programs you want to start, regardless of which DE you use.

4) Finally, Pulse must be configured to work with Jack. I used a config provided by jebba, one of the Freeeee developers. It goes in ~/.pulse/default.pa .


$ vi ~/.pulse/default.pa


Then fill it with this

load-module module-jack-sink channels=2 channel_map=front-left,front-right
#load-module module-jack-source channels=2 channel_map=front-left,front-right
load-module module-native-protocol-unix
## The following is not mandatory
load-module module-volume-restore
load-module module-default-device-restore
load-module module-rescue-streams
.nofail
.ifexists /usr/lib/pulse-0.9/modules//module-x11-publish.so
load-module module-x11-publish
.endif
## The following can conflict with this config.
#.ifexists /usr/lib/pulse-0.9/modules//module-gconf.so
#load-module module-gconf
#.endif
set-default-sink jack_out
set-default-source jack_in


# no
#load-module module-alsa-sink
#load-module module-alsa-source device=hw:1,0
#load-module module-oss device="/dev/dsp" sink_name=output source_name=input
#load-module module-oss-mmap device="/dev/dsp" sink_name=output source_name=input
#load-module module-null-sink
#load-module module-pipe-sink

### Load several protocols
.ifexists module-esound-protocol-unix.so
load-module module-esound-protocol-unix
.endif
# no workie
#load-module module-native-protocol-unix


### Network access (may be configured with paprefs, so leave this commented
### here if you plan to use paprefs)
load-module module-esound-protocol-tcp
load-module module-native-protocol-tcp
load-module module-zeroconf-publish

load-module module-rtp-recv
load-module module-null-sink sink_name=rtp format=s16be channels=2 rate=44100 description="martin RTP Multicast Sink"
load-module module-rtp-send source=rtp.monitor

### Publish connection data in the X11 root window
.ifexists module-x11-publish.so
.nofail
load-module module-x11-publish
.fail
.endif


5) Finally, log out of your Gnome or KDE session, to come back to your login screen. Under 'session type' you should see a new option for 'Custom' next to KDE and Gnome. Login using that session type. Remember, Gnome or KDE will load up, depending on which you have put in .xsession . If you don't see that option, you may need to restart your computer, in order to restart your display manager. (If you're savy, you can just 'init 3' and then 'init 5' from the command line as root.)

6) One of the nice things about this configuration is that if either Pulse or Jack die, for any reason at all, you only need to open up a terminal and run the following commands.


$ killall pulseaudio ; killall jackd
$ jackd --realtime -d alsa & && pulseaudio &


Since they both now have sane default configurations, there is very little muck that has to be typed in at the command line to restart them.

Happy Hacking!

Palin Should be Impeached, Not Run for Elections

I don't normally use this blog to speak about politics, and certainly not in such definite negative terms. The American Election is a very flame heavy topic i've tried to leave alone, because i know i will probably run the foul side of pretty much everyone on Fedora Planet with my views. I like to keep my political views completely separate from work as much as possible. Occasionally I need to have my say though, and for many of the people who participate in this Planet, this is a very important topic.

To put it plain and simple, Governor Sarah Palin's email accounts were hacked and i'm assuming 99% of all Americans over 18 know about this. For international readers, the email accounts of Governor Sarah Palin, vice presidential candidate for the Republican party, were hacked. The hacker, an amateur, posted both the results and the attack vector online to 4chan, an internet forum, which led to the quick discovery of his identity. It turns out that he is the son of a democratic Senator, although since nothing has gone to court yet, no one knows what the penalties are.

I direct you to this soundbite from the McCain campaign:

"This is a shocking invasion of the governor's privacy and a violation of law."

Indeed, one reason why it's illegal is to protect people from using such private information to harass her and her family, by using cell phones numbers and other private information retrieved out of the email accounts. Most importantly though, this highlights the dangers of having private information leaked to anyone.

Which brings me to my next point. Sarah Palin should be impeached and removed from all political office. I direct you to the information leaked about the attack.

http://wikileaks.org/leak/sarah-palin-hack-2008/email-index.txt

If you have a look through those emails, some of them are personal or friendly, but in between there are numerous emails regarding State business. Anyone who works at Red Hat and is forced to use the bloatware called collaboration software, instead of being allowed to use a more convenient GMail account should automatically understand why this is a severe problem.

Yahoo email is neither private nor secure. The government can do all the handwaving magic it sees fit, but nothing will change the fact that Yahoo is an international company that presents a very high target to hackers. Thus Yahoo is 100% insecure. Furthermore, Yahoo has full access to emails they host and transmit. Therefore, there is no privacy, and any sensitive government issues could be revealed to the whole world. When you consider the information a VP is privy to, let alone a governor, the potential for misuse is scary. If you want to talk about radicals destroying a large nation for fun or profit, this is the way to go. If you want to discuss shady deals between politicians being revealed, carelessness like this will only make life much easier. Each state has an IT infrastructure as does the US Government, which is secured by one of the best security systems in the world. Using a Yahoo account is downright foolish.

Furthermore, privacy is also there to protect individuals from other malicious individuals. It's tragic, but a matter of ironic justice that Sarah Palin's family is suffering from the exact issues that are brought up when discussing the importance of privacy and data protection.

Yahoo mail is also run outside of the government systems. If you think the data retention policies of the Whitehouse have been bad in the past 8 years, just wait till Sarah Palin comes to power. They won't even exist! Data retention is the law for many reasons, least of all to track the doings of public officers and employees of public corporations. These rules are there to protect the American people and anyone doing business with Americans. While discussions about the data retention policies are reasonable, the bottom line is, without one, our individual rights are doomed. By using Yahoo mail, Sarah Palin has given the middle finger to the systems that keep politicians on the straight and narrow. As a politician who 'cleaned up' Alaska, she should understand this better than most politicians.

My take on this is that Palin didn't do this out of any malicious intent herself. I can imagine that she found the state provided system slow, complicated and onerous, so she decided to use an 'easier' solution. Given the magnitude of discovery, in most places i've worked, anyone who screwed up so bad would probably be fired. There is plenty of room here to make the 'ignorant/unintelligent soccer mom' argument, but she is still an elected politician, which means even a soccer mom would be smarter than that....

Despite this, it's clear that Sarah Palin knows very little about technology, the Internet, and most importantly the laws surrounding it. What ever reason she had for not using a state provided email address shows she thinks some need of hers trumps years of experience that IT professionals have had with privacy and security. On a personal note, it's insulting and disgusting. On a broader note, Sarah Palin is not fit to be a politician in the 21st century, let alone Vice President. Finally, it's clear she's doing something very wrong in Alaska already. She needs to be impeached.

Review Request

Can someone review this review request?
https://bugzilla.rpmfusion.org/show_bug.cgi?id=48

This is FreeJ in all its mighty glory. One nice thing about it is that the upstream developers have asked me to become a maintainer, just to keep the spec file upstream. The FreeJ developers certainly 'get' open source :).

RPMs of FreeJ for Fedora 9

I've been a bit pokey about this too, but I've finally compiled the BLAG Linux SRPMs of FreeJ for Fedora 9. Unfortunately, FreeJ depends on libffmpeg, and cannot go into Fedora proper. Nevertheless, it can have a very welcome place in RPMFusion or Livna.

To rip off the description from the homepage:

FreeJ is a vision mixer: an instrument for realtime video manipulation used in the fields of dance teather, veejaying, medical visualisation and TV.

FreeJ lets you interact with multiple layers of video, filtered by effect chains and then mixed together. Controllers can be scripted for keyboard, midi and joysticks, to manipulate images, movies, live cameras, particle generators, text scrollers, flash animations and more.
All the resulting video mix can be shown on multiple and remote screens, encoded into a movie and streamed live to the internet.

FreeJ can be controlled locally or remotely, also from multiple places at the same time; it can be automated via javascript to be operated via MIDI, Joysticks, wiimotes, mices and keyboards.

FreeJ can be found at its homepage, at: http://freej.org/

The RPMs are available on my fedorahosted space. Since the RPMs depend on 'non-free' software, although do not ship any non-free bits themselves, i have created a new non-free repo available at:

http://ynemoy.fedorapeople.org/repo.nonfree/

I would like to put this package up for review. To those maintainers responsible, would i be better off submitting a review to livna or to rpmfusion?

Ubuntu's Netbook packages available for Fedora

Jonathan Roberts has so kindly updated and rebuilt packages for optimizing Gnome to work on Netbooks and other small screen devices. These packages are a few tools for Gnome that have been written by a few developers inside Canonical that work on any Gnome system. I've updated my repo to host them for now, until they can pass the review. They are available at

http://ynemoy.fedorapeople.org/repo/

The SRPMs and SPEC files are there too, so feel free to rebuild them for other versions of Fedora or other RPM based distros. If you send the packages to me, i'll throw them up there in that space too.