another new server

The old new server didn't work out at all. It had a weird glitch that when the memory usage went up to 100% and it started to want to try and use the swap the machine would lock up. Sometimes I could preempt this with rebooting, but other times it was stuck until I got home to hard power cycle it. Luckily I got it from someone at work and after we had tried different memory modules he took it back. It was lucky really since I'd been looking around and really wanted something a little better and was more into the idea of actually having a Home Theater PC (HTPC) in the rack. Especially since my daughter is starting to want to watch movies with us and I don't want to mess around with opening DVD's and stuff.

I did a little research and with some help from the egg heads at work came up with this:

• PHENOM(9XXX)QUAD,AM2+/AM2,780G,4DDR2,ATX
• Athlon 64 X2 Dc 5200+ AM2 2.7GHZ 1MB 65NM 65W 2000MHZ Pib
• CORSAIR XMS2 4GB ( 2 X 2GB ) PC2-6400 800MHz 240-pin DDR2 CL5 Dual Channel Desktop Memory Kit - TWIN2X4096-6400C5
• Coolmax 14616 V-400 ATX Power Supply

Pretty sweet for under $200! When it arrived I started pulling apart my old 4u case to put this new stuff in and while I was halfway through, I thought, you know... I really should just smoke test this new stuff before installing it. I dismantled more stuff while that thought was still nagging in my head and finally stopped and set it up on the bench. Sure enough it didn't work! I tried lots of things and couldn't get it to work. Filed a support ticket with the board manufacturer and put my old server back together. The next day I bought a new motherboard and power supply from Frys to test some theories. When I got them home they too didn't work and I finally hit jackpot when I tried each of the memory modules separately. One was bad! I ordered a whole new set and will mail one set back once I find two that work and then took the other stuff back to Frys.

Now my server was working!

This motherboard had so many awesome ports I was delighted, especially with the optical audio out, external SATA and the hdmi connector. I was sad it didn't have svideo and didn't have a com port either (I have an old X10 remote that has a serial port and also a weather station I plan to hook up at some point). However graphics cards with svideo are available for $20 (ATI X1050) and you can get usb to serial port adapters for under $10 so that's not a big deal. Once I read the manual I found out there were lots of headers for more ports so I got another 4 USB, 1 firewire and more audio ports for a $15 floppy disk sized header and ordered the official com port header (I thought about making my own, but it was just too much trouble).

Another thing that pleased me was that I could adjust the bios settings over the DVI port, which means I can mess with my system in the den, rather than dragging it into the bedroom where the only VGA display in the house is. This allowed me to find the second easter egg. Even though every bios ever made has allowed you to set what happens when the power comes back on, my old motherboard didn't. I was delighted I could make my server come back on automatically after a power cut with the new board. Yipee. Of course once I put the X1050 graphics card in there I couldn't access the bios at all on either DVI port, but at least once the system booted I could see the Linux console on the X1050 DVI port. Good signs I'll be able to get svideo working at some point.

So now I'm setting up the new machine and I might make one or two posts about that as time goes on. I will leave you with this quick (and dirty) performance number. The old server reported just under 1,800 BogoMips and the new system with 2 core's clocks in at 10,800 (that's 5,400 per core). Now to find something to do with all that power.

a new server

I'm moving to a new server at home sometime soon, so if things go screwy, that's why. I dropping the huge 4U rack mounted beast I've posted about before. In its place is going to be a teeny shuttle case with one PCI slot which is really all I need for my security cam capture cards. I'm going through the stages of setting up ubuntu 8 (hardy) on it right now. I got the PC for $120 which I thought was a deal. I ordered a 1TB SATA drive ($100!) to throw in there and it'll just have one drive for now. It has room for another drive, so at some point I might add another 1.5TB or so. If I need it.

Installing ubuntu was a doddle, I installed the server version since that has the longest support lifetime. To get it usable I'm listing the packages I installed incase I need to do it again:

sudo apt-get update
sudo apt-get upgrade
sudo apt-get install gcc python-dns binutils-doc autoconf automake1.9 bison flex gcc-doc gcc-multilib gdb libtool make manpages-dev gcc-4.2-doc gcc-4.2-locales gcc-4.2-multilib libgcc1-dbg libgomp1-dbg libmudflap0-4.2-dev libmudflap0-dbg libc6-dev libc-dev
apt-get install gcc python-dns binutils-doc autoconf automake1.9 bison flex gcc-doc gcc-multilib gdb libtool make manpages-dev gcc-4.2-doc gcc-4.2-locales gcc-4.2-multilib libgcc1-dbg libgomp1-dbg libmudflap0-4.2-dev libmudflap0-dbg
sudo apt-get install mailman
sudo apt-get install smokeping curl libauthen-radius-perl libio-socket-ssl-perl libnet-dns-perl libnet-ldap-perl libnet-telnet-perl libsocket6-perl libio-socket-inet6-perl echoping
sudo apt-get install apache2 ssh libapache2-mod-python php5 libapache2-mod-perl2 bind9 mailman screen tkdiff xauth mailx screen cvs python-cheetah python-mysqldb pychecker x11-apps resolvconf libapache2-mod-python-doc python-egenix-mxdatetime python-mysqldb-dbg tclreadline mesa-utils pdksh libbsd-resource-perl xterm

spec's are: Celeron 420, 1 GB of DDR2 RAM, Shuttle KPC K45 SFF

Moving to google code

I'm slowly moving all my code over to being hosted Google Code. This will allow me to learn how to use subversion and also allow me to have RSS Feeds for my checkins. I can also have a wiki page per project and then let this blog be a blog about updates to each of the projects (or more likely new projects, I mean, really, who wants to maintain their old code).

I've already checked in some stuff, some of it hasn't ever been released before! Go check it out at: http://wtwf.googlecode.com/

If you want to follow the feed of stuff I check in the feed url is: http://code.google.com/feeds/p/wtwf/svnchanges/basic

If you use google reader, you can get a preview of what the source code changes feed looks like in google reader.

So far I'm liking it a lot. You can't beat the price!

Perhaps one day I'll even have people submit fixes (yay!), or file bugs (boo!) using the google code interface.

Shutup Jaiku

In my last post I bitched about the Jaiku IM gateway and how it was too chatty and all I wanted to do was post to Jaiku but not hear all the feed noise in my chat windows. Well a quick poke around and I found it was close to trivial to write a jabber robot. I used PyGtalkRobot which uses xmmppy (needed to install this from source) and pydns (the one from apt-get was fine for me).

Within an hour I had something working but it needed the Jaiku API key, not your Jaiku password. Some more hacking and I had a code snippet that given a Jaiku username and password would find the Jaiku API key. Since I'd found some useful snippets on snippets.dzone.com I uploaded my own useful snippet.

Meanwhile a helpful person suggested the ping.fm or imified bots but I couldn't work out what they did or didn't have a beta invite so I just kept plugging away at my thing. A little bit more time of cleanup and adding options and some minimal documentation and it was all good to go.

Normally I usually just release this stuff and let you run it. But this time I've decided to run it as a service and see how it goes. However beware! you'll be giving me your Jaiku user name and password. You really shouldn't do that, but you can trust me.

So just add shutupjaiku@wtwf.com as a talk contact and send it the same sign in you would to jaiku@jaiku.com

sign in username password

and then start sending messages and they'll show up on your Jaiku page. That address is ONLY used for chat, do not send it mail no one will read it.
Obviously there's lots of ways this little server could go, right now it does what I need.

I was delighted how easy PyGtalkRobot and the Jaiku API made everything. I was little irked there wasn't a nice way to get a Jaiku API key from a user name and password and also a few things irked me about PyGtalkRobot. Spitting out lots of output without using the logging api. Sending lines longer than 80 chars. And the crazy way you specify handlers, using the doc string to contain the regex that triggers the handler is just bizarre! Why not just register handlers like Python Httpd servers do?

If you'd like to download the source and run your own copy go for it! Hope you find it useful.

unwelcome user agents

The following useragents are not welcome on my website:

SetEnvIfNoCase User-Agent "^Biz360" bad_bot
SetEnvIfNoCase User-Agent "^Blogslive" bad_bot
SetEnvIfNoCase User-Agent "^Cazoodle" bad_bot
SetEnvIfNoCase User-Agent "^EmailSiphon" bad_bot
SetEnvIfNoCase User-Agent "^FeedLounge" bad_bot
SetEnvIfNoCase User-Agent "^OmniExplorer" bad_bot
SetEnvIfNoCase User-Agent "^Sphere" bad_bot
SetEnvIfNoCase User-Agent "^SurveyBot" bad_bot
SetEnvIfNoCase User-Agent "^edgeio" bad_bot
SetEnvIfNoCase User-Agent "^ia_archiver" bad_bot
SetEnvIfNoCase User-Agent "^nutch" bad_bot
SetEnvIfNoCase User-Agent "^panscient.com" bad_bot
SetEnvIfNoCase User-Agent "^ping.blo.gs" bad_bot
SetEnvIfNoCase User-Agent "^topicblogs" bad_bot
SetEnvIfNoCase User-Agent "^Moreoverbot" bad_bot
SetEnvIfNoCase User-Agent "^BlogSearch" bad_bot
SetEnvIfNoCase User-Agent "Twiceler" bad_bot
SetEnvIfNoCase User-Agent "^BlogPulse" bad_bot
SetEnvIfNoCase User-Agent "FreeMyFeed" bad_bot
# SetEnvIfNoCase User-Agent "^Java" bad_bot

I didn't have the courage to deny all Java folks.

Then I just have this in my apache config

<Directory /home/ark/html/>
   Order allow,deny
   allow from all
   Deny from env=bad_bot
</Directory>

I'll try and keep this post up to date. Mostly you get on this list if you're a robot that's crawling (and indexing) my rss feeds.

Anonymous mailman lists

I have a need for an anonymous mailman list and here is how I set it up.

1. create the list (for me I had to run /usr/lib/mailman/bin/newlist the web ui didn't work for me). In this example I'm going to use the name "privatelist".
   
2. In the web ui, on the general options page, under "Hide the sender of a message, replacing it with the list address (Removes From, Sender and Reply-To fields)" select Yes
3. In the web ui under "Archiving Options" just say NO to "Archive Messages?
4. In the web ui under "Digest options" select NO to "Can list members choose to receive list traffic bunched in digests?"
5. modify /usr/lib/mailman/Mailman/Handlers/Cleanse.py to add the following underneath if mlist.anonymous_list:

   
   del msg['received-spf']
   del msg['authentication-results']
   del msg['domainkey-signature']
   del msg['dkim-signature']

6. rm Cleanse.py[oc]
7. python -O -c 'import py_compile; py_compile.compile("Cleanse.py")'
8. python -c 'import py_compile; py_compile.compile("Cleanse.py")'
   
9. cd /var/lib/mailman/lists/privatelist; rm digest.mbox; ln -s /dev/null digest.mbox
10. remove /var/lib/mailman/archices/*/privatelist
11. replace /var/lib/mailman/archives/private/privatelist.mbox/privatelist.mbox with a link to /dev/null
12. restart mailman
    

That's what I did, post to your list with a different email address as a member and then view the message in your recipient's account. Make sure to view all headers and see if there is anything in there that might indicate the original sender. There may be something I missed, if so let me know!

There go my nines

I'm pretty sure no one noticed, but my server was down for quite some time this weekend...

Short Version:

Before	After
Linux Server Pentium III 600Mgz 256Meg Ram Beige Dell Box on a shelf Windows Server AMD Athlon 1Ghz 512 Meg ram Sweet 4U rack mount case Running Security Cam Software	Linux Server AMD Athlon 1Ghz 512 Meg ram Sweet 4U rack mount case Running webserver, email, Security cam Software, stuff

Long Version:
But I think it's back up for a while now and it's better than ever!
Someone at work was selling an old Motherboard, processor, 0.5Gig memory and a fancy graphics card for cheap, so I made a low offer, told them I hoped they got a more reasonable offer and then ended up 'winning' it. It was caked in some horrible brown gunk which I suspect was due to someone smoking in the house. I powered down my server (a really old dell Pentium 3 600Mhz), threw in the new motherboard and powered it up. I had heard about dell power supplies being wired weird but thought my system predated that silliness. It turns out it didn't. So now I needed a new case and power supply (of course a standard power supply wouldn't fit in a dell case!). I figured I should finally do it right and get a rack mount so the computer could actually fit in the 17" rack I have in my AV closet (where my current server just sits on a shelf like a dork). Boy those things are expensive I was looking at paying over $150 for a not particularly nice one that was likely going to be a total hassle (getting cards to fit in it) since I didn't have room for another 4U case in the rack. You see I already have a 4U server in the rack, it was my old home theater PC and has since been re purposed to run the security software than records my security cameras. This machine had been down for a little while ever since I needed a windows PC to fix the dead disk in my TiVo. A TiVo backup that was too old to update itself caused me to have to buy instant cake which I begrudgingly did, but it ended up working so well I should have done that right from the start, I did feel like a failed TiVo hacker for not being able to do it myself though. But since the machine was down and out, I plonked the new board on top, hooked up the ATX power supply and booted it up and sure enough it worked fine. Even better the fancy graphics card sent signal to the DVI connector from the start s I could set bios settings with it connected to my apple display (DVI only) in the den. Before that I'd have to lug my machine into the bedroom, balance it on a wooden stand and connect it to the LCD TV in there (the only display in the house to accept VGA input!). I emailed who I bought the motherboard from and told them it worked fine. I decided to setup my new server in parallel with my old one (minimum downtime!) and installed ubuntu on a 200 gig drive I had, I couldn't get the damn thing to boot, I tried everything, other drives, swapping master/slave, all sorts of bios jiggery pokery, nothing worked, also the bios power on cycle took a really long time which made it more frustrating. In the end I gave up, emailed the seller with the bad news and fortunately they agreed to take it back. I still had the idea of putting my linux server in a rack case, and it always bothered me that I ran a windows server at home, The only reason I ran it was because I had convinced myself no Linux drivers could possibly exist for my 4 input video capture card. A quick Google search and I was proved wrong, quickly negating the need for a windows server, I started moving the drives over to my 4U case and they kept having disk errors so I reinstalled Ubuntu on a spare 200 gig drive and got ssh working and then started copying over all the files I'd need to get my server back up and running. Finally it's all working great! I like the new security software since it's free and open source and I can tweak it if I need to. I really like that I have one less sever and a windows one at that, so my AV Closet is much tidier now!

Everything feels right about this.

Here's the Ubuntu packages I had to install to make it usable for me:

apache2 ssh libapache2-mod-python2.4 php5 libapache2-mod-perl2 bind9 mailman screen tkdiff xauth mailx screen xlogo cvs python-cheetah python-mysqldb pychecker

TODO(ark) find xv somewhere

Here's the configuration files I had to copy over, I really should make sure these are backed up:

• /etc/apt/sources.list
• /etc/apache2
• /etc/bind
• /etc/postfix
• /etc/init.d/postfix
• /etc/mailman
• /var/lib/mailman
• /usr/lib/mailman