Tag: this

Categories
Linux / Gentoo Linux

Fixing XML-RPC’s in KDE4

After posting that Bilbo worked out of the box, I was bound to run into a world of trouble – and so I did…

You see, when I was trying out KBlogger, I ran into the “Invalid request payload xmlrpc element STRING cannot be child of DATA” error you find in some KBlogger related posts. After playing around a little I realised that the error prevented me from posting anything and I ditched KBlogger as being buggy as hell (which it actually still is).

But when I installed Bilbo, I ran into the same problem! Only this time I knew that it should work as I posted before using Bilbo. After investigating a bit I found the culprit: KBlog. This backend library handles the communication with the server over the XML-RPC protocol. The error is produced on the server side and to be exact it is PHP XML-RPC that generates the error.

Because the XML-RPC library on the server is very strict (and probably error free), I turned to see what Bilbo actually sent to the server. The problem appears to be in the category structure that is generated. In my first posting, I did not select a category to post to. But when you select one or more categories, a QStringList object is populated with the category names. This is converted into XML somewhere in KDE (I haven’t figured out where yet).

After talking to the Bilbo devs for a bit they pointed me to the WordPress API in KBlog. For some reason, that one generates a string holding the XML for the server. And in that class, the metaWeblog.newPost function is called using a properly constructed category array.

Alas, switching to the WordPress API did not help: “STRUCT cannot be child of PARAM” and whether I used categories or not, the error remained. A quick inspection showed where that error comes from: the ‘struct’ element should not be directly put inside a ‘param’ element. The XML-RPC specifications state that the ‘struct’ should be placed in a ‘value’ element first. After fixing this and recompiling Bilbo, everything is working now!

Well, not everything as the metaWeblog API is still broken. I have filed a bug report with KDE upstream so they can add my fixes to the WordPress API and hopefully someone with more KDE programming experience can fix the metaWeblog API. Or rather, fix the conversion from a QStringList to proper XML-RPC compatible XML.

On a side note: while debugging the XML-RPC protocol I discovered that the Joomler XML-RPC plugin for Joomla! did not apply the categories that are sent along. I’ve modified my (already heavely patched) version of Joomler and once I’ve tested it a bit more, I’ll send the patches to the Joomler devs. Hooray for open source I guess ^^.

Categories
Programming

Release of new utility: Bacula Reports 0.9

I write a lot of code, most of it unsuitable for release to the public but this little gem is worth a public release.

After using Bacula to backup all my servers (both Windows and Linux) for some time, the large number of mailings you get when using it on a small server park drove me insane. Even when using filters to sort out new mail, it is hard to see if everything is going as it should be.

Enter Bacula Reports: a mail aggregator for Bacula 2.x and 3.x.

Bacula Reports consists of a faux mail command (which does not send out reports by mail but rather analyses and stores them) and a report generator which aggregates all the stored reports into one mailing with an overview and some HTML styling to make it more readable (if you don’t want HTML, modify the template to generate plain text).

By integrating the scripts into the Bacula configuration at 2 points (a mail command used for sending out reports and a job to send out the combined report), the storm of daily mails changes into one neat report at the end of the backup cycle.

Normal error messages and operator messages are unaffected and will be delivered as they used to be, only the backup reports per job are redirected to Bacula Reports.

Requirements:

  • A linux server (32 or 64 bit, tested on CentOS 5.2 and Gentoo 2008)
  • A working Bacula 2.x or 3.x installation
  • PHP as a command line interpreter (run ‘php –v’ to see if you have it)
  • 10 minutes of your time to set everything up

The cool thing of the scripts is that they require only 2 small changes in the director configuration to reroute the status mailings and if you don’t like it or run into trouble, reverting is normally a matter of simply commenting out the modified lines and restoring the old ones.

One drawback for some people: it requires PHP on the command line (as stated before). The reason for this is very simple: I want to use the same code in the future for a web GUI and my unix-scripting skills are virtually non-existing compared to PHP or Java.

Even though its PHP, the scripts have a small footprint and run very fast – they should be easy to add to any existing Bacula environment.

{jd_file file==5} {jd_file file==6}

Categories
Linux / Gentoo Linux

Postfix Queues and Amavis trouble

After I had not received mail for a couple of days I suddenly started to receive postmaster errors from one of the servers I maintained. The message told me the mail could not be delivered. On further inspection it seemed that this was a problem because postfix could not connect to [127.0.0.1].

This meant of course that some daemon on the server was out for lunch and it was screwing up the mail deliveries. Using ‘postqueue -p’ to inspect the queues I found out that 4000 messages where packed up in the queue, filling her up to the maximum I allowed.

After running ‘postsuper -r ALL’ (which means, re-queue all messages for instant delivery) I tailed the ‘/var/log/mail.log’ file to find out that 20% of the messages got delivered and the rest wound up back in line.

One error that kept flashing by was: ‘TROUBLE in check_mail: parts_decode_ext FAILED: run_command (open pipe): Can’t fork at…’. It had something to do with the FileIO class. Now the message came from Amavis and in fact, amavisd is nothing more than a Perl framework, hooking all kinds of filters together. This means its memory hungry but won’t totally crash upon hitting the floor.

This explained why a part of the messages made it through: a few threads fitted in memory, allowing for completing the delivery, however most mail got rejected once again and returned faithfully to the waiting queue. Another inspection of the queue confirmed this: after 2 attempts 3200 messages where still there.

Then it dawned on me: I had not enabled the swap memory! After fixing this (and adding 1GB swap to the 256MB of RAM) and reloading Amavis I tried once again to requeue all mail. This time the mail got through the filters in one piece (and well within the memory boundaries). I wish all issues with servers were as simple as this ^-^.

Categories
Japan Blog Study Tour

Universities and cool labs

IMG_3313 This morning we checked out of the Tsukuba Daily Inn after having a mixed style breakfast: it was not really Japanese because they had bread and it wasn’t really Western style because it had Japanese cabbage as well. We hopped on a bus back to Tsukuba Center and then on a bus to the Tsukuba University. Because of some miscommunication and some idiots we got off way too early and walked for 35 minutes until we finally found the right building (a campus for 17,000 students is huge, whoever came up with the idea that we should walk should be shot).

Continue reading “Universities and cool labs”