Linux / Gentoo Linux

Fixing Midnight Commander Locale errors

If you have a pretty recent Linux system like I have, you most likely have had the pleasure or the following message when starting mcedit:

Chosen display charset or source codeset does not match one set via locale.

You can then “Fix” it or “Skip” and choose to remember it. For me, fixing or skipping the dialog does not seem to do anything but that is most likely because I don’t use the charset completely (no ‘weird’ characters in my text files). Even worse, if you find it to do something — chances are you want to use the Fix option — and tick “Do not ask again”, the next time the file is broken again! This is because the tick box does not respect your choice but rather takes it as “Skip any problem detection in the future”…

Annoying as it may be, the real cause is the fact that you are most likely not using the correct character encoding on your system by default. Chances are, when you update your system you get this annoying “Generating code page” message a million times over. If so, you just generated support to view every text encoding known to man — good for you!

I on the other hand, will only view UTF-8 files and I’m not really interested in anything else. So I decided to reduce codepage support to 2 languages: Dutch and English. Note that the Dutch support in this case tends to translate messages in programs (ugh) but it also adds support for the Euro sign: ‘€’

Open up /etc/locale.gen and add the following lines:

en_GB ISO-8859-1
en_GB.UTF-8 UTF-8
nl_NL UTF-8
nl_NL@euro UTF-8
nl_NL.utf8 UTF-8

I cheated a bit by replacing the default ISO-8859-1 or ISO-8859-15 character mapping for Dutch with the UTF-8 one. It seems this choice is allowed and gives me proper UTF-8 support with the nl_NL locale.

Save the file and run locale-gen:

cyberxps ~ # locale-gen
* Generating 5 locales (this might take a while) with 1 jobs
*  (1/5) Generating en_GB.ISO-8859-1...   [ ok ]
*  (2/5) Generating en_GB.UTF-8...   [ ok ]
*  (3/5) Generating nl_NL.UTF-8...   [ ok ]
*  (4/5) Generating nl_NL.UTF-8@euro...   [ ok ]
*  (5/5) Generating nl_NL.UTF-8...   [ ok ]
* Generation complete

Looks familiar? Yes that is the dreaded locale generation but this time it only created the locales we will use meaning it completes in a second instead of 10 minutes… Run locale -a to make sure you got what you wanted:

cyberxps ~ # locale -a

Now open up /etc/env.d/02locale and put the locale selection in and reload afterwards:

cyberxps ~ # echo 'LANG="nl_NL.utf8@euro"' >> /etc/env.d/02locale
cyberxps ~ # env-update >> source /etc/profile
>>> Regenerating /etc/
cyberxps ~ # export | grep LANG
declare -x LANG="nl_NL.utf8@euro"

All done, we switched to the new locale and mcedit can stop complaining now. As an added bonus each file you edit will be created in UTF-8.

Linux / Gentoo Linux

Debugging nVidia EDID resolutions

After installing an nVidia card into my HTPC I ran into a problem I never had before: the driver will not allow me to use the full 1280×720 resolution of my TV. After running xrandr on the console, I can confirm that Xorg reports the following supported resolutions:

Screen 0: minimum 320 x 240, current 720 x 480, maximum 1920 x 1080
default connected 720x480+0+0 0mm x 0mm
720x480        50.0*    51.0
680x384        52.0     53.0
640x480        54.0     55.0
512x384        56.0
400x300        57.0     58.0
320x240        59.0
480i/60        30.0
480p/60        60.0
720p/60        60.0
1080i/60       30.0
1080p/60       60.0

Now if you read the above closely, you can see that the display should have modes up to 1920×1080 (interlaced in my case but still). This proves that in fact the EDID information (which should allow your graphics card to auto-detect monitor properties) is coming through but that it is not working properly. Read on to solve this problem.

Linux / Gentoo Linux

Solving VMware HD trashing

VMware Server 1.x and 2.x are great for some quick and easy visualization. Granted, they lack some options their big brothers have (live migration for example) but overall they are fine. It is quite annoying to find one of your virtual servers trashing the IO on your host in such a way that all systems (virtual and physical) come crawling to a halt.

Note that I am not talking about actual throughput caused by a heavy loaded system. I am talking about a few hundred kilobytes to a few megabytes of disk throughput that manage to clog up the host system while the virtual system is not doing anything (no network activity, no CPU activity, no disk IO on the virtual side and still the host gets trashed).

It seems like the trashing can come from clashes between the memory manager of VMware and your virtual OS and/or the virtual drive controller and the guest disk driver. The result seems a looping ‘optimization’ which never ends and instead of speeding up the system, it completely grinds down to a halt.

The solution? Well there isn’t a clear cut solution to all the problems out there but so far this one seems to work for me. It tells VMware to allocate all the memory it could be needing (instead of allocating it when its needed) and disables the paging of memory completely (as everything fits in one go).

File /etc/vmware/config:

prefvmx.useRecommendedLockedMemSize = "TRUE"
prefvmx.minVmMemPct = "100"

Guest “.vmx” file:

sched.mem.pshare.enable = "FALSE"
mainMem.useNamedFile = "FALSE"
MemTrimRate = "0"
MemAllowAutoScaleDown = "FALSE"
Linux / Gentoo Linux

Wine sound errors

After a while, the sound errors from Wine started to annoy me as they seem to indicate something is wrong while it is not. At the start of each program I see the following errors:

err:alsa:ALSA_CheckSetVolume Could not find 'PCM Playback Volume' element
ALSA lib pcm_dmix.c:935:(snd_pcm_dmix_open) The dmix plugin supports 
 only playback stream
err:alsa:ALSA_CheckSetVolume Could not find 'PCM Playback Volume' element
ALSA lib seq_hw.c:457:(snd_seq_hw_open) open /dev/snd/seq failed: No such 
 file or directory

The first errors are related to my Intel HD audio card which has multiple PCM channels. The result is that none of them are called ‘PCM Playback Volume’ and Wine seems to be hard-coded to search for that specific channel to check upon starting.

Harmless but annoying none the less. A bit of searching seems to indicate that this is something I can’t fix.

The second error about dmix seems to mean that the dmix soundcard that Wine is using does not support recording sound. This is correct as it is a softmix device used to let multiple programs ‘claim’ the soundcard without locking each other out. Beats the hell out of the mess called PulseAudio and was a breeze to set up.

The 3rd error is MIDI related: Wine is trying to access the sequencer device. I can’t figure out why Wine has no option to disable midi support altogether but without that device you will keep getting those errors.

To fix this, enable the MIDI drivers in ALSA and load them on boot, the driver will be called ‘snd-seq-device’. In my case this was found in (kernel 2.6.30):

Device Drivers
<*> Sound card support ---->
    <*> Advanced Linux Sound Architecture --->
        <*> Sequencer support
Linux / Gentoo Linux

Restoring Exchange 2003/2008 using Bacula

Because the instructions in the Bacula documentation left me hanging on how to actually restore the Exchange data from a backup, I am writing this little summary after extracting all information from the mailing lists.

In my case, I have one server called ‘axmail-fd’ running Exchange 2003 and another server called ‘axemail-fd’ running Exchange 2003 SP2. The following steps are needed to restore into the Recovery Storage Group on the new server in order to migrate the mail from the old to the new server.

To prepare, please create the Recovery Storage Group on the target Exchange server. Create the database name you want to recover, for example “Mailbox Store (AXMAIL)”, which will generate a “Mailbox Store (AXMAIL).edb” in the Recovery Storage Group folder. Clear out any log files or other remnants of previous restores as Exchange tends to get confused if data from multiple databases are in there.

Start bconsole, select the restore mode and select the Exchange backup to restore:

First you select one or more JobIds that contain files to be restored. 
You will be presented several methods of specifying the JobIds. Then 
you will be allowed to select which files from those JobIds are to be restored.
To select the JobIds, you have the following choices:     
1: List last 20 Jobs run                             
2: List Jobs where a given File is saved             
3: Enter list of comma separated JobIds to select    
4: Enter SQL list command                            
5: Select the most recent backup for a client        
6: Select backup for a client before a specified time     
7: Enter a list of files to restore                       
8: Enter a list of files to restore before a specified time     
9: Find the JobIds of the most recent backup for a client      
10: Find the JobIds for a backup for a client before a specified time    
11: Enter a list of directories to restore for found JobIds              
12: Cancel                                                           
Select item:  (1-12): 5                                                 
 Defined Clients:                                                              
4: axmail-fd     
10: axemail-fd
Select the Client (1-10): 4

The defined FileSet resources are:
1: AXMAIL Full Data Set
2: Exchange
Select FileSet resource (1-2): 2
| JobId | Level | JobFiles | JobBytes      | StartTime           | VolumeName                    |
|    90 | F     |       13 | 5,313,968,371 | 2009-06-24 15:36:10 | Deventer_Exchange_Backup_0013 |
|    90 | F     |       13 | 5,313,968,371 | 2009-06-24 15:36:10 | Deventer_Exchange_Backup_0014 |
|    91 | I     |        5 |     2,671,174 | 2009-06-24 17:28:25 | Deventer_Exchange_Backup_0014 |
|    92 | I     |        5 |       233,882 | 2009-06-24 18:00:01 | Deventer_Exchange_Backup_0014 |
|   118 | I     |       17 |    40,099,025 | 2009-06-25 18:00:02 | Deventer_Exchange_Backup_0014 |

You have selected the following JobIds: 90,91,92,118
Building directory tree for JobId(s) 90,91,92,118 ...
24 files inserted into the tree.

Now we want to select the entire First Storage Group to restore, except for the Public Folders store.

Note: If you have the mailbox store defined, perhaps the restoration will work. An old mailing list conversation in 2008 stated that it was only possible to restore one database at a time – hence we now unmark the Public Folder.

You are now entering file selection mode where you add (mark) and
remove (unmark) files to be restored. No files are initially added, unless
you used the "all" keyword on the command line.                           
Enter "done" to leave this mode.                                          
cwd is: /$ mark *                                       
29 files marked.                               
$ cd "@EXCHANGE/Microsoft Information Store/First Storage Group"
cwd is: /@EXCHANGE/Microsoft Information Store/First Storage Group/
$ unmark Public*                                                   
4 files unmarked.                                                  
$ lsmark                                                           
*C:\Program Files\Exchsrvr\mdbdata\E0002FC5.log                    
*C:\Program Files\Exchsrvr\mdbdata\E0002FC6.log                    
*C:\Program Files\Exchsrvr\mdbdata\E0002FC7.log                    
*C:\Program Files\Exchsrvr\mdbdata\E0002FC8.log                    
*C:\Program Files\Exchsrvr\mdbdata\E0002FC9.log                    
*C:\Program Files\Exchsrvr\mdbdata\E0002FCA.log                    
*C:\Program Files\Exchsrvr\mdbdata\E0002FCB.log                    
*C:\Program Files\Exchsrvr\mdbdata\E0002FCC.log                    
*C:\Program Files\Exchsrvr\mdbdata\E0002FCD.log                    
*C:\Program Files\Exchsrvr\mdbdata\E0002FCE.log                    
*C:\Program Files\Exchsrvr\mdbdata\E0002FCF.log                    
*C:\Program Files\Exchsrvr\mdbdata\E0002FD0.log                    
*C:\Program Files\Exchsrvr\mdbdata\E0002FD1.log                    
*C:\Program Files\Exchsrvr\mdbdata\E0002FD2.log                    
*C:\Program Files\Exchsrvr\mdbdata\E0002FD3.log                    
*C:\Program Files\Exchsrvr\mdbdata\E0002FD4.log                    
*C:\Program Files\Exchsrvr\mdbdata\E0002FD5.log                    
*C:\Program Files\Exchsrvr\mdbdata\E0002FD6.log                    
*Mailbox Store (AXMAIL)/                                           
*C:\Program Files\Exchsrvr\mdbdata\priv1.edb                       
*C:\Program Files\Exchsrvr\mdbdata\priv1.stm                       
$ done

Bootstrap records written to /var/bacula/axnet-dir.restore.23.bsr  
The job will require the following   
Volume(s)                 Storage(s)                SD Device(s)
Deventer_Exchange_Backup_ File                      FileStorage

25 files selected to be restored.

Run Restore job
JobName:         RestoreFiles
Bootstrap:       /var/bacula/axnet-dir.restore.23.bsr
Where:           /tmp/bacula-restores
Replace:         always
FileSet:         Empty FileSet
Backup Client:   axmail-fd
Restore Client:  axmail-fd
Storage:         File
When:            2009-06-26 00:44:50
Catalog:         MyCatalog
Priority:        10
Plugin Options:  *None*

We now need to change the target to the new server and clear out the Where setting:

OK to run? (yes/mod/no): m
Parameters to modify:    
1: Level    
2: Storage    
3: Job    
4: FileSet    
5: Restore Client    
6: When    
7: Priority    
8: Bootstrap    
9: Where   
10: File Relocation   
11: Replace   
12: JobId   
13: Plugin Options
Select parameter to modify (1-13): 5                

The defined Client resources are:    
1: bartje-fd    
2: nakor-fd    
3: hermione-fd
4: snape-fd
5: hagrid-fd
6: axnet-fd
7: axweb-fd
8: axmail-fd
9: axexact-fd
10: axklant-fd
11: axemail-fd
Select Client (File daemon) resource (1-11): 11

Run Restore job
JobName:         RestoreFiles
Bootstrap:       /var/bacula/axnet-dir.restore.23.bsr
Where:           /tmp/bacula-restores
Replace:         always
FileSet:         Empty FileSet
Backup Client:   axmail-fd
Restore Client:  axemail-fd
Storage:         File
When:            2009-06-26 00:44:50
Catalog:         MyCatalog
Priority:        10
Plugin Options:  *None*
OK to run? (yes/mod/no): m

Parameters to modify:    
1: Level    
2: Storage    
3: Job    
4: FileSet    
5: Restore Client    
6: When    
7: Priority    
8: Bootstrap    
9: Where   
10: File Relocation   
11: Replace   
12: JobId   
13: Plugin Options
Select parameter to modify (1-13): 9
Please enter path prefix for restore (/ for none): /

Run Restore job
JobName:         RestoreFiles
Bootstrap:       /var/bacula/axnet-dir.restore.23.bsr
Replace:         always
FileSet:         Empty FileSet
Backup Client:   axmail-fd
Restore Client:  axemail-fd
Storage:         File
When:            2009-06-26 00:44:50
Catalog:         MyCatalog
Priority:        10
Plugin Options:  *None*
OK to run? (yes/mod/no): y

Job queued. JobId=132
You have messages.

Notice how we did not use any renaming on the database paths: if Exchange 2003 has a Recovery Storage Group defined, that group will receive any restores, making manual adjustment of the paths unneeded.


Error 0x7fe1f42

If you get this error: “Error: HrESERestoreAddDatabase failed with error 0xc7fe1f42 – Database not found. Check that the Database you are trying to restore actually exists in the Storage Group you are restoring to”. You have not created the database in the Exchange manager in the Recovery Storage Group. Make sure you created the database in the manager and check the name.

Error 0xc7ff1004

I ran into this message after figuring out how to restore the data: “Error: HrESERestoreComplete failed with error 0xc7ff1004 – Unknown error”. The error is given by the FD after the data has been restored and the FD crashed after that.

You can run the eseutil.exe against the .edb file to check the state (use eseutil /mh filename.edb) and you will probably see the state as “Dirty Shutdown”.

The cause is a problem during the backup and is making the restore fail. If you have all the log files (E00xxxx.log in ‘restore’ in the Recovery Storage Group folder) you can use the eseutil /cc restore command to replay the log files and fix the database.

After replaying the logs, the database should mount fine and all the mail should be there.

Linux / Gentoo Linux

Growing LVM storage

After adding some new storage to the backup system at work, I needed a way to transfer the roughly 400GB of backups to the new installation – without the use of a USB disk or another server (would take way too long). So after installing CentOS 5.3 with LVM2 enabled, I dedicated the better part of 1TB to the new backup storage on the new hard drive.

After mounting the old storage, I copied the old archives to the new LVM storage and did the following.

Prepping partition for LVM

Use fdisk to change the partition type to ‘Linux LVM’, hex code ‘8e’: fdisk /dev/sdb

Then create the structure needed for use with LVM on the new partition: pvcreate /dev/sdb1

Finally, add the new LVM partition to the LVM caches by letting the automated scan picking it up: vgscan

Now to actually append the new partition to the storage pool: vgextend VolGroup00 /dev/sdb1

Adding to LVM

If you run vgdisplay, you should now see the space you just added as free space in the LVM storage pool.

You can now add the free space to one or more storage volumes that need it: lvextend -L +353.5G /dev/VolGroup00/LogVolBackups

Finally, let the filesystem on the drive resize to fill the new space: resize2fs /dev/VolGroup00/LogVolBackups

And there we go, 353.5GiB added to the LVM2 storage. Now next time I won’t think twice to use LVM2.

Linux / Gentoo Linux

Fixing XML-RPC’s in KDE4

After posting that Bilbo worked out of the box, I was bound to run into a world of trouble – and so I did

You see, when I was trying out KBlogger, I ran into the “Invalid request payload xmlrpc element STRING cannot be child of DATA” error you find in some KBlogger related posts. After playing around a little I realised that the error prevented me from posting anything and I ditched KBlogger as being buggy as hell (which it actually still is).

But when I installed Bilbo, I ran into the same problem! Only this time I knew that it should work as I posted before using Bilbo. After investigating a bit I found the culprit: KBlog. This backend library handles the communication with the server over the XML-RPC protocol. The error is produced on the server side and to be exact it is PHP XML-RPC that generates the error.

Because the XML-RPC library on the server is very strict (and probably error free), I turned to see what Bilbo actually sent to the server. The problem appears to be in the category structure that is generated. In my first posting, I did not select a category to post to. But when you select one or more categories, a QStringList object is populated with the category names. This is converted into XML somewhere in KDE (I haven’t figured out where yet).

After talking to the Bilbo devs for a bit they pointed me to the WordPress API in KBlog. For some reason, that one generates a string holding the XML for the server. And in that class, the metaWeblog.newPost function is called using a properly constructed category array.

Alas, switching to the WordPress API did not help: “STRUCT cannot be child of PARAM” and whether I used categories or not, the error remained. A quick inspection showed where that error comes from: the ‘struct’ element should not be directly put inside a ‘param’ element. The XML-RPC specifications state that the ‘struct’ should be placed in a ‘value’ element first. After fixing this and recompiling Bilbo, everything is working now!

Well, not everything as the metaWeblog API is still broken. I have filed a bug report with KDE upstream so they can add my fixes to the WordPress API and hopefully someone with more KDE programming experience can fix the metaWeblog API. Or rather, fix the conversion from a QStringList to proper XML-RPC compatible XML.

On a side note: while debugging the XML-RPC protocol I discovered that the Joomler XML-RPC plugin for Joomla! did not apply the categories that are sent along. I’ve modified my (already heavely patched) version of Joomler and once I’ve tested it a bit more, I’ll send the patches to the Joomler devs. Hooray for open source I guess ^^.

Linux / Gentoo Linux

Bilbo: Finally a decent blogging program for KDE?

Bilbo interface While the development on KBlogger has somewhat picked up and some features have been pushed back to allow for a semi-working release, I just found a little gem hidden on the big wide web.

In a mailing list post on the development on KDE, someone is suggesting to take a look at Bilbo instead of KBlogger for the ‘extra-gear’ package. Of course, I was expecting a simple blog editor similar to all the other projects out there that never quite made it. Boy, was I in for a surprise!

Bilbo has the features I need in a blogging program (similar to the Windows Live Writer interface) and even better, it should be in working order!

I will report back once I get it to install on KDE 4.3 using Gentoo.

Linux / Gentoo Linux

Kblogger coming to KDE 4.3

In a previous article I wrote about KBlogger and its nomination for a vapor-ware award. Most of the commits up until then seemed like KDE global search and replace changes, rather than actual work on the KBlogger application itself.

On the Kblogger page on kde-apps, an anouncement was made that the 3rd alpha was released in Febuari 2009. After installing and playing around for a bit, it felt more like a working blogging tool than it did before.

The downside was the numerous features which didn’t work:

  • Settings autodetection leaves you hanging and makes setting up a blog account more of a guess than anything else
  • If kwallet is not installed or activated (the service needs to run), kblogger crashes
  • If you try to read the category list on a Moveable Type API site, you get a ‘Method not supported’ error
  • Reading previously posted articles works (sort of) unless you have media attached in which case it crashes

The biggest problem is not the continuous crashing of the application but rather that I haven’t been able to post one article on any of my sites using Kblogger – it truely is an alpha quality application, if that.

It seems like the development is still going, even if the kde-apps page and homepage seem to be dead. Hopefully, they will work out the crashes and get it going in some form before KDE 4.3 ships out in June 2009.

Because of the current state this seems doubtful but according to the plans, it is still a part of KDE 4.3 PIM-libs – although it was on the schedule for 4.0, 4.1 and 4.2 as well.

In a desperate attempt to make the KDE 4.3 deadline, the current Kblogger feature plan seems to have dropped support for media management. This means that a lot of the current issues become a problem for later on and the core of the program (the blogging engine) is the only thing that needs to be fixed.

Lets hope this June we get a pleasant surprise but I am currently putting my money on KDE 4.4 or even KDE 4.5 as a release platform. On the bright side, perhaps the SVN versions become stable enough to use in the mean time.

On a side note, I am still looking for a blogging program for KDE 🙂



Linux / Gentoo Linux

KNetworkConf in KDE4 and Gentoo Baselayout-2

For a while now, I wondered why they ship KNetworkConf with KDE4 if its not working at all. The GUI shows up fine but none of the interfaces actually show up (making it pretty hard to configure them).

Today I found out why: the configuration panel is not incomplete, it is incompatible. You see, Gentoo is starting to switch to baselayout-2, a new and improved system layout which should solve a million quirks that have been part of the system for years now.

According to this bug report on, this is because the network listing is no longer the way it was: it is now a new kind of array (bash compatible or something).

Marvelous as this may be, it breaks knetworkconf in KDE 4.2.2 and older. Lets just hope that the new (service) release fixes this ‘ minor’ glitch.