Happy Software Engineer

2007-04-13T09:34:00.001+02:00

I have decided to reactivate XEland project. XELand is a small program, which generates night landscapes as stereo pairs for cross-eye viewing. Original version was written in C++/Gnome and had continually problems with compilation (due to lots of dependencies). Java version is of course free of such problems. Currently it uses two map generation methods (plus some variations). Generated height maps are seamless and can be saved as PNG images. The program requires only JDK 1.5 or above, and is very easy to use. To take full advantage of XEland you should learn cross-eye viewing. Support for anaglyph is expected soon.

Java Generics: Instantiating Objects Of Type Parameter Without Passing Class Literal To Instance

2007-03-25T22:53:00.000+02:00

Cause generics in Java 1.5 and 1.6 are implemented using erasure, in most cases you can not get class object for type parameter, which means that in your generic classes you can not instantiate objects of type parameters. Usually this is solved by special setup of generic class instance (by passing class literals to methods which instantiate objects of type parameter class) but it does not look too god. Fortunately, you can avoid such ugly code in some situations and make generic class smart enough to handle it by itself. More...

C/GTK or Java/Swing Stopwatch

2007-03-23T08:58:00.000+01:00

Maybe it's hard to believe but there is not too much stopwatch programs for Linux. So I have written one by myself. First was the C/GTK version. Coding it was rather painful for Java programmer accustomed to comfortable Eclipse IDE. So next day I decided to implement same functionality in Java/Swing. The Java version supports simple task tracking and can be started from the command line, via WebStart, or used as applet. Check project's page for downloads section.

How To Enable UUID In EMF Generated Model To Get Copy&Paste Working

2007-03-10T16:44:00.000+01:00

If you get very strange results using Copy&Paste in GMF diagram editor then in all likelihood your EMF model doesn’t use Universally Unique Identifiers. EMF’s XMIResourceImpl supports generation of UUID transparently but it’s disabled by default. Read more here.

The Blog Has Been Moved

2007-03-05T16:49:00.000+01:00

All old and new stuff can be found at new place.
Click here

Color Scheme Generator for Charts

2006-11-26T18:23:00.000+01:00

Sometimes one needs to show many data series on one chart. In such cases, usually distinction between those data series is achieved by using different colors for each data set. Most charting libraries leave colors selection to developer. This can be problem when there is need to choose more than six colors which can be easily distinguished by most people. Below you can find Python program which can be helpful.

Program generates palette of colors where each color can be easily distinguished from other colors in its neighborhood. Difference between colors is computed using W3C's formula for color contrast. Minimum contrast (brightness and color), palette size, neighborhood diameter and maximum number of iterations can be easily set in first few lines of program code.

New colors are created using pseudo-random generators and tested for minimum contrast. If contrast test fails, the color is dropped and new one is created. Besides of minimum required contrast, the size of neighborhood has great impact on probability that test will fail. Too big neighborhood diameter could make it impossible to create complete neighborhood and lock the program. To prevent this, there is limit for number of iterations.

#!/usr/bin/python

from random import randint

class Color:
   __c = [0, 0, 0]
   __cB = None
   def __init__(self, r=-1, g=-1, b=-1):
       if r == -1:
           r = randint(1,255)
       if g == -1:
           g = randint(1,255)
       if b == -1:
           b = randint(1,255)
       self.__c = [r, g, b]
   def __getitem__(self, key):
       return self.__c[key]
   def __setitem__(self, key, value):
       self.__c[key] = value
       self.__cB = None
   def getBritness(self):
       if self.__cB:
           return self.__cB
       else:
           self.__cB = ((self.__c[0] * 299) +
                       (self.__c[1] * 587) +
                       (self.__c[2] * 114)) / 1000
           return self.__cB
   def __sub__(self, other):
       return ((max(self.__c[0], other[0]) - min(self.__c[0], other[0])) +
               (max(self[1], other[1]) - min(self[1], other[1])) +
               (max(self[2], other[2]) - min(self[2], other[2])))
   def __str__(self):
       return "%02x%02x%02x" % (self.__c[0], self.__c[1], self.__c[2])

colors = []
num = 25
comparedNum = 5

c = Color()
colors.append(c)
stopper = 100000

minBrit = 35
minColDiff = 100

print "searching...",
for n in range(num)[1:]:
   while stopper:
       newC = Color()
       passed = 0
       for back in range(comparedNum):
           testedNum = n-(back+1)
           if testedNum < 0:
               passed += 1
               continue
           c = colors[testedNum]
           bDiff = abs(c.getBritness() - newC.getBritness())
           cDiff = c - newC
           if  cDiff >= minColDiff and  bDiff > minBrit:
               passed += 1
           else:
               break
       if passed == comparedNum:
           colors.append(newC)
           print "found:", newC,
           break

       stopper -= 1

if not stopper > 0:
   print "searching failed"
else:
   print "finished"


html = """
< head>

< /head>
< body>
""" % (800/n)

python = "colors = ("

for c in colors:
   html += '\n' % c
   python += '"%s", ' % c

python = python[:-2] + ")"
html += "< /body>"

out = file("colors.html", "w")

print >>out, html
print python

out.close()

Synchronizing EMF and GMF Generated Editors

2006-11-21T12:42:00.000+01:00

Frequently one does not want graphical representation for objects of all classes of semantic model in a graphical diagram editor, but graphical objects usually have references to non-graphical and there is still need for creating and editing all of them. In that case it would be nice to reuse EMF generated editor for non-graphical elements as a helper for diagram editor, so there would be no need for creation of custom edit forms. Full article is here.

Regular Expression To Match IPv4 Addresses

2006-11-16T17:20:00.000+01:00

I needed a regex pattern to match IPv4 addresses today. I thought that this is quite common thing and it will be easy to find one on the Net. I have found a lot of them indeed. Unfortunately no one was good enough. Most of them was too simple and could match a string which is not IPv4 address. On the other hand, those more specific didn't match some fully legal IPv4 addresses (for example the one from regexlib.com). So I have developed my own:

((2[0-5]{2}|2[0-4]\d|1\d{2}|[1-9]\d|\d)\.){3}(2[0-5]{2}|2[0-4]\d|1\d{2}|[1-9]\d|\d)
(/(3[012]|[12]\d|\d))?

This pattern matches all (I hope) IPv4 addresses in dot-decimal notation (first line) and network blocks in CIDR notation (when you join second line).

Visualization of Progress in MUDding using Python and RRDTool

2006-11-02T18:12:00.000+01:00

I like skill-based, level-less MUDs, so this will be about one of such MUDs, but provided here scripts probably can be adopted to any MUD system or even to any log file. Skill-based MUDs send special message to player when he archives progress in character development. Such message can look like this:

 "You learned a trick of casting snake rune"

Let's try to write some code which can visualise progress of character development on time axis. As input data we will use log of all mud sessions. Saving mud session to log-file can be configured in most mud clients. First we have to extract progression marks and their essence. This can be easily done using following *nix commands:

grep 'You learned a trick of ' atp_log_0 | \
sort | \
sed -e 's/You learned a trick of casting \([a-zA-Z ]\+\).*/\1/'

As a result we will get something like this:

...
snake rune
stoneskin
stoneskin
stoneskin
summon
summon
...

Now the harder part, we have to count occurrences of the same strings and put them to some database. To see changes in time we have to do this regularly, for example every hour. In this case rrdtool would be very good as database engine. Following Python script will do everything what we need:

#!/usr/bin/python

import os
import rrdtool
from sys import argv
from sys import exit

if len(argv) != 2:
    print "USAGE: %s prepared_log" % argv[0]
    exit(1)

lines = open(argv[1], "r").readlines()

counter = {}

spell = ""
for line in lines:
    line = line[:-1]
    if spell == line:
        counter[line] += 1
    else:
        counter[line] = 1
    spell = line

for spell in counter.keys():
    print spell + ": " + str(counter[spell])
    rrdFileName = spell + ".rrd"
    if not os.path.exists(rrdFileName):
        # create rrd file
        rrdtool.create(rrdFileName, '-s', '3600',
            'DS:data:COUNTER:86400:0:50000',
            'RRA:AVERAGE:0.5:1:5000')
        print "created: " + rrdFileName
    rrdtool.update(rrdFileName, 'N:' + str(counter[spell]))

Now we have to play MUD for few hours and collect data. Finally we can attempt to perform visualisation. Manual generation of image[s] with charts from all .rrd files can be tiring, so we will use following Python script:

#!/usr/bin/python

import os

RRD_EXT = ".rrd"

colors = ("B81818","5CDE00","0089DE",
    "1A1F14","DEC400","486B20","1216A1",
    "432A96","913F64","9A3C17","F4A179","4B7B47",
    "7E6C67","51D9F4","559E7B","4B4D35","D8980E","A32245")

defs = ""
lines = ""
num = 0;
numc = 0;
files = os.listdir(".")
for file in files:
    base,ext = os.path.splitext(file)
    if ext == RRD_EXT:
        color = colors[numc]
        defs += 'DEF:d%d="%s":data:AVERAGE ' % (num, file)
        lines += 'AREA:d%d#%s:"%s":STACK ' % (num, color, base)
        num += 1
        if len(colors) == numc:
            numc = 0;
        else:
            numc += 1

command = 'rrdtool graph test.png --end now --start end-58h ' + \
'--height 400 --width 600 ' + defs + ' ' + lines

k = os.system(command)
if k == 0:
    os.system("eog test.png")
else:
    print "error code: ", k

"eog" is a standard Gnome image viewer. Of course you can put here your favourite one.

Cisco/Catalyst Vlans on Ports - final solution

2006-10-31T20:45:00.000+01:00

With great help from Cisco NMS EMEA TAC, I have finally found solution to map tagged and untagged vlans to physical ports on the Cisco Catalyst switches, without using SNMP Community String Indexing (CSI). With CSI one can select entity/instance of multi-instanced MIBs, but it is quite awkward and not well supported on most NMSs because you have to use separate community strings for every instance. On Catalysts, BRIDGE-MIB is a multi-instanced MIB, instances of which exist separately for every VLAN. Its dot1dBaseTable is the only way to map bridge port numbers to interfaces from ifTable. Some SNMP tables use bridge port numbering to point physical ports, so to bind data from that tables to something more tangible than bridge port number (like interface from IF-MIB or physical port from ENTITY-MIB) one have to scan dot1dBaseTable 3000 times , in case of having 3000 vlans!

I came up against BRIDGE-MIB and CIS in Vlan->Port mapping case because of the port numbering used in vmMembershipSummaryTable (CISCO-VLAN-MEMBERSHIP-MIB). I have been using that table to get access ports (ports with only one untagged vlan) for given vlan. CISCO-VLAN-MEMBERSHIP-MIB has another table - vmMembershipTable - which looked promising, but due to some kind of blackout of my mind, I thought that vmMembershipTable is write only and I have to use that damn vmMembershipSummaryTable. Now I know that vmMembershipTable can be read, is indexed in the the same manner as interfaces in ifTable and has one very useful column: vmVlan.

So for tagged vlans use vlanTrunkPortTable and for untagged - vmMembershipTable. Booth contain links to interfaces in ifTable, which can be easily linked to physical ports in ENTITY-MIB, then.

Beryl on Xgl

2006-10-20T23:48:00.000+02:00

Finally I have Beryl running on my desktop at home. If you want to better feel how it works (this is very dynamic gadget) then check this video. I don't like this wavy animation of window [un]minimization, so I've changed it to something else. Beryl setting manager allows you to change almost everything without pain in the ass. Usually all changes take affect immediately and you don't have to restart window manager, but in random cases it doesn't work.

JGrups on Linux With Multiple Network Interfaces

2006-10-20T17:43:00.000+02:00

Few hints for those, who want to run JBoss Cache in clustered mode ([a]synchronous replication/invalidation) on Linux machine with multiple network interface cards (NICs) and using UDP/multicasting in OSI layers 4/3 . Here is situation overview: We need multicast communication through eth1 to other JBoss Cache nodes.
JBoss Cache uses JGroups for network communication. And I'm going to focus on configuration of this service (Cache configuration is very easy). First of all, for device with multiple NICs, you need to specify a bind address in configuration file (of course if use xml file to configure services) in section ClusterConfig/UDP. AFAIK this can't be 0.0.0.0. Probably you can use any address from your NICs, but I would suggest using the one from eth1 (this will be logical but it won't change process of packet generation and sending). Without this parameter service won't start throwing an exception.

Second thing is to specify outgoing NIC. On Linux platform process run by non-root user can't specify source address and outgoing interface for it's packets. The only way to control this is through routing table. Without special entry in routing table for your cache's multicast address, all packets will go outside of machine using DEFAULT route, which in case from above picture will mean eth0. This is not what we want to archive. You can correct it like this:

$ ip r a 228.1.2.3/24 dev eth0

The last thing is packet fragmentation issue. Maybe only I had this problem, but I spend a lot of time trying to solve it. Every replication of bigger amount of data resulted in timeout exception. To get things working I had to set timeout to 120 seconds! Of course that wasn't right solution. Finally I've changed max_xmit_size and frag_size parameters to MTU minus approximate size of all headers. This is about 1400 B.

If you still have problems then check your Ethernet switch connected to eth1. For example if it has IGMP SPAN mode enabled then check if it works correctly. I had problems with it on 3Com SuperStack 3 Switch 3870 with software version 1.01.

Cisco MIBs horror

2006-10-17T16:04:00.000+02:00

Suppose you want to know what vlans are configured/allowed on individual ports in one of recent Cisco Catalyst switches - quite basic information. SNMP seems to be right protocol for this task. Structure of most Catalyst switches can be discovered using ENTITY-MIB and that's good news. In entPhysicalTable you can find ports, modules, chassis and many other parts with defined relationships between them. Now you want to find vlans for discovered ports and this is where scary part starts. Since port can have many vlans and vice versa, such mapping can be done in SNMP using one column table with row index build by combination of two numbers (portID.vlanID). As value of this row could be:

1 meaning native vlan
2 meaning tagged vlan

Lack of port-vlan combination in table would mean that this port isn't present in given vlan. Simple? Yes, but too simple for Cisco engineers.

Cisco found another solution. In order to get via SNMP vlans per port information for Cat2950, Cat2970, Cat4500 and Cat6500 (with recent IOSes) you have to correctly combine following tables:

ifTable from IF-MIB entPhysicalTable from ENTITY-MIB, entAliasMappingTable from ENTITY-MIB, vtpVlanTable from CISCO-VTP-MIB, vlanTrunkPortTable from CISCO-VTP-MIB, vmMembershipSummaryTable from CISCO-VLAN-MEMBERSHIP-MIB, dot1dBasePortTable from BRIDGE-MIB

Some of those tables are 3D, have thousands of columns (one bit per column) and all of them use different, non-continuous indexes. But that would be still to easy. To get mapping from ports in vmMembershipSummaryTable to ports in ifTable you have to query dot1dBasePortTable using separate SNMP community for every vlan. Thank you Cisco!

Does anyone have any other idea how to solve this problem?

E-mail Posting Test

2006-10-15T16:43:00.001+02:00

Currently I try to combine Eclipse Platform (OSGi, EMF, GMF) with Spring Framework (as Beans container and factory) and JBoss Cache (synchronous distributed cache) and many other useful pieces of software (like SNMP4J) in order to build network management application.

All this software is very good. The problem is that Eclipse and JBoss suffer lack of documentation for newly developed stuff while I'm mostly interested in newly developed stuff.

Some measures for today: Execution time analysis for cached/direct access to following SNMP tables: VlanTrunkPortTable(from Cisco MIBS) and IfTable combined with InterfacesTable:

direct run time: 1308
cached run time: 2684 (empty cache)
cached run time: 6 (full cache)

So building and propagating cache takes: 1376 milliseconds. That's not good.

About This Blog

2006-10-15T15:29:00.000+02:00

As always my first choice wasn't good. I had to immediately change blog template to completely different. The old one could break your eyes. I like new things, so I've converted my blog to http://beta.blogger.com. It's one way process, but I have nothing to loose. Let's see how this all works. In the mean time I'll worm up my tea in microwave. This blog will be mostly about me versus resistance of some cutting edge software development technologies (usually open-source Java and Python frameworks, libraries and tools).

Let's start

2006-10-15T15:27:00.000+02:00

World comes to the an end. I have a blog.