Testing jgroups communication (multicasting)

Basically we're following this:

http://www.jgroups.org/manual/html/ch02.html#ItDoesntWork

The main tools are included in the "all" JBoss server lib dir-- jgroups.jar (ex:{jbossdir}/servers/all/bin/jgroups.jar).

CD into that directory, and try this stuff:

Make sure your machine is set up correctly for IP multicast. There are 2 test programs that can be used to detect this: McastReceiverTest and McastSenderTest. Start McastReceiverTest, e.g.

java -cp ./jgroups.jar org.jgroups.tests.McastReceiverTest -mcast_addr 224.10.10.10 -port 5555
        

Then start McastSenderTest:

java -cp ./jgroups.jar org.jgroups.tests.McastSenderTest -mcast_addr 224.10.10.10 -port 5555
        

If you want to bind to a specific network interface card (NIC), use -bind_addr 192.168.0.2, where 192.168.0.2 is the IP address of the NIC to which you want to bind. Use this parameter in both sender and receiver.

You should be able to type in the McastSenderTest window and see the output in the McastReceiverTest. If not, try to use -ttl 32 in the sender. If this still fails, consult a system administrator to help you setup IP multicast correctly. If you are the system administrator, look for another job :-)

Other means of getting help: there is a public forum on JIRA

for questions. Also consider subscribing to the javagroups-users mailing list to discuss such and other problems.

2.9. The instances still don't find each other !

In this case we have to use a sledgehammer (running only under JDK 1.4. and higher): we can enable the above sender and receiver test to use all available interfaces for sending and receiving. One of them will certainly be the right one... Start the receiver as follows:

java -cp ./jgroups.jar org.jgroups.tests.McastReceiverTest1_4 -mcast_addr 228.8.8.8 -use_all_interfaces
        

The multicast receiver uses the 1.4 functionality to list all available network interfaces and bind to all of them (including the loopback interface). This means that whichever interface a packet comes in on, we will receive it. Now start the sender:

java -cp ./jgroups.jar org.jgroups.tests.McastSenderTest1_4 -mcast_addr 228.8.8.8 -use_all_interfaces
        

The sender will also determine the available network interfaces and send each packet over all interfaces.

This test can be used to find out which network interface to bind to when previously no packets were received. E.g. when you see the following output in the receiver:

java  -cp ./jgroups.jarorg.jgroups.tests.McastReceiverTest1_4 -mcast_addr 228.8.8.8 -bind_addr 192.168.168.4
            Socket=0.0.0.0/0.0.0.0:5555, bind interface=/192.168.168.4
            dd [sender=192.168.168.4:5555]
            dd [sender=192.168.168.1:5555]
            dd [sender=192.168.168.2:5555]
        

you know that you can bind to any of the 192.168.168.{1,2,4} interfaces to receive your multicast packets. In this case you would need to modify your protocol spec to include bind_addr=192.168.168.2 in UDP, e.g. "UDP(mcast_addr=228.8.8.8;bind_addr=192.168.168.2):..."

Setting up multicasting can be a PITA if you don't control your network. Luckily, we do, so it's no big deal, but there are a lot of things to check if it's not working.

I'd start with verifying that multicast forwarding is enabled on your NIC, and in your configs (this varies by OS, but google is quite helpful). Then move onto the rest of your network (firewalls, routers, etc.).

Start with the firewall: On a CentOS 5 box, you'll need to tweak the iptables config (/etc/sysconfig/iptables) to allow UDP traffic on the right interfaces.

Here's an example (put it under the existing udp stuff in sysconfig/iptables, which BTW, should also give you your UDP mcast addr (...-p udp --dport 5555 -d 2xx.0.0.x)):

-A RH-Firewall-1-INPUT -p udp -i eth3 -m udp --dport 4000:6000 -j ACCEPT

The above says "open ports 4000:6000 on the eth3 interface", basically. For my purposes, eth3 is a NIC dedicated to back-end private network traffic only. Having a separate NIC for inter-server communication is really tits.

After you add that line, you'll need to reload iptables:

iptables-restore < /etc/sysconfig/iptables

Depending on how you decided to have your nodes talk to each other, you may need to configure some stuff in jgroups-channelfactory-stacks.xml (jgroupschannel-factory.sar) - see this. I used something along these lines:

<FD timeout="10000" max_tries="5" shun="true"></FD>

Configuring jgroups for jboss AS 5 is easy as pie, here's some info for params and whatnot:

Isolation and JGroups ChannelFactory and Shared Transport in JBoss AS 5 and Buddy Replication and Session Data, although I had better links a while ago. Eh, those should get you going.

Yesh, just a copy/paste entry, mostly, but what the hell.

Comments (Comment Moderation is enabled. Your comment will not appear until approved.)
BlogCFC was created by Raymond Camden. This blog is running version 5.9.3.000. Contact Blog Owner