Yrurtj

This my setup using Debian 6.0. 6. It's role is a proxy server and a load balancer using eight 3G modems.

Whenever a ppp connection dials it takes over the default gateway which is ok but it also cuts me out.The reason it's ok is that in Ubuntu I noticed that it does not do that by default. By default it will leave the ethX gateway untouched. I could have added in the ppp options file replacedefaultroute but problems with iptables in Ubuntu made me switch to Debian.

Now I cannot ping other hosts on the LAN but other hosts can ping me. I also lose connectivity from the outside and the only way I can get access is through another computer from inside the LAN.Also all the hosts in the LAN can use the proxy.

What I found out is that if I add a separate routing table called e1 and copy

192.168.2.0/24 dev eth0  proto kernel  scope link  src 192.168.2.126

default via 192.168.2.3 dev eth0

From the main table into table e1 and then do:

ip rule add table e1

It works, but this is not what I want since all the connections now flow through the eth0 interface.

What can I try to restore the connections comming from outside, and to communicate with the local computers. However all traffic comming from the LAN must still be made through the ppp links.

root@proxy:~# iptables -L OUTPUT -t mangle -v

Chain OUTPUT (policy ACCEPT 433K packets, 217M bytes)

pkts bytes target     prot opt in     out     source               destination

433K  217M CONNMARK   all  --  any    any     anywhere             anywhere            CONNMARK restore

 929 61011 MARK       all  --  any    any     anywhere             anywhere            state NEW statistic mode nth every 8 MARK set 0x1

 929 61721 MARK       all  --  any    any     anywhere             anywhere            state NEW statistic mode nth every 8 packet 1 MARK set 0x2

 929 61461 MARK       all  --  any    any     anywhere             anywhere            state NEW statistic mode nth every 8 packet 2 MARK set 0x3

 929 61438 MARK       all  --  any    any     anywhere             anywhere            state NEW statistic mode nth every 8 packet 3 MARK set 0x4

 929 61530 MARK       all  --  any    any     anywhere             anywhere            state NEW statistic mode nth every 8 packet 4 MARK set 0x5

 929 61022 MARK       all  --  any    any     anywhere             anywhere            state NEW statistic mode nth every 8 packet 5 MARK set 0x6

 929 61738 MARK       all  --  any    any     anywhere             anywhere            state NEW statistic mode nth every 8 packet 6 MARK set 0x7

 928 61224 MARK       all  --  any    any     anywhere             anywhere            state NEW statistic mode nth every 8 packet 7 MARK set 0x8

433K  217M CONNMARK   all  --  any    any     anywhere             anywhere            CONNMARK save





root@proxy:~# iptables -L POSTROUTING -t nat -v

Chain POSTROUTING (policy ACCEPT 285 packets, 18881 bytes)

 pkts bytes target     prot opt in     out     source               destination

  743 49005 MASQUERADE  all  --  any    ppp0    anywhere             anywhere

  705 47291 MASQUERADE  all  --  any    ppp1    anywhere             anywhere

  679 45581 MASQUERADE  all  --  any    ppp2    anywhere             anywhere

  679 45598 MASQUERADE  all  --  any    ppp3    anywhere             anywhere

  670 45177 MASQUERADE  all  --  any    ppp4    anywhere             anywhere

  638 42447 MASQUERADE  all  --  any    ppp5    anywhere             anywhere

  724 48671 MASQUERADE  all  --  any    ppp6    anywhere             anywhere

  679 45182 MASQUERADE  all  --  any    ppp7    anywhere             anywhere

root@proxy:~#



root@proxy:~# ip rule

0:      from all lookup local

32758:  from all fwmark 0x8 lookup d8

32759:  from all fwmark 0x7 lookup d7

32760:  from all fwmark 0x6 lookup d6

32761:  from all fwmark 0x5 lookup d5

32762:  from all fwmark 0x4 lookup d4

32763:  from all fwmark 0x3 lookup d3

32764:  from all fwmark 0x2 lookup d2

32765:  from all fwmark 0x1 lookup d1

32766:  from all lookup main

32767:  from all lookup default



root@proxy:~# ip ro sh t d1

default via 10.64.64.64 dev ppp0

root@proxy:~# ip ro sh t d2

default via 10.64.64.65 dev ppp1

root@proxy:~# ip ro sh t d3

default via 10.64.64.66 dev ppp2

root@proxy:~# ip ro sh t d4

default via 10.64.64.67 dev ppp3

root@proxy:~# ip ro sh t d5

default via 10.64.64.68 dev ppp4

root@proxy:~# ip ro sh t d6

default via 10.64.64.69 dev ppp5

root@proxy:~# ip ro sh t d7

default via 10.64.64.70 dev ppp6

root@proxy:~# ip ro sh t d8

default via 10.64.64.71 dev ppp7





root@proxy:~# ip ro

10.64.64.67 dev ppp3  proto kernel  scope link  src 10.90.33.221

10.64.64.66 dev ppp2  proto kernel  scope link  src 10.18.11.90

10.64.64.65 dev ppp1  proto kernel  scope link  src 10.90.14.235

10.64.64.64 dev ppp0  proto kernel  scope link  src 10.18.27.226

10.64.64.71 dev ppp7  proto kernel  scope link  src 172.22.201.81

10.64.64.70 dev ppp6  proto kernel  scope link  src 10.80.131.6

10.64.64.69 dev ppp5  proto kernel  scope link  src 172.20.17.183

10.64.64.68 dev ppp4  proto kernel  scope link  src 10.80.61.34

192.168.2.0/24 dev eth0  proto kernel  scope link  src 192.168.2.126

default via 192.168.2.3 dev eth0

root@proxy:~#

If you can give me some ideas I would appreciate it.

.

1. Access from outside was good when I forwarded the damn SSH port on the router.


2. 
   

   Communication with other computers in the 192.168.2.0/24 netowrk is ok only after issuing this command in iptables
   In this instances I only wanted .253 (Local DNS) to talk to me (.126)

   
   
   

   iptables -A OUTPUT -t mangle -s 192.168.2.126 -d 192.168.2.253 -j MARK --set-mark 16

   
   

In the iptables list I have rules that basically state, each new connection should be tagged and, perform the routing decision and then go thorugh POSTROUTING chain.
Since now I'm tagging packets with specific criteria (tag mark 16) the ip rules will not match the fwmarks so the only route is found in ip rule's default main table

192.168.2.0/24 dev eth0  proto kernel  scope link  src 192.168.2.126  metric 1 

In OUTPUT, you are marking every new packet, even whose destination address is in your LAN.

Your ppp routing tables only have a default entry. That mean every packet ever going to one of these tables will always get routed via your gateway on ppp*. Even if that packet's destination address is in your LAN. That's because the rules are evaluated in increasing priority order, and your d[0-8] tables are tried before main.

As such, pings to your lan get routed to your ppp devices, and get NATed... So if they work, it's because the selected ppp gateway has another host with the same IP address in his routing tables.

I would put every LAN rule in a separate table which is tried before the d[0-8] tables, so that traffic that should go to your lan, goes on your lan.