Re: BGP over satellite link - update!!

[Brian Candler <B.Candler at pobox.com>]

On Mon, Apr 28, 2003 at 04:17:08PM +0300, Mark Tinka wrote:
> Some of this new technology includes "the not-so-common" SCPC modulation
> hardware/satellite modems. Newer equipment now connects directly to one's
> ethernet segment. This equipment/box could run proprietory or even
> open/closed source operating systems. This box is located at both ends of
> the satellite link, naturally.
> 
> There are [Cisco] routers behind both these boxes, to handle the critical
> routing requirements. Even though BGP is open-standards, it won't run on
> either of these boxes. 
> 
> However, these boxes are running the basic routing services, so they know
> how to reach your ethernet segment and the remote box on the other end. More
> advanced routing is handled by the routers behind them. 
> 
> As a client, I would declare, in my router, that box as my default gatway to
> the remote provider over the satellite. Once it gets the packets, it knows
> how to send them over the satellite connection to the remote provider, and
> then onto the Internet. 
> 
> Now, since we know we aren't running BGP on these boxes, yet we need to peer
> with the remote provider, we shall need to use "ebgp-multihop", because the
> remote BGP speaker is not directly connected to the satellite medium in the
> way you would have a router connected to an ordinary satellite modem. So,
> the remote BGP speaker is 3 hops away; I would tell this to my
> "ebgp-multihop" statement.

This assumes that the black-boxes at both ends of the satellite link are IP
routers (layer 3), which decrement the TTL. If they were ethernet bridges
(layer 2) then this would not be an issue.

> However, Phillip said something interesting; that if there's a link failure
> on the primary circuit, and we had a backup, that perhaps our router would
> try to re-initialise the TCP connection to the remote BGP speaker over the
> backup circuit, and the path taken by the traffic would be far from what is
> expected. However, being that the "ebgp-multihop" statement categorically
> states that that remote peer is 3 hops away, wouldn't the formation of a
> neighborship fail, since the remote BGP speaker is more than 3 hops away,
> although reachable?

Yes, that should set the TTL to 3 on the IP packets, so they should go three
hops over the alternative route and then be discarded.

> What I am trying to say is; newer technology is becoming more manageable and
> easier to deploy. This may, at times, take away what we/other networks have
> been used to. In this case, wouldn't "ebgp-multihop" have a solid case, if
> used right?

It seems to me bad design that the 'black boxes' are IP routers without
dynamic forwarding tables. Consider:

                             sat
                            .   .
       Your network        .     .        Provider network
      --- R1 ----------- B1       B2 ------- R2 ------------> Internet
                                              |
                                             CX

If B1 and B2 are IP routers which don't speak BGP, then they must be
configured with static routes to your network, and to any of your customers'
networks behind it (i.e. anyone else to whom you offer transit). They
presumably will have default routes for traffic in the other direction,
B1->B2 and B2->R2.

In the event that the satellite link is up, but your network is down, those
static routes will still be there. So you are reliant on R2 withdrawing its
route via B2 if the eBGP session goes down.

Suppose R2 were configured with a static route to your network via B2 (which
would be the simplest case). If the satellite link went down, traffic from
another customer CX (which is also connected to R2) might also try to go
CX->R2->B2 and fail.

I *think* this can be made to work. R2 picks up a route to your network via
multihop eBGP with a next-hop of R1i (R1's IP address). So if R2 is
configured with a /32 floating static route to R1i via B2, I think that it
will work:

       ! Config for router R2

       ! R1 loopback = 1.2.3.4
       ! B2 ethernet = 6.7.8.9

       ip route 1.2.3.4 255.255.255.255 6.7.8.9

       ! route 1.2.3.0 255.255.255.0 1.2.3.4 is learned via eBGP multihop
       ! There is no directly-connected 1.2.3.4 next-hop, but a recursive
       ! route lookup for 1.2.3.4 finds 6.7.8.9

As you can see this is messy, it relies on recursive next-hop lookups,
and static routes being correctly configured on B1 and B2.

Nobody said that eBGP multihop cannot be made to work. We said that it's a
last-resort solution, and that you are likely to be bitten by the
consequences. For instance, every time your network changes, and you
advertise a new route via eBGP from R1 to R2 but your provider forgets to
reconfigure B1 or B2, you are going to suffer a catastrophic routing loop
which will probably soak up all your available satellite bandwidth.

In the same way, you can build your whole network using static routes if you
like. But if the topology changes, your forwarding tables won't dynamically
correct themselves to the new topology, and you will be in trouble.

Regards,

Brian.

-----
This is the afnog mailing list, managed by Majordomo 1.94.5

To send a message to this list, e-mail afnog at afnog.org
To send a request to majordomo, e-mail majordomo at afnog.org and put
your request in the body of the message (i.e use "help" for help)

This list is maintained by owner-afnog at afnog.org