eBGP lab with FRR routers in Docker (3 ASes)
Build an eBGP lab with three FRR routers in Docker, one per AS. Bring up sessions, exchange prefixes, read the AS path and filter a route.
You will build three autonomous systems on your laptop, each with one FRRouting (FRR) router in Docker, and connect them with eBGP. You will see sessions come up, prefixes travel across an AS in the middle, and a route disappear when you filter it. All the output below comes from a real run of these commands.
What you'll build
AS 65001 AS 65002 AS 65003
+--------+ 10.0.12.0/24 +--------+ 10.0.23.0/24 +--------+
| r1 |.1 ---------- .2| r2 |.2 ---------- .3| r3 |
+--------+ +--------+ +--------+
lo 172.16.1.1/24 lo 172.16.2.1/24 lo 172.16.3.1/24
Each router announces the /24 on its loopback. r2 sits in the middle, so r1 and r3 can only reach each other through AS 65002. That makes r2 a transit AS, like an ISP between two customers.
How eBGP differs from OSPF
OSPF finds neighbors on its own and computes the shortest path inside one network. BGP connects networks run by different people, so it works differently:
- Neighbors are configured by hand. Each
neighborline names a peer address and its AS. The routers open a TCP session on port 179. - Routes carry attributes, not a cost. The main one is the AS path, the list of ASes a route has crossed. Shorter paths win when nothing else decides.
- Policy comes first. Each AS decides what it accepts and what it announces. That is why route-maps and prefix-lists appear in almost every BGP config.
- eBGP and iBGP. A session between two different ASes is eBGP. A session inside one AS is iBGP and follows other rules. This lab is all eBGP.
What you need
- Docker on Linux, or Docker Desktop on macOS or Windows. Tested on Docker 29.
- The FRR image from the project's registry:
quay.io/frrouting/frr:10.7.1.
If you have not built the OSPF lab with FRR in Docker yet, it explains the same container setup in more detail.
Step 1: Create the link networks
Each link between two routers is its own Docker network with its own subnet:
docker network create --subnet 10.0.12.0/24 --gateway 10.0.12.254 bgp-r1r2
docker network create --subnet 10.0.23.0/24 --gateway 10.0.23.254 bgp-r2r3
By default Docker gives .1 to the host bridge. Moving the gateway to .254 frees .1 for r1. Without --gateway, starting r1 fails with Address already in use.
Step 2: Write the FRR configs
Copy the image's daemons file and switch bgpd on for each router:
mkdir -p r1 r2 r3
for n in 1 2 3; do
docker run --rm quay.io/frrouting/frr:10.7.1 cat /etc/frr/daemons \
| sed 's/^bgpd=no/bgpd=yes/' > r$n/daemons
touch r$n/vtysh.conf
done
r1/frr.conf:
frr defaults traditional
hostname r1
!
interface lo
ip address 172.16.1.1/24
!
router bgp 65001
bgp router-id 1.1.1.1
neighbor 10.0.12.2 remote-as 65002
!
address-family ipv4 unicast
network 172.16.1.0/24
neighbor 10.0.12.2 route-map ALLOW-ALL in
neighbor 10.0.12.2 route-map ALLOW-ALL out
exit-address-family
!
route-map ALLOW-ALL permit 10
!
r2/frr.conf has two neighbors:
frr defaults traditional
hostname r2
!
interface lo
ip address 172.16.2.1/24
!
router bgp 65002
bgp router-id 2.2.2.2
neighbor 10.0.12.1 remote-as 65001
neighbor 10.0.23.3 remote-as 65003
!
address-family ipv4 unicast
network 172.16.2.0/24
neighbor 10.0.12.1 route-map ALLOW-ALL in
neighbor 10.0.12.1 route-map ALLOW-ALL out
neighbor 10.0.23.3 route-map ALLOW-ALL in
neighbor 10.0.23.3 route-map ALLOW-ALL out
exit-address-family
!
route-map ALLOW-ALL permit 10
!
r3/frr.conf:
frr defaults traditional
hostname r3
!
interface lo
ip address 172.16.3.1/24
!
router bgp 65003
bgp router-id 3.3.3.3
neighbor 10.0.23.2 remote-as 65002
!
address-family ipv4 unicast
network 172.16.3.0/24
neighbor 10.0.23.2 route-map ALLOW-ALL in
neighbor 10.0.23.2 route-map ALLOW-ALL out
exit-address-family
!
route-map ALLOW-ALL permit 10
!
A few things to note:
remote-asdiffers from the local AS, so each session is eBGP.network 172.16.1.0/24announces a prefix only if it is already in the routing table. The loopback address puts it there.ALLOW-ALLis a route-map with onepermitentry and nomatch, so it accepts everything. It looks pointless. It isn't, as the next section shows.
Why the route-maps?
FRR follows RFC 8212: by default, an eBGP session with no policy accepts and sends nothing. The FRR BGP documentation says bgp ebgp-requires-policy is enabled by default with the traditional profile. Here is r2 from a first run without the route-maps:
Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd PfxSnt Desc
10.0.12.1 4 65001 3 3 1 0 0 00:00:14 (Policy) (Policy) N/A
10.0.23.3 4 65003 3 4 1 0 0 00:00:14 (Policy) (Policy) N/A
The sessions are up, but (Policy) replaces the prefix counts. This is the most common reason a new FRR eBGP lab "works" and still learns no routes. In a lab you could also turn the check off with no bgp ebgp-requires-policy under router bgp, but a route-map is the habit worth building.
Step 3: Start the routers
IMG=quay.io/frrouting/frr:10.7.1
docker create --name bgp-r1 --hostname r1 --privileged --sysctl net.ipv4.ip_forward=1 \
--network bgp-r1r2 --ip 10.0.12.1 -v "$PWD/r1:/etc/frr" $IMG
docker create --name bgp-r2 --hostname r2 --privileged --sysctl net.ipv4.ip_forward=1 \
--network bgp-r1r2 --ip 10.0.12.2 -v "$PWD/r2:/etc/frr" $IMG
docker network connect --ip 10.0.23.2 bgp-r2r3 bgp-r2
docker create --name bgp-r3 --hostname r3 --privileged --sysctl net.ipv4.ip_forward=1 \
--network bgp-r2r3 --ip 10.0.23.3 -v "$PWD/r3:/etc/frr" $IMG
docker start bgp-r1 bgp-r2 bgp-r3
r2 needs IP forwarding to carry traffic between r1 and r3. Unlike OSPF, eBGP has no 40-second wait. Sessions come up within a few seconds.
Step 4: Check sessions and prefixes
Start in the middle:
docker exec bgp-r2 vtysh -c 'show bgp summary'
IPv4 Unicast Summary:
BGP router identifier 2.2.2.2, local AS number 65002 VRF default vrf-id 0
BGP table version 3
RIB entries 3, using 480 bytes of memory
Peers 2, using 46 KiB of memory
Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd PfxSnt Desc
10.0.12.1 4 65001 14 12 3 0 0 00:00:10 1 3 N/A
10.0.23.3 4 65003 18 13 3 0 0 00:00:10 1 3 N/A
A number under State/PfxRcd means the session is Established. r2 receives one prefix from each neighbor and sends three to each.
Now the BGP table on r1:
docker exec bgp-r1 vtysh -c 'show bgp ipv4 unicast'
Network Next Hop Metric LocPrf Weight Path
*> 172.16.1.0/24 0.0.0.0 0 32768 i
*> 172.16.2.0/24 10.0.12.2 0 0 65002 i
*> 172.16.3.0/24 10.0.12.2 0 65002 65003 i
Read the Path column from left to right: the nearest AS first, the origin AS last. r3's prefix crossed AS 65002 before it reached r1. The i at the end is the origin code (IGP, from a network statement). r1's own prefix shows next hop 0.0.0.0 and weight 32768, FRR's marks for a locally originated route.
To see only what one neighbor sent, use:
docker exec bgp-r1 vtysh -c 'show bgp ipv4 unicast neighbors 10.0.12.2 routes'
Network Next Hop Metric LocPrf Weight Path
*> 172.16.2.0/24 10.0.12.2 0 0 65002 i
*> 172.16.3.0/24 10.0.12.2 0 65002 65003 i
Only two routes, yet r2's summary said it sent three. The third is r1's own 172.16.1.0/24, sent back with the path 65002 65001. r1 sees its own AS in the path and drops it. That is BGP's loop prevention, and it needs no config.
The best routes go into the kernel. show ip route bgp on r1:
B>* 172.16.2.0/24 [20/0] via 10.0.12.2, eth0, weight 1, 00:00:10
B>* 172.16.3.0/24 [20/0] via 10.0.12.2, eth0, weight 1, 00:00:10
20 is the administrative distance of eBGP. Finally, test the data plane between the two edge ASes. Use r1's loopback as the source, because r3 has no route back to the 10.0.12.0/24 link:
docker exec bgp-r1 ping -c 3 -I 172.16.1.1 172.16.3.1
PING 172.16.3.1 (172.16.3.1) from 172.16.1.1: 56 data bytes
64 bytes from 172.16.3.1: seq=0 ttl=63 time=0.119 ms
64 bytes from 172.16.3.1: seq=1 ttl=63 time=0.072 ms
64 bytes from 172.16.3.1: seq=2 ttl=63 time=0.081 ms
Step 5: Filter a prefix
Now make r1 refuse AS 65003's prefix. Open a shell on r1:
docker exec -it bgp-r1 vtysh
Then enter:
configure terminal
ip prefix-list FROM-R2 seq 5 deny 172.16.3.0/24
ip prefix-list FROM-R2 seq 10 permit 0.0.0.0/0 le 32
route-map FROM-R2 permit 10
match ip address prefix-list FROM-R2
exit
router bgp 65001
address-family ipv4 unicast
neighbor 10.0.12.2 route-map FROM-R2 in
end
clear bgp ipv4 unicast 10.0.12.2 soft in
The new inbound route-map replaces ALLOW-ALL for that neighbor. The soft clear re-applies the policy without dropping the session. Check the table again:
Network Next Hop Metric LocPrf Weight Path
*> 172.16.1.0/24 0.0.0.0 0 32768 i
*> 172.16.2.0/24 10.0.12.2 0 0 65002 i
172.16.3.0/24 is gone, and show bgp summary on r1 now shows 1 under State/PfxRcd. Type exit to leave vtysh. These changes live only in the running config. Add the same lines to r1/frr.conf to keep them after a restart.
If something doesn't work
- State
ActiveorConnectinstead of a number. The TCP session isn't up. Check the neighbor address andremote-ason both sides, and ping the peer's link address. (Policy)in the summary. A route-map is missing on that neighbor, in one direction or both.- A prefix you announce is missing. The
networkline must match a route in the routing table exactly, mask included. Address already in useat start. The network was created without--gateway, so Docker holds.1.
Clean up
docker rm -f bgp-r1 bgp-r2 bgp-r3
docker network rm bgp-r1r2 bgp-r2r3
Going further
- Prefer a path. Add a direct link between r1 and r3, then use
set local-preferenceor AS-path prepending in a route-map to choose which way traffic leaves. - Stop being transit. Give r1 and r3 a second upstream and an outbound prefix-list that only allows their own /24. That is what every multihomed customer should do.
- Add iBGP. Put two routers in AS 65002, run OSPF between them, and learn why
next-hop-selfexists. - Add BFD. Enable
bfddin the daemons file and watch failover get faster when youdocker stopa router.
The protocol itself is RFC 4271. For more lab tools than raw Docker, see our comparison of GNS3 alternatives.
Draw it instead
In Isothrum's editor you can draw these three routers in the browser, with no account, and pick FRR for each one. It exports an isoloom.yml plus Docker Compose files that you run with Docker, as here. Each cable is its own small network, which matches eBGP links well. You still write the BGP config, which is the point of the network lab.