Carrier-of-Carriers with VPN Service Provider as the Customer on a vMX

Implementation description

In this article, I will build a Carrier-of-Carriers with VPN Service Provider as the customer setup on a single vMX. Each individual router will be configured as a logical router. The vMX router has a back-to-back connection between ge-0/0/0 and ge-0/0/1 ports. For each different link, a different VLAN number will be configured to separate the traffic generated based on router ID number.

Network Diagram

IP Allocation

  • AS 1: 192.168.0.0/16
    • CE 1: 192.168.0.1/32
    • CE 2: 192.168.0.2/32
    • Link 111: 192.168.1.0/31
    • Link 216: 192.168.1.2/31
  • AS 100: 10.10.0.0/16
    • PE 11: 10.10.0.11/32
    • P 12: 10.10.0.12/32
    • C-CE 13: 10.10.0.13/32
    • C-CE 14: 10.10.0.14/32
    • P 15: 10.10.0.15/32
    • PE 16: 10.10.0.16/32
    • Internal Links: 10.10.1.0/16
  • AS 200: 172.16.0.0/16
    • C-PE 21: 172.16.0.21/32
    • C-P 22: 172.16.0.22/32
    • C-PE 23: 172.16.0.23/32
    • Link 1321: 172.16.255.0/31
    • Link 1423: 172.16.255.2/31

Full Configuration

root@MX# show | no-more 
## Last changed: 2017-12-27 03:07:53 UTC
version 15.1F4.15;
groups {
    isis {
        logical-systems {
            <*> {
                protocols {
                    isis {
                        level 1 disable;
                        interface <*> {
                            point-to-point;
                        }
                    }
                }
            }
        }
    }
}
apply-groups isis;
system {
    host-name MX;
    root-authentication {
        encrypted-password "$5$L3F31155$kVyagZl2v/WM9s32/hi7VCXxM5o0vupYD.LO3uvCif4"; ## SECRET-DATA
    }
    services {
        ssh;
        netconf {
            ssh;
        }
    }
    syslog {
        user * {
            any emergency;
        }
        file messages {
            any notice;
            authorization info;
        }
        file interactive-commands {
            interactive-commands any;
        }
    }
}
logical-systems {
    1-CE {
        interfaces {
            ge-0/0/0 {
                unit 111 {
                    vlan-id 111;
                    family inet {
                        address 192.168.1.0/31;
                    }
                }
            }
            lo0 {
                unit 1 {
                    family inet {
                        address 192.168.0.1/32;
                    }
                }
            }
        }
        protocols {
            bgp {
                group AS100 {
                    type external;
                    export to-bgp;
                    peer-as 100;
                    neighbor 192.168.1.1;
                }
            }
        }
        policy-options {
            policy-statement to-bgp {
                from protocol direct;
                then accept;
            }
        }
        routing-options {
            autonomous-system 1;
        }
    }
    11-PE {
        interfaces {
            ge-0/0/0 {
                unit 1112 {
                    vlan-id 1112;
                    family inet {
                        address 10.10.1.0/31;
                    }
                    family iso;
                    family mpls;
                }
            }
            ge-0/0/1 {
                unit 111 {
                    vlan-id 111;
                    family inet {
                        address 192.168.1.1/31;
                    }
                }
            }
            lo0 {
                unit 11 {
                    family inet {
                        address 10.10.0.11/32;
                    }
                    family iso {
                        address 49.0100.0100.1000.0011.00;
                    }
                }
            }
        }
        protocols {
            mpls {
                interface all;
            }
            bgp {
                group internal {
                    type internal;
                    local-address 10.10.0.11;
                    neighbor 10.10.0.13 {
                        family inet {
                            labeled-unicast {
                                resolve-vpn;
                            }
                        }
                    }
                    neighbor 10.10.0.16 {
                        family inet-vpn {
                            any;
                        }
                    }
                }
            }
            isis {
                interface ge-0/0/0.1112;
                interface lo0.11;
            }
            ldp {
                interface ge-0/0/0.1112;
                interface lo0.11;
            }
        }
        routing-instances {
            VPN {
                instance-type vrf;
                interface ge-0/0/1.111;
                route-distinguisher 10.10.0.11:1;
                vrf-target target:100:1;
                vrf-table-label;
                protocols {
                    bgp {
                        group vpn {
                            peer-as 1;
                            as-override;
                            neighbor 192.168.1.0;
                        }
                    }
                }
            }
        }
        routing-options {
            autonomous-system 100;
        }
    }
    12-P {
        interfaces {
            ge-0/0/0 {
                unit 1213 {
                    vlan-id 1213;
                    family inet {
                        address 10.10.1.2/31;
                    }
                    family iso;
                    family mpls;
                }
            }
            ge-0/0/1 {
                unit 1112 {
                    vlan-id 1112;
                    family inet {
                        address 10.10.1.1/31;
                    }
                    family iso;
                    family mpls;
                }
            }
            lo0 {
                unit 12 {
                    family inet {
                        address 10.10.0.12/32;
                    }
                    family iso {
                        address 49.0100.0100.1000.0012.00;
                    }
                }
            }
        }
        protocols {
            mpls {
                traffic-engineering {
                    bgp-igp;
                }
                interface all;
            }
            isis {
                interface all;
            }
            ldp {
                interface ge-0/0/0.1213;
                interface ge-0/0/1.1112;
                interface lo0.12;
            }
        }
    }
    13-C-CE {
        interfaces {
            ge-0/0/0 {
                unit 1321 {
                    vlan-id 1321;
                    family inet {
                        address 172.16.255.0/31;
                    }
                    family mpls;
                }
            }
            ge-0/0/1 {
                unit 1213 {
                    vlan-id 1213;
                    family inet {
                        address 10.10.1.3/31;
                    }
                    family iso;
                    family mpls;
                }
            }
            lo0 {
                unit 13 {
                    family inet {
                        address 10.10.0.13/32;
                    }
                    family iso {
                        address 49.0100.0100.1000.0013.00;
                    }
                }
            }
        }
        protocols {
            mpls {
                traffic-engineering {
                    bgp-igp;
                }
                interface all;
            }
            bgp {
                group internal {
                    type internal;
                    local-address 10.10.0.13;
                    neighbor 10.10.0.11 {
                        family inet {
                            labeled-unicast;
                        }
                    }
                }
                group to-isp {
                    type external;
                    export internal;
                    peer-as 200;
                    neighbor 172.16.255.1 {
                        family inet {
                            labeled-unicast;
                        }
                    }
                }
            }
            isis {
                interface all;
            }
            ldp {
                interface ge-0/0/1.1213;
                interface lo0.13;
            }
        }
        policy-options {
            policy-statement internal {
                term internal {
                    from protocol [ isis direct ldp ];
                    then accept;
                }
                term reject {
                    then reject;
                }
            }
        }
        routing-options {
            autonomous-system 100;
        }
    }
    14-C-CE {
        interfaces {
            ge-0/0/0 {
                unit 1415 {
                    vlan-id 1415;
                    family inet {
                        address 10.10.1.4/31;
                    }
                    family iso;
                    family mpls;
                }
                unit 1423 {
                    vlan-id 1423;
                    family inet {
                        address 172.16.255.2/31;
                    }
                    family mpls;
                }
            }
            lo0 {
                unit 14 {
                    family inet {
                        address 10.10.0.14/32;
                    }
                    family iso {
                        address 49.0100.0100.1000.0014.00;
                    }
                }
            }
        }
        protocols {
            mpls {
                traffic-engineering {
                    bgp-igp;
                }
                interface all;
            }
            bgp {
                group internal {
                    type internal;
                    local-address 10.10.0.14;
                    neighbor 10.10.0.16 {
                        family inet {
                            labeled-unicast;
                        }
                    }
                }
                group to-isp {
                    type external;
                    export internal;
                    peer-as 200;
                    neighbor 172.16.255.3 {
                        family inet {
                            labeled-unicast;
                        }
                    }
                }
            }
            isis {
                interface all;
            }
            ldp {
                interface ge-0/0/0.1415;
                interface lo0.14;
            }
        }
        policy-options {
            policy-statement internal {
                term internal {
                    from protocol [ isis direct ldp ];
                    then accept;
                }
                term reject {
                    then reject;
                }
            }
        }
        routing-options {
            autonomous-system 100;
        }
    }
    15-P {
        interfaces {
            ge-0/0/0 {
                unit 1516 {
                    vlan-id 1516;
                    family inet {
                        address 10.10.1.6/31;
                    }
                    family iso;
                    family mpls;
                }
            }
            ge-0/0/1 {
                unit 1415 {
                    vlan-id 1415;
                    family inet {
                        address 10.10.1.5/31;
                    }
                    family iso;
                    family mpls;
                }
            }
            lo0 {
                unit 15 {
                    family inet {
                        address 10.10.0.15/32;
                    }
                    family iso {
                        address 49.0100.0100.1000.0015.00;
                    }
                }
            }
        }
        protocols {
            mpls {
                traffic-engineering {
                    bgp-igp;
                }
                interface all;
            }
            isis {
                interface all;
            }
            ldp {
                interface ge-0/0/0.1516;
                interface ge-0/0/1.1415;
                interface lo0.15;
            }
        }
    }
    16-PE {
        interfaces {
            ge-0/0/1 {
                unit 216 {
                    vlan-id 216;
                    family inet {
                        address 192.168.1.3/31;
                    }
                }
                unit 1516 {
                    vlan-id 1516;
                    family inet {
                        address 10.10.1.7/31;
                    }
                    family iso;
                    family mpls;
                }
            }
            lo0 {
                unit 16 {
                    family inet {
                        address 10.10.0.16/32;
                    }
                    family iso {
                        address 49.0100.0100.1000.0016.00;
                    }
                }
            }
        }
        protocols {
            mpls {
                interface all;
            }
            bgp {
                group internal {
                    type internal;
                    local-address 10.10.0.16;
                    neighbor 10.10.0.11 {
                        family inet-vpn {
                            any;
                        }
                    }
                    neighbor 10.10.0.14 {
                        family inet {
                            labeled-unicast {
                                resolve-vpn;
                            }
                        }
                    }
                }
            }
            isis {
                interface ge-0/0/1.1516;
                interface lo0.16;
            }
            ldp {
                interface ge-0/0/1.1516;
                interface lo0.16;
            }
        }
        routing-instances {
            VPN {
                instance-type vrf;
                interface ge-0/0/1.216;
                route-distinguisher 10.10.0.16:1;
                vrf-target target:100:1;
                vrf-table-label;
                protocols {
                    bgp {
                        group vpn {
                            peer-as 1;
                            as-override;
                            neighbor 192.168.1.2;
                        }
                    }
                }
            }
        }
        routing-options {
            autonomous-system 100;
        }
    }
    2-CE {
        interfaces {
            ge-0/0/0 {
                unit 216 {
                    vlan-id 216;
                    family inet {
                        address 192.168.1.2/31;
                    }
                }
            }
            lo0 {
                unit 2 {
                    family inet {
                        address 192.168.0.2/32;
                    }
                }
            }
        }
        protocols {
            bgp {
                group AS100 {
                    type external;
                    export to-bgp;
                    peer-as 100;
                    neighbor 192.168.1.3;
                }
            }
        }
        policy-options {
            policy-statement to-bgp {
                from protocol direct;
                then accept;
            }
        }
        routing-options {
            autonomous-system 1;
        }
    }
    21-C-PE {
        interfaces {
            ge-0/0/0 {
                unit 2122 {
                    vlan-id 2122;
                    family inet {
                        address 172.16.1.0/31;
                    }
                    family iso;
                    family mpls;
                }
            }
            ge-0/0/1 {
                unit 1321 {
                    vlan-id 1321;
                    family inet {
                        address 172.16.255.1/31;
                    }
                    family mpls;
                }
            }
            lo0 {
                unit 21 {
                    family inet {
                        address 172.16.0.21/32;
                    }
                    family iso {
                        address 49.0200.1720.1600.0021.00;
                    }
                }
            }
        }
        protocols {
            mpls {
                interface all;
            }
            bgp {
                group pe-pe {
                    type internal;
                    local-address 172.16.0.21;
                    neighbor 172.16.0.23 {
                        family inet-vpn {
                            any;
                        }
                    }
                }
            }
            isis {
                interface all;
            }
            ldp {
                interface ge-0/0/0.2122;
                interface lo0.21;
            }
        }
        routing-instances {
            vpn-isp {
                instance-type vrf;
                interface ge-0/0/1.1321;
                route-distinguisher 172.16.0.21:100;
                vrf-target target:200:100;
                protocols {
                    bgp {
                        group to-isp {
                            peer-as 100;
                            neighbor 172.16.255.0 {
                                family inet {
                                    labeled-unicast;
                                }
                                as-override;
                            }
                        }
                    }
                    mpls {
                        interface all;
                    }
                }
            }
        }
        routing-options {
            autonomous-system 200;
        }
    }
    22-C-P {
        interfaces {
            ge-0/0/0 {
                unit 2223 {
                    vlan-id 2223;
                    family inet {
                        address 172.16.1.2/31;
                    }
                    family iso;
                    family mpls;
                }
            }
            ge-0/0/1 {
                unit 2122 {
                    vlan-id 2122;
                    family inet {
                        address 172.16.1.1/31;
                    }
                    family iso;
                    family mpls;
                }
            }
            lo0 {
                unit 22 {
                    family inet {
                        address 172.16.0.22/32;
                    }
                    family iso {
                        address 49.0200.1720.1600.0022.00;
                    }
                }
            }
        }
        protocols {
            mpls {
                interface all;
            }
            isis {
                interface all;
            }
            ldp {
                interface ge-0/0/0.2223;
                interface ge-0/0/1.2122;
                interface lo0.22;
            }
        }
    }
    23-C-PE {
        interfaces {
            ge-0/0/1 {
                unit 1423 {
                    vlan-id 1423;
                    family inet {
                        address 172.16.255.3/31;
                    }
                    family mpls;
                }
                unit 2223 {
                    vlan-id 2223;
                    family inet {
                        address 172.16.1.3/31;
                    }
                    family iso;
                    family mpls;
                }
            }
            lo0 {
                unit 23 {
                    family inet {
                        address 172.16.0.23/32;
                    }
                    family iso {
                        address 49.0200.1720.1600.0023.00;
                    }
                }
            }
        }
        protocols {
            mpls {
                interface all;
            }
            bgp {
                group pe-pe {
                    type internal;
                    local-address 172.16.0.23;
                    neighbor 172.16.0.21 {
                        family inet-vpn {
                            any;
                        }
                    }
                }
            }
            isis {
                interface all;
            }
            ldp {
                interface ge-0/0/1.2223;
                interface lo0.23;
            }
        }
        routing-instances {
            vpn-isp {
                instance-type vrf;
                interface ge-0/0/1.1423;
                route-distinguisher 172.16.0.23:100;
                vrf-target target:200:100;
                protocols {
                    bgp {
                        group to-isp {
                            peer-as 100;
                            neighbor 172.16.255.2 {
                                family inet {
                                    labeled-unicast;
                                }
                                as-override;
                            }
                        }
                    }
                    mpls {
                        interface all;
                    }
                }
            }
        }
        routing-options {
            autonomous-system 200;
        }
    }
}
interfaces {
    ge-0/0/0 {
        description "to ge-0/0/1";
        vlan-tagging;
    }
    ge-0/0/1 {
        description "to ge-0/0/0";
        vlan-tagging;
    }
    ge-0/0/2 {
        description "to ge-0/0/3";
        vlan-tagging;
    }
    ge-0/0/3 {
        description "to ge-0/0/2";
        vlan-tagging;
    }
    ge-0/0/4 {
        description "to ge-0/0/5";
        vlan-tagging;
    }
    ge-0/0/5 {
        description "to ge-0/0/4";
        vlan-tagging;
    }
    ge-0/0/6 {
        description "to ge-0/0/7";
        vlan-tagging;
    }
    ge-0/0/7 {
        description "to ge-0/0/6";
        vlan-tagging;
    }
    ge-0/0/8 {
        description "to eth1";
    }
    ge-0/0/9 {
        description "to eth2";
    }
    fxp0 {
        unit 0 {
            family inet {
                address 192.168.83.11/24;
            }
        }
    }
}

Verification

root@MX:1-CE> show route 

inet.0: 5 destinations, 5 routes (5 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

192.168.0.1/32     *[Direct/0] 07:13:24
                    > via lo0.1
192.168.0.2/32     *[BGP/170] 00:05:32, localpref 100
                      AS path: 100 100 I, validation-state: unverified
                    > to 192.168.1.1 via ge-0/0/0.111
192.168.1.0/31     *[Direct/0] 01:40:58
                    > via ge-0/0/0.111
192.168.1.0/32     *[Local/0] 01:40:59
                      Local via ge-0/0/0.111
192.168.1.2/31     *[BGP/170] 00:05:32, localpref 100
                      AS path: 100 I, validation-state: unverified
                    > to 192.168.1.1 via ge-0/0/0.111

root@MX:1-CE> ping 192.168.0.2 source 192.168.0.1 count 3 
PING 192.168.0.2 (192.168.0.2): 56 data bytes
64 bytes from 192.168.0.2: icmp_seq=0 ttl=46 time=21.322 ms
64 bytes from 192.168.0.2: icmp_seq=1 ttl=46 time=13.263 ms
64 bytes from 192.168.0.2: icmp_seq=2 ttl=46 time=14.254 ms

--- 192.168.0.2 ping statistics ---
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max/stddev = 13.263/16.280/21.322/3.588 ms

Label Operation

1. Router 1-CE will send simple traffic:

root@MX:1-CE> show route 192.168.0.2  

inet.0: 5 destinations, 5 routes (5 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

192.168.0.2/32     *[BGP/170] 00:29:05, localpref 100
                      AS path: 100 100 I, validation-state: unverified
                    > to 192.168.1.1 via ge-0/0/0.111

2. Router 11-PE will push L3VPN label, BGP-LU label and LDP trasport label:

root@MX:11-PE> show route 192.168.0.2 

VPN.inet.0: 5 destinations, 6 routes (5 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

192.168.0.2/32     *[BGP/170] 00:30:40, localpref 100, from 10.10.0.16
                      AS path: 1 I, validation-state: unverified
                    > to 10.10.1.1 via ge-0/0/0.1112, Push 16, Push 300112, Push 299856(top)

3. Router 12-P is PHP and will pop LDP label:

root@MX:12-P> show route table mpls.0 label 299856 

mpls.0: 8 destinations, 8 routes (8 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

299856             *[LDP/9] 01:22:43, metric 1
                    > to 10.10.1.3 via ge-0/0/0.1213, Pop      
299856(S=0)        *[LDP/9] 01:22:43, metric 1
                    > to 10.10.1.3 via ge-0/0/0.1213, Pop      

3. Router 13-C-CE is transit router for BGP-LU tunnel and will swap BGP label:

root@MX:13-C-CE> show route label 300112 

mpls.0: 16 destinations, 16 routes (16 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

300112             *[VPN/170] 00:36:20
                    > to 172.16.255.1 via ge-0/0/0.1321, Swap 300096

4. Router 21-C-PE will swap BGP label and push LDP label:

root@MX:21-C-PE> show route label 300096  

vpn-isp.mpls.0: 8 destinations, 8 routes (8 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

300096             *[VPN/170] 00:38:24, metric2 1, from 172.16.0.23
                    > to 172.16.1.1 via ge-0/0/0.2122, Swap 300080, Push 299840(top)

root@MX:21-C-PE> show route 192.168.0.2 

root@MX:21-C-PE> show route 10.10.0.16 

vpn-isp.inet.0: 12 destinations, 13 routes (12 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

10.10.0.16/32      *[BGP/170] 00:39:31, MED 1, localpref 100, from 172.16.0.23
                      AS path: 100 I, validation-state: unverified
                    > to 172.16.1.1 via ge-0/0/0.2122, Push 300080, Push 299840(top)

root@MX:21-C-PE> show route 172.16.0.23 

inet.0: 7 destinations, 7 routes (7 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

172.16.0.23/32     *[IS-IS/18] 02:17:08, metric 20
                    > to 172.16.1.1 via ge-0/0/0.2122

inet.3: 2 destinations, 2 routes (2 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

172.16.0.23/32     *[LDP/9] 02:17:08, metric 1
                    > to 172.16.1.1 via ge-0/0/0.2122, Push 299840

4. Router 22-C-P is PHP and will pop LDP label:

root@MX:22-C-P> show route label 299840 

mpls.0: 8 destinations, 8 routes (8 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

299840             *[LDP/9] 02:19:45, metric 1
                    > to 172.16.1.3 via ge-0/0/0.2223, Pop      
299840(S=0)        *[LDP/9] 02:19:45, metric 1
                    > to 172.16.1.3 via ge-0/0/0.2223, Pop      

5. Router 23-C-PE will swap BGP label:

root@MX:23-C-PE> show route 10.10.0.16 

vpn-isp.inet.0: 12 destinations, 13 routes (12 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

10.10.0.16/32      *[BGP/170] 00:45:36, MED 1, localpref 100
                      AS path: 100 I, validation-state: unverified
                    > to 172.16.255.2 via ge-0/0/1.1423, Push 300064

root@MX:23-C-PE> show route table mpls.0 label 300080 

mpls.0: 12 destinations, 12 routes (12 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

300080             *[VPN/170] 00:56:09
                    > to 172.16.255.2 via ge-0/0/1.1423, Swap 300064

6. Router 14-C-CE will swap LDP label:

root@MX:14-C-CE> show route label 300064 

mpls.0: 16 destinations, 16 routes (16 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

300064             *[VPN/170] 00:57:14
                    > to 10.10.1.5 via ge-0/0/0.1415, Swap 299872

root@MX:14-C-CE> show route 10.10.0.16 

inet.0: 14 destinations, 16 routes (14 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

10.10.0.16/32      *[LDP/9] 01:47:03, metric 1
                    > to 10.10.1.5 via ge-0/0/0.1415, Push 299872
                    [IS-IS/18] 01:47:03, metric 20
                    > to 10.10.1.5 via ge-0/0/0.1415

Note: Here BGP-LU is, in fact, internal LDP tunel. Only one transport label is needed.

7. Router 15-P is PHP router and will pop the LDP label:

root@MX:15-P> show route label 299872 

mpls.0: 8 destinations, 8 routes (8 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

299872             *[LDP/9] 01:56:46, metric 1
                    > to 10.10.1.7 via ge-0/0/0.1516, Pop      
299872(S=0)        *[LDP/9] 01:56:46, metric 1
                    > to 10.10.1.7 via ge-0/0/0.1516, Pop      

8. Router 16-PE will receive the traffic with only L3VPN label. It will pop the label and forward the traffic to de destination:

root@MX:16-PE> show route 192.168.0.2 

VPN.inet.0: 5 destinations, 6 routes (5 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

192.168.0.2/32     *[BGP/170] 02:45:28, localpref 100
                      AS path: 1 I, validation-state: unverified
                    > to 192.168.1.2 via ge-0/0/1.216

Documentation

One thought on “Carrier-of-Carriers with VPN Service Provider as the Customer on a vMX

  1. Pingback: Use Python Script to Create New SecureCRT Sessions from Data File | Silvique Tech Notes

Leave a Reply

Your email address will not be published. Required fields are marked *